PHPSerbia 2017

Michael Bodnarchuk

BEYOND TESTING

Hi, I am Michael @davert

and I have stickers!

​

Codeception

​

http://codeception.com

• Full stack testing framework
• Based on top of PHPUnit
• Have modules that provide common steps for testing
• Designed for Growth: PageObjects, Helpers, Extensions
• BDD-ready with Gherkin

Testing

Better way to do it

Ideal Test

• Condition (Given)
• Action (When)
• Assertion (Then)
• Action
• Assertion

DON'T DO IT THIS WAY

move to configuration

use traits

IMPLEMENT YOUR OWN ASSERTIONS

custom assertions make code more readable

SIMPLIFY IT!



How to Write Unit / Integration Test

• Separate configuration from a test
• Do not use hierarchy for testcases (use traits)
• Separate test code from support code
• Make test simple and verbose

Unit vs Integration Tests

• Unit Tests ⇒ Domain Logic
• Integration Tests ⇒ Infrastructure (+ Domain)

Questions to be Asked

• Does our architecture allow us to write unit tests?
• Is unit testing enough for us?
• Does it check User Interface / UX?

Testing Levels

• Unit
• Integration
• Functional
• UI (Acceptance)

Know Pros and Cons

Specification

Detail

TDD Process

• Write most outer API failing test
• Implement a failing test for the domain
• Implement domain logic
• Implement infrastructure
• Proceed until tests pass

Every app can be tested!

• Choose the testing level you are comfortable with
• Don’t bother about speed of tests
• Don’t wait for refactoring to write tests
• Don’t take too much time into testing. Create business value
• Do slow but constant refactoring

What is Hard to Test

• Asynchronous stuff
• Remote Services
• Real Data

Beyond Testing

what they never tell you about testing

Data Management

for integration / functional / acceptance

Accessing Database

• Using internal database connection (ORM)
• HINT: Rollback transaction in the end of a test
• External database connection
• External API
• HINT: Use REST API for data in acceptance tests

Data Isolation Strategies

• Create/delete data needed only for a single test
• Recreate the database between tests
• HINT: Use a container to restart database after each test
• Create non-intersecting data

Defining Test Data With

• Database Dumps
• Fixtures (nelmio/alice)
• Factories (thephpleague/factory-muffin)

FACTORY_MUFFIN IN REAL LIFE

$fm->define(User::class)->setDefinitions([

'name' => Faker::name(),

'email' => Faker::email(),

'body' => Faker::text(),

// generate a profile and return its Id

'profile_id' => 'factory|Profile'

);

​

Testing APIs

REST APIs

How To Test JSON Responses

• by string comparison of response body
• by data inclusion
• by structure inclusion
• by schema (Swagger, JSON-Schema)

GET /tickets/3

{� "ticket": {� "id": 3,� "from": "web",� "description": "Lorem ipsum...",� "priority": "important",� "priority_value": 1,� "report": {� "user_agent": "Mozilla...",� "url": "/tasks", � "window": "1280x525",� "resolution": "1600x1200"� },� "reporter_info": {� "name": "davert",� "email": "davert@codeception.com",� }� "created_at": "2016-08-21T20:16:37Z",� "updated_at": "2016-09-11T15:13:47Z" � }�}

How we test in Codeception: Data Inclusion

$I->wantTo('get a ticket by its id');

$I->sendGET('/api/tickets/3');

$I->seeResponseCodeIs(HttpCode::OK); // 200

$I->seeResponseIsJson();

// check data in response

$I->seeResponseContainsJson([

'ticket' =>

'id' => 3,

'from' => 'web'

'report' => [

'url' => '/tasks'

]]);

​

How we test in Codeception: Structure Inclusion

// check the structure of response

$I->seeResponseMatchesJsonType([

'ticket' => [

'id' => 'integer',

'description' => 'string|null',

'priority' => 'string',

'created_at' => 'string:date',

'reporter_info' => [

'email' => 'string:email'

]]]);

​

How we test in Codeception: Schema Check

$I->sendGET('/api/tickets/3');

// use custom helper

$I->seeResponseMatchesSwaggerSchema('ticket');

​

​

• extend Codeception in Helper\Api
• implement seeResponseMatchesSwaggerSchema

Mocking APIs

• Record and reuse API responses (php-vcr)
• HINT: Use PHP-VCR library github.com/php-vcr/php-vcr
• Mock and Stub APIs using HTTP server
• HINT: Use Phiremock library github.com/mcustiel/phiremock
• or Mountebank http://www.mbtest.org

​

WEB UI Testing

why it is important for developers

How Developers can Improve Acceptance Testing

• Communicate with QA team
• Suggest better locators

HINT: add locator classes or data-attributes to HTML elements

• Use the same language (PHP)
• Solve data management issues (via REST API)
• Suggest better tools

Choose The Right Tool

• Selenium + Browsers
• HINT: use Docker for headless browsers
• PhantomJS (headless browser, unmaintained)
• Cloud Testing Service (SauceLabs, BrowserStack)
• HINT: make sure you use them in your region
• Browser emulation via HTTP client

Atomic Acceptance Tests with Data Management

Feature: CRUD for Post

Scenario:

When I create a post

And I open a post

And I edit a post

Then I see it has changed

Then I delete a post

Feature: CRUD for Post

Scenario: create a post

Scenario: view post

Scenario: edit a post

Scenario: delete a post

​

POST is created via API for each test which requires it

One Post for everything :(

Parallel Testing

one day tests started to be slow

PARALLEL TESTING

• Tests separation
• Processes isolation

SET IT UP

• Manually
• Use multiple nodes
• Run several concurrent processes
• Ensure data is not interfering
• Using Docker
• Everything is isolated by design

HOW TO RUN PARALLEL TESTS WITH DOCKER

• Pack application into container
• Create script for running the tests
• Use Jenkins Matrix to setup concurrent builds

What’s inside the container?

• Prepared databases
• ./runtests.sh starts all required services (nginx, mysql, selenium, redis, etc)
• Container stops when tests are finished
• No supervisors: we can execute one process per run

#!/bin/sh

echo "Starting Services...."

service elasticsearch start > /dev/null 2>&1

service nginx start > /dev/null 2>&1

service php5-fpm start > /dev/null 2>&1

service mysql start > /dev/null 2>&1

phantomjs --webdriver=4444 > /dev/null 2>&1 &

mailcatcher -f > /dev/null 2>&1 &

​

echo "Running tests"

cd /project/$1 # switch to application

codecept run $2 # run tests from specific suite

​

​

docker run -it -v $WORKSPACE:/project app ./runtests.sh $SUITE

Conclusions

• Testing is not just about unit tests
• Understand Pros and Cons of testing levels
• Use proper data management strategy
• Build a proper test infrastructure for your CI

​

Constantly improve code by refactoring!

It is safe to do this with tests.

TIME FOR QUESTIONS!

• My name is Michael Bodnarchuk
• Twitter: @davert
• GitHub: DavertMik
• Projects: Codeception, CodeceptJS, Robo Task Runner