1 of 100

Testing Business Critical Features With Behat

2 of 100

Hi, I'm Andrew

Developer Programs Engineer at Pantheon.

  • ataylorme on GitHub and Twitter
  • Some things I enjoy away from the computer
    • Time with friends and family
    • Running and weight training
    • Hiking, off roading and camping
    • Traveling and good food

3 of 100

Maintainer of Websites

4 of 100

Changes Happen Frequently

Client Requests

Security Updates

Bug Fixes

New Features

5 of 100

Who Does QA On A

Staging Environment?

6 of 100

Typical Staging Environment Workflow

New Feature

Dependency Updates

Bug Fix

QA on Staging

Deploy To Production

7 of 100

Acceptance Testing

Also called Integration Testing. Testing the entire application to ensure the defined business requirements are met in an acceptable way.

Acceptance Testing

Unit Testing

Integration Testing

System Testing

8 of 100

Checking Everything

Is Time Consuming

9 of 100

Doing It For Every�Change Is Boring

10 of 100

Sometimes QA�Doesn't Go Well

11 of 100

Find (And Fix)

The Issues

12 of 100

Probably Out Of Time

13 of 100

There's Got To Be

A Better Way

14 of 100

Can We Test More Often Without Doing More Work?

15 of 100

Make The Robots �Do The Work

16 of 100

Automated Testing

Feature: Visibility of the homepage

In order to have confidence that my site is accessible

As a site administrator

I want to verify I can visit the homepage�� Scenario: Verify the homepageWhen I am on the homepageThen the response status code should be 200�

1 scenario (1 passed)�2 steps (2 passed)�0m0.13s (4.16Mb)

17 of 100

So Many Testing�Tools, Why Behat?

18 of 100

Battle Tested

19 of 100

Established Community

20 of 100

Robust Ecosystem

21 of 100

Laravel

22 of 100

Symfony

23 of 100

Drupal

24 of 100

WordPress

25 of 100

Behat API Extension

26 of 100

Browserstack

Extension

27 of 100

Human Readable Tests

@no_auth @privacy

Feature: Visibility of the privacy policy

In order to be in compliance with privacy laws

As a site administrator

I want to verify the privacy policy

Scenario: Verify the privacy policy is accessible from the homepage

When I am on the homepage

And I follow "Privacy Policy"

Then the response status code should be 200

And I should see "This website is the personal, non-commercial website of Andrew Taylor."

And I should see "Contact forms"

And I should see "Cookies"

And I should see "Comments"

And I should see "User accounts"

28 of 100

Backed By PHP

29 of 100

Run Tests Locally

30 of 100

Integrate With Version Control

31 of 100

Test On Pull Requests

32 of 100

Always

Be

Testing

33 of 100

There Is A Lot To Learn

34 of 100

Why Are Tests Written In Gherkin?

@javascript

Feature: Visibility of the latest posts

In order to have confidence that published articles show up

As a content author

I want to see the latest post on the home page

Background:

Given there are users:

| user_login | user_pass | user_email | role |

| test_author | test | test@example.com | author |

And there are posts:

| post_title | post_content | post_status | post_author |

| Author article | The content of my author article | publish | test_author |

And the Pantheon cache has been cleared

Scenario: Verify new post on the front-end and in the admin

Given I am an anonymous user

When I go to "/blog"

Then I should not be logged in

And I should see "Not an Author article"

The text "Author article" was not found anywhere in the text of the current page. (Behat\Mink\Exception\ResponseTextException)

35 of 100

Most importantly, the stories are the result of conversations between the project stakeholders, business analysts, testers and developers. BDD is as much about the interactions between the various people in the project as it is about the outputs of the development process.

Behavior Driven Development (BDD)

Dan North

https://dannorth.net/whats-in-a-story/

BDD is all about communication

36 of 100

We need a

privacy policy

Feature: Visibility of the privacy policy

Scenario: Verify the privacy policy exists

When I am on "/privacy"

Then the response status code should be 200

37 of 100

Why do we need a

privacy policy?

It is a legal requirement and will help avoid potential litigation

@no_auth @privacy

Feature: Visibility of the privacy policy

In order to be in compliance with privacy laws

As a site administrator

I want to verify the privacy policy

Scenario: Verify the privacy policy is accessible from the homepage

When I am on the homepage

And I follow "Privacy Policy"

Then the response status code should be 200

And I should see "This website is the personal, non-commercial website of Andrew Taylor."

And I should see "Contact forms"

And I should see "Cookies"

And I should see "Comments"

And I should see "User accounts"

38 of 100

Ogres Have Layers �Behat Has Layers

39 of 100

Behat is the foundation

Behat

40 of 100

Example ls test

Feature: ls� In order to see the directory structure� As a UNIX user� I need to be able to list the current directory's contents�� Scenario: List 2 files in a directory� Given I am in a directory "test"� And I have a file named "foo"� And I have a file named "bar"� When I run "ls"� Then I should get:� """� bar� foo� """

41 of 100

Backed by PHP

# features/bootstrap/FeatureContext.php

<?php

class FeatureContext extends BehatContext

{

/**

* @Given /^I am in a directory "([^"]*)"$/

*/

public function iAmInADirectory($dir)

{

if ( !file_exists($dir) ) {

mkdir($dir);

}

chdir($dir);

}

}

42 of 100

Mink Focuses

On The Web

Behat

Mink

43 of 100

A browser is the window through which web users interact with web applications and other users.

So, in order to test that our web application behaves correctly, we need a way to simulate this interaction between the browser and the web application in our tests.

Mink

Mink

http://mink.behat.org/en/latest/

An open source browser controller written in PHP

44 of 100

Selenium2 Driver

45 of 100

Headless Chrome Driver

46 of 100

Check a radio button

/**

* @When /^I check the "([^"]*)" radio button$/

*/

public function iCheckTheRadioButton($labelText)

{

$page = $this->getSession()->getPage();

$radioButton = $page->find('named', ['radio', $labelText]);

if ($radioButton) {

$select = $radioButton->getAttribute('name');

$option = $radioButton->getAttribute('value');

$page->selectFieldOption($select, $option);

return;

}

throw new \Exception(

"Radio button with label {$labelText} not found"

);

}

47 of 100

Framework extensions �add focus

Behat

Mink

Framework Extension

48 of 100

Remember WordHat?

49 of 100

General Mink

When I to go "/login.php"��And I fill in

"user_login" with "admin"

And I fill in

"user_pass" with "admin"

And I press "submit"

WordHat

Given I am logged in as an administrator

VS

50 of 100

Framework Focused

Feature: Managing themes

As an administrator I should be able to switch themes

Background:

Given I am logged in as an administrator

Scenario: Activate a theme

When I switch the theme to "twentynineteen"

And I am on the dashboard

Then I should see "Twenty Nineteen" in the "#wp-version a" element

51 of 100

Let's Do This

@ataylorme • Slides: goo.gl/V7QtNw

52 of 100

Download Dependencies

composer require --dev \

behat/behat \

behat/mink-extension \

behat/mink-goutte-driver \

behat/mink-selenium2-driver

53 of 100

Initialize Behat

./vendor/bin/behat --init

54 of 100

Create behat.yml

default:

suites:

default:

contexts:

- FeatureContext

- Behat\MinkExtension\Context\MinkContext

extensions:

Behat\MinkExtension:

base_url: https://ataylorme-wordpress.lndo.site

goutte: ~

selenium2: ~

sessions:

default:

goutte:

guzzle_parameters:

verify: false # Allow self-signed SSL certificates

55 of 100

List Available Steps

./vendor/bin/behat -dl

default | Given /^(?:|I )am on (?:|the )homepage$/

default | When /^(?:|I )go to (?:|the )homepage$/

default | Given /^(?:|I )am on "(?P<page>[^"]+)"$/

default | When /^(?:|I )go to "(?P<page>[^"]+)"$/

default | When /^(?:|I )reload the page$/

default | When /^(?:|I )move backward one page$/

default | When /^(?:|I )move forward one page$/

default | Then /^the response status code should be (?P<code>\d+)$/

...

56 of 100

Write a feature file

./features/visit-homepage.feature

@no_auth

Feature: Visibility of the homepage

In order to have confidence that my site is accessible

As a site administrator

I want to verify I can visit the homepage

Scenario: Verify the homepage

When I am on the homepage

Then the response status code should be 200

57 of 100

Run Behat

./vendor/bin/behat --strict --colors \

--format-settings='{"paths": false}'

Feature: Visibility of the homepage

In order to have confidence that my site is accessible

As a site administrator

I want to verify I can visit the homepage�� Scenario: Verify the homepageWhen I am on the homepageThen the response status code should be 200�

1 scenario (1 passed)2 steps (2 passed)0m0.08s (11.12Mb)

58 of 100

Add WordHat

composer require paulgibbs/behat-wordpress-extension:^3

59 of 100

Update behat.yml

default:

suites:

default:

contexts:

- PaulGibbs\WordpressBehatExtension\Context\WordpressContext

- FeatureContext

- Behat\MinkExtension\Context\MinkContext

- PaulGibbs\WordpressBehatExtension\Context\ContentContext

- PaulGibbs\WordpressBehatExtension\Context\DashboardContext

- PaulGibbs\WordpressBehatExtension\Context\SiteContext

- PaulGibbs\WordpressBehatExtension\Context\UserContext

- PaulGibbs\WordpressBehatExtension\Context\EditPostContext

- PaulGibbs\WordpressBehatExtension\Context\WidgetContext

- PaulGibbs\WordpressBehatExtension\Context\ToolbarContext

extensions:

Behat\MinkExtension:

base_url: https://ataylorme-wordpress.lndo.site

browser_name: chrome

default_session: default

javascript_session: selenium2

sessions:

default:

goutte:

guzzle_parameters:

verify: false # Allow self-signed SSL certificates

selenium2:

selenium2: ~

PaulGibbs\WordpressBehatExtension:

default_driver: wpcli

path: /Users/andrewtaylor/Development/ataylorme-wordpress/web/wp

site_url: https://ataylorme-wordpress.lndo.site/wp/

users:

-

roles:

- administrator

username: admin

password: admin

database:

restore_after_test: false

60 of 100

extensions:

Behat\MinkExtension:

base_url: https://ataylorme-wordpress.lndo.site

browser_name: chrome

default_session: default

javascript_session: selenium2

sessions:

default:

goutte:

guzzle_parameters:

verify: false # Allow self-signed SSL certificates

selenium2:

selenium2: ~

PaulGibbs\WordpressBehatExtension:

default_driver: wpcli

path: /Users/andrewtaylor/Development/ataylorme-wordpress/web/wp

site_url: https://ataylorme-wordpress.lndo.site/wp/

users:

-

roles:

- administrator

username: admin

password: admin

database:

restore_after_test: false

Site front-end URL

Path to WordPress and

Back-end URL

Admin username and password

61 of 100

List Available Steps

./vendor/bin/behat -dl

default | When I activate the :name plugin

default | When I deactivate the :name plugin

default | When I switch the theme to :name

default | Given /^(?:there are|there is a) users?:/

default | Given /^(?:I am|they are) an anonymous user/

default | Given /^(?:I am|they are) logged in as an? (.+)$/

default | Given /^I am on the edit screen for "(?P<title>[^"]*)"$/

default | When /^I change the title to "(?P<title>[^"]*)"$/

default | When /^I publish the (post|changes?)$/

...

62 of 100

Write a WordPress enabled feature file

./features/blogname.feature

@auth

Feature: Change blogname

As a maintainer of the site

I want to be able to change basic settings

So that I have control over my site

Scenario: Saving blogname

Given I am logged in as an administrator

Given I am on the dashboard

When I go to the "Settings > General" menu

And I fill in "blogname" with "Testing with Behat Rocks!"

And I press "submit"

Then I should see "Settings saved."

63 of 100

Run Behat

./vendor/bin/behat --strict --colors \

--format-settings='{"paths": false}'

@auth

Feature: Change blogname

As a maintainer of the site

I want to be able to change basic settings

So that I have control over my site

Scenario: Saving blogname

Given I am logged in as an administrator

Given I am on the dashboard

When I go to the "Settings > General" menu

And I fill in "blogname" with "Testing with Behat Rocks!"

And I press "submit"

Then I should see "Settings saved."

64 of 100

Simple Behat Example Repository

65 of 100

What Do We Do With This?

66 of 100

Avoid 3 AM Phone Calls

67 of 100

Privacy Policy Example

As seen on my personal site https://ataylor.me/privacy-policy

68 of 100

Write a Test With Built-In Steps

./features/privacy-policy.feature

@no_auth @privacy

Feature: Visibility of the privacy policy

In order to be in compliance with privacy laws

As a site administrator

I want to verify the privacy policy

Scenario: Verify the privacy policy is accessible from the homepage

When I am on the homepage

And I follow "Privacy Policy"

Then the response status code should be 200

And I should see "This website is the personal, non-commercial website of Andrew Taylor."

And I should see "Contact forms"

And I should see "Cookies"

And I should see "Comments"

And I should see "User accounts"

69 of 100

Contact Form Example

As seen on my personal site https://ataylor.me/contact

70 of 100

  • Fill out first name
  • Fill out last name
  • Fill out email
  • Fill out the comment/message
  • Submit the form
  • Verify the form submission

Contact Form QA Process

71 of 100

Testing Manually Is Tedious

72 of 100

Write a Test From The Manual QA Steps

  • Visit the contact page �without being logged in
  • Fill out first name, last name, �email and message
  • Submit the form
  • Verify the confirmation message

Background:

Given I am on the contact page

Given I am an anonymous user

Scenario: Confirm contact form submissions

When I set the contact form first name to "Andrew"

And I set the contact form last name to "Taylor"

And I set the contact form email to "andrew@pantheon.io"

And I set the contact form message to "Testing the contact form with Behat!"

When I submit the contact form

Then I should see "Thanks for reaching out! I will be in touch with you shortly."

73 of 100

Behat Isn't Magic

74 of 100

We Need To Do�Some Work

75 of 100

Generic Mink

When I fill in

"wpforms-117-field_0" with "Andrew"

And I fill in

"wpforms-117-field_0-last" with "Taylor"

And I fill in "wpforms-117-field_1"

with "andrew@pantheon.io"

Custom Definitions

When I set the contact form first name to "Andrew"

And I set the contact form last name to "Taylor"

And I set the contact form email to "andrew@pantheon.io"

VS

76 of 100

Contexts

Contexts are PHP Objects to encapsulate a set of functionality

<?php

declare(strict_types=1);

namespace ataylorme\WordHatHelpers\Contexts;

use ataylorme\WordHatHelpers\Contexts\RawWPContext;

use PaulGibbs\WordpressBehatExtension\Context\Traits\UserAwareContextTrait;

use PaulGibbs\WordpressBehatExtension\PageObject\LoginPage;

use Behat\Mink\Exception\ExpectationException;

use RuntimeException;

use FailAid\Context\FailureContext;

/**

* Define WordPress specific features from the specific context.

*/

class WordPressContext extends RawWPContext�{� use UserAwareContextTrait;

77 of 100

Step Definitions

Step definitions are methods of the context classes

/**

* Sets the contact form first name field

* Example: When I set the contact form first name to "Andrew"

*

* @When I set the contact form first name to :first_name

*

* @param string $first_name

*/

public function fillContactFormFirstNameField(string $first_name)

{

$this->getSession()->getPage()->fillField('wpforms-117-field_0', $first_name);

}

78 of 100

RawContexts

Contexts are meant to be extended. To avoid having steps defined multiple times keep helpers in their own Context and extend it to create other contexts

/**

* Log into WordPress as an admin

*

* @throws \RuntimeException

*

* @return void

*/

protected function loginAsWordPressAdmin()

{

// Get the admin user

$found_user = $this->getAdminUser();

// Stash the current URL to redirect to

$previous_url = $this->getSession()->getCurrentUrl();

$this->logIn(

$found_user['username'],

$found_user['password'],

$previous_url

);

FailureContext::addState('source_url', $previous_url);

FailureContext::addState('username', $found_user['username']);

}

79 of 100

Human

Readable�Gherkin

@javascript @contact @no_auth

Feature: Verify the contact form

As the site owner who likes to hear from users

I want the contact form to submit properly

Background:

Given I am on the contact page

Given I am an anonymous user

Scenario: Confirm contact form submissions

When I set the contact form first name to "Andrew"

And I set the contact form last name to "Taylor"

And I set the contact form email to "andrew@pantheon.io"

And I set the contact form message to "Testing the contact form with Behat!"

When I submit the contact form

Then I should see "Thanks for reaching out! I will be in touch with you shortly."

80 of 100

Backed by PHP

/**

* Submit the contact form

* Example: When I submit the contact form

*

* @When I submit the contact form

*/

public function submitContactForm()

{

$session = $this->getSession();

$page = $session->getPage();

$page->pressButton('wpforms-submit-117');

// Looks for the '#wpforms-confirmation-117' element, giving up after 5 seconds.

$session->wait( 5000, "document.getElementById('wpforms-confirmation-117')" );

}

81 of 100

Look Under The Hood

82 of 100

Mink Source Code

/**

* Fills in form field with specified id|name|label|value

* Example: When I fill in "username" with: "bwayne"

* Example: And I fill in "bwayne" for "username"

* @When /^(?:|I )fill in "(?P<field>(?:[^"]|\\")*)" with "(?P<value>(?:[^"]|\\")*)"$/

* @When /^(?:|I )fill in "(?P<field>(?:[^"]|\\")*)" with:$/

* @When /^(?:|I )fill in "(?P<value>(?:[^"]|\\")*)" for "(?P<field>(?:[^"]|\\")*)"$/

*/

public function fillField($field, $value)

{

$field = $this->fixStepArgument($field);

$value = $this->fixStepArgument($value);

$this->getSession()->getPage()->fillField($field, $value);

}

83 of 100

Modify To Fit �Your Needs

/**

* Sets the contact form last name field

* Example: I set the contact form last name to "Taylor"

*

* @When I set the contact form last name to "Taylor"

*

* @param string $last_name

*/

public function fillGiveLastNameField(string $last_name)

{

$this->getSession()->getPage()->fillField(

'wpforms-117-field_0', $last_name

);

}

84 of 100

WordHat �Source Code

public function getContentFromTitle(string $title, string $post_type = ''): array

{

$post = $this->getDriver()->content->get(

$title, ['by' => 'title', 'post_type' => $post_type]

);

return array(

'id' => (int) $post->ID,

'slug' => $post->post_name,

'url' => $post->url,

);

}

85 of 100

Modify To Fit �Your Needs

/**

* Go to a specific post by ID

* Example: Given I am viewing the post with an ID of 1234

* @Given I am viewing the post with an ID of :post_id

* @param string $post_id

* @throws \UnexpectedValueException

*/

public function iAmViewingPostID($post_id)

{

$post = $this->getDriver()->content->get(

$post_id, ['by' => 'ID']

);

$this->visitPath($post->url);

}

86 of 100

Iterate Until Tests Are Complete

@javascript @contact @no_auth

Feature: Verify the contact form

As the site owner who likes to hear from users

I want the contact form to submit properly

Background:

Given I am on the contact page

Given I am an anonymous user

Scenario: Confirm contact form submissions

When I set the contact form first name to "Andrew"

And I set the contact form last name to "Taylor"

And I set the contact form email to "andrew@pantheon.io"

And I set the contact form message to "Testing the contact form with Behat!"

When I submit the contact form

Then I should see "Thanks for reaching out! I will be in touch with you shortly."

87 of 100

Test Locally

88 of 100

Test On Pull Requests

89 of 100

Benefits of Adopting Automation

90 of 100

Reduced Overhead

@ataylorme • Slides: goo.gl/oXqt1a

91 of 100

Consistency

@ataylorme • Slides: goo.gl/oXqt1a

92 of 100

Risk Mitigation

@ataylorme • Slides: goo.gl/oXqt1a

93 of 100

Confidence

@ataylorme • Slides: goo.gl/oXqt1a

94 of 100

Automation is �a State of Mind

95 of 100

Resources

@ataylorme • Slides: goo.gl/oXqt1a

96 of 100

Simple Behat Example Repo

}

97 of 100

ataylor.me Repository

98 of 100

Resource Links

99 of 100

Framework Extensions

100 of 100

Questions?Slides: bit.ly/ataylorme-behat-testing-slides@ataylorme

@ataylorme • Slides: goo.gl/oXqt1a