1 of 53

5 Levels of Automated Testing�of an Ecommerce Project - There Is No Silver Bullet

Petr Heinz

Lead Developer & Architect

2 of 53

The lock works well, but...

3 of 53

Testing

4 of 53

Goal: Assert that the application works as intended

5 of 53

You have to define �the expected behavior

6 of 53

Who does the testing?

Automated tests

Manual testing

petr-heinz@localhost:~/projects/shopsys-framework$ ./phing tests-unit

�Shopsys Framework > tests-unit:��PHPUnit 7.1.5 by Sebastian Bergmann and contributors.��................................................. 50 / 242 ( 21%)

................................................. 100 / 242 ( 41%)

................................................. 150 / 242 ( 62%)

................................................. 200 / 242 ( 83%)

......................................... 242 / 242 (100%)��Time: 134 ms, Memory: 12.00MB��OK (242 tests, 420 assertions)��BUILD FINISHED��Total time: 8.6212 seconds�

​

7 of 53

Different approaches�to automated testing

8 of 53

Unit tests

9 of 53

Test the smallest�amount of functionality�(i.e. a method)

10 of 53

Lightning fast execution

11 of 53

The tested code must be designed well to be testable

12 of 53

Great for testing isolated well-defined components

13 of 53

14 of 53

Functional tests

15 of 53

Test the interoperability�of classes

16 of 53

Can use services from�the DI container

17 of 53

May access a database or other storages

18 of 53

I recommend having�testing data fixtures

19 of 53

20 of 53

HTTP smoke tests

21 of 53

Checking just the HTTP status codes for all routes

22 of 53

“Is my app 200 OK?”

23 of 53

Almost maintenance free

24 of 53

Tests all new routes automatically

25 of 53

Prevents running into�a 500 error on�an unrelated page

26 of 53

Requires isolation of the database and other storages

27 of 53

shopsys / http-smoke-testing

a ready-to-use testing package for Symfony applications

28 of 53

Acceptance tests

29 of 53

Test end-to-end scenarios

30 of 53

Simulate real browser interaction

31 of 53

Excellent for business critical scenarios

32 of 53

Take a while to execute

33 of 53

We use Codeception and Selenium (via Docker)

34 of 53

35 of 53

Performance tests

36 of 53

Developing on a small dataset is dangerous

37 of 53

It can hide performance issues of new changes

38 of 53

Run the code on a larger dataset before deploying

39 of 53

Keep track of the changes�of response times

40 of 53

Try Gatling for load tests

41 of 53

You don’t want to run�them on your localhost

42 of 53

I ran out of memes…

...so no demo for you

43 of 53

​

Unit

Functional

Smoke

Acceptance

Performance

Subject of testing

results of method calls

interoperability of services

HTTP status codes of requests

end-to-end scenarios

response times with a large dataset

Ideal application

isolated well-defined components

code that’s hard to unit test

protection from unhandled exceptions

business critical scenarios

preventing collapse on production data load

Length of execution

very fast

fast

decent

slow

very slow

Difficulty of maintenance

moderate

moderate

very easy

hard

depends

44 of 53

Now you can be pretty sure about the behavior...

45 of 53

...but can we do more?

46 of 53

Coding standards

prevent problems with cooperation of devs

47 of 53

Static analysis - PHPStan

find errors without running the code

48 of 53

Architecture sniffs

applying CS to check your architecture

49 of 53

Automate it!

50 of 53

Automate it!

51 of 53

Automate it!

52 of 53

There is no silver bullet

53 of 53

Petr Heinz

Lead Developer & Architect

github.com/shopsys/shopsys

petr.heinz@shopsys.com

linkedin.com/in/petr-heinz

To learn more and check the slides go to�https://shopsys.com/u/tests

​

​

Wanted: Quality Assurance Developer�see https://jobs.shopsys.cz/ for details

Icons made by Freepik from www.flaticon.com.