1 of 26

Patrick Poulin

�API Fortress

a Sauce Labs company

1

2 of 26

Keys to API Testing

2

3 of 26

API FORTRESS

Patrick Poulin

DIRECTOR OF PRODUCT

API FORTRESS

Who are we?

API Fortress has been in the API testing and monitoring space since 2014 when we started building the platform’s SaaS version

Now we have thousands of users spread all over the world, and offer both a SaaS and self-hosted version

3

TEXT & IMAGE

4 of 26

Fun API facts

4

5 of 26

API FUN FACTS

Of security fails

Of website traffic

83% of all web traffic in 2018 was API traffic

Through 2022, at least 95% of cloud security failures will be the customer’s fault

5

95%

83%

State of the Internet 2018

Jay Heiser, VP Analyst

6 of 26

FUNCTIONAL TESTING = SECURITY

India Public Utilities

Twitter

United States Postal Service

3 recent vulnerabilities

Exposed the phone numbers of 17 million accounts

1.1 billion customer identities exposed

Exposed the data of 60 million users

6

7 of 26

RECENT VULNERABILITIES

7

Exposed the phone numbers of 17 million accounts

Twitter’s breach may have been due to a simple design flaw. During registration, you could connect with people in your Contacts.

The problem was a lack of rate limiting. This led to bad actor countries guessing phone numbers and connecting people’s personal numbers with their Twitter accounts. Not everyone wants their Twitter identities tied to them personally and professionally for reasons that may include blackmail.

Twitter

8 of 26

RECENT VULNERABILITIES

8

1.1 Billion customer identities exposed

In India, there was a huge breach with a company named Indane, a large utility company in India. When creating an account, they would verify your personal information using the Aadhar API.

The problem was that it was publicly accessible and had no rate limiting. This meant that anyone could guess millions of “social security” numbers, and if they were right, they got someone’s personal information. Aadhar exposed 1.1 billion customers as part of the breach.

India & Public Utilities

9 of 26

RECENT VULNERABILITIES

9

Exposed data of 60 million users

The USPS had 60 million customers with private information potentially exposed due to their package tracker API.

All a bad actor had to do was when making the API call, search for a package. Then the hacker would replace tracking numbers with an asterisk (wildcard), causing it to return all customer data.

US Postal Service

10 of 26

What is continuous quality?

10

11 of 26

CONTINUOUS QUALITY

11

MONITORING

PERFORMANCE

TESTING

AUTOMATED

CI/CD

FUNCTIONAL + INTEGRATION

TESTING

12 of 26

SHOW ME HOW

One Successful Work Flow

API developers write the first version of tests in staging that works across teams. Preferably from a spec file.�
They hand off to a professional QA team that writes more robust integration tests.

12

13 of 26

REMOVE THE SILOS

13

14 of 26

START WITH A SPEC

Open API initiative

Swagger

Postman

14

15 of 26

FUNCTIONAL TEST

15

[

{

quantity: 5,

color: ["white","red"],

price: 29.99,

imageURL: http://apif.com/baseball_cap.jpg,

description: "Boston Red Sox Baseball Cap",

id: 1,

category: "head",

},

{

quantity: 7,

color: ["blue","yellow","red"],

price: 39.99,

imageURL: http://apif.com/long_sleev_shirt.jpg,

description: "A wonderful long sleeve shirt",

id: 2,

category: "body",

},

{

quantity: 50,

color: ["red","gray"],

price: 49.99,

imageURL: http://apif.com/earmuffs.jpg,

description: "Keep those ears warm in the winter!",

id: 3,

category: "ears",

},

]

http://demoapi.apifortress.com/api/retail/product?q=red

16 of 26

REPRODUCE USER FLOWS

16

17 of 26

END-TO-END / INTEGRATION TEST

17

[

{

quantity: 50,

color: ["red","gray"],

price: 49.99,

imageURL: http://apif.com/earmuffs.jpg,

description: "Keep those ears warm in the winter!",

id: 3,

category: "ears",

},

]

http://demoapi.apifortress.com/api/retail/product?q=red

http://demoapi.apifortress.com/api/retail/product/3

18 of 26

Who Owns Functional Uptime?

18

19 of 26

MONITORING PLATFORMS

19

20 of 26

PLATFORM SHORTCOMINGS

Synthetic Testing�A euphemism for minimal API testing capabilities��
User Profile�People in charge of the monitors aren’t testing professionals.

Using a Different Platform�Tests written by stakeholders with little/no domain knowledge�
Test Intelligence�Tests must capture real consumer flows or user behavior

20

21 of 26

Show Me

21

22 of 26

Best Practices

22

23 of 26

BEST PRACTICES

Test everything

Test objects & data

Monitor production using function tests

23

24 of 26

Variabilize for test reuse

Confirm business logic and datatypes

Use dynamic data

Integration / End-to-end test

Confirm response matches request

BEST PRACTICES

24

z

25 of 26

Thank you

25

PATRICK.POULIN@SAUCELABS.COM

Patrick Poulin

Director of Product

1 of 26

2 of 26

3 of 26

4 of 26

5 of 26

6 of 26

7 of 26

8 of 26

9 of 26

10 of 26

11 of 26

12 of 26

13 of 26

14 of 26

15 of 26

16 of 26

17 of 26

18 of 26

19 of 26

20 of 26

21 of 26

22 of 26

23 of 26

24 of 26

25 of 26

26 of 26