1 of 25

1

2 of 25

Hello!�Nice to meet you!

2

I am Christina Thalayasingam
Test Engineering Manager at Northwestern Mutual
I have 7+ years experience in Test Automation and Non-Functional Testing.
I’ve spoken at various Technical Test Meetups and at Selenium Conf ‘16, London, Selenium Conf ‘17, Austin, Texas, Colombo Test Automation Conference 2017 Selenium Conf ‘18, Chicago, Perf Guild 2019, Quality & BE Summit 2019 & 2021, Selenium Conf ‘19, London, Test Leadership Congress 2020, PNSQC 2020, Selenium pre-Conf 2020, QA ATL 2020, Open Quality Conference 2020, Testflix 2020 and ConTEST 2021

Find me at:

@ChristinaThalay

3 of 25

3

CONTINUOUS API TESTING

4 of 25

4

5 of 25

5

6 of 25

DRIVING POSTMAN TO MEET NEWMAN FOR REST

“The Challenges Faced, And Decisions Made.”

Christina Thalayasingam

Test Engineering Manager

7 of 25

AGENDA

Introduction to API Testing
Comparing API Testing Tools
Challenges faced
Use of Newman for CI
Demos

7

8 of 25

API�TESTING

API

"API" stands for "Application Programming Interface."

API is the layer that sits between the user interface (UI) and the database layer in the creation of software applications (apps). APIs allow data and communication to flow from one software system to another. ��API Testing��API testing is a type of software testing that examines APIs directly, including their functionality, dependability, performance, and security. API testing, which is part of integration testing, tests the logic of the build architecture in a short amount of time.

8

9 of 25

TYPES OF API TESTING

Validation Testing
Functional testing
Load testing
Runtime error detection
UI testing
Security testing
Penetration Testing
Fuzz-testing

9

Validation Testing

Validation testing occurs among the final steps and plays an essential role in the development process. It verifies the aspects of product, behavior, and efficiency. In other words, validation testing can be seen as an assurance of the correct development.

Functional testing

Includes testing particular functions in the codebase. These features are the representation of specific scenarios to make sure the API functions are handled well within the planned parameters.

UI testing

UI testing is defined as a test of the user interface for the API and other integral parts. UI testing focuses more on the interface which ties into the API rather than the API testing itself. Although UI testing is not a specific test of API in terms of codebase, this technique still provides an overview of the health, usability, and efficiency of the app’s front and back ends.

Security testing

This practice ensures the API implementation is secure from external threats. Security testing also includes additional steps such as validation of encryption methodologies, and of the design of the API access control. It also includes user rights management and authorization validation.

Load testing

Load testing generally occurs after a specific unit or the whole codebase has been completed. This technique checks if the theoretical solutions work as planned. Load testing monitors the app's performance at both normal and peak conditions.

Runtime and error detection

This testing type is related to the actual running of the API — particularly with the universal results of utilizing the API codebase. This technique focuses on one of the below aspects: monitoring, execution errors, resource leaks, or error detection.

Penetration testing

Penetration testing is considered the second test in the auditing process. In this type, users with limited API knowledge will try to assess the threat vector from an outside perspective, which is about functions, resources, processes, or aim to the entire API and its components.

Fuzz testing

Fuzz testing is another step in the security audit process. In fuzz testing, a vast amount of random data (referred to as "noise" or "fuzz") will be input into the system to detect any forced crashes or negative behaviors. This technique tests the API’s limits to prepare for the "worst-case scenarios."

10 of 25

WHAT DID WE LOOK FOR IN THE TOOL WE PICKED

Minimum coding skills required
Ability to work with MAuth Proxy and similar Authentication Methods

- OAuth Tokens

- AWS Signature

Ability to share the scripts
Incorporate Continuous Testing

10

Validation Testing

Validation testing occurs among the final steps and plays an essential role in the development process. It verifies the aspects of product, behavior, and efficiency. In other words, validation testing can be seen as an assurance of the correct development.

Functional testing

Includes testing particular functions in the codebase. These features are the representation of specific scenarios to make sure the API functions are handled well within the planned parameters.

UI testing

UI testing is defined as a test of the user interface for the API and other integral parts. UI testing focuses more on the interface which ties into the API rather than the API testing itself. Although UI testing is not a specific test of API in terms of codebase, this technique still provides an overview of the health, usability, and efficiency of the app’s front and back ends.

Security testing

This practice ensures the API implementation is secure from external threats. Security testing also includes additional steps such as validation of encryption methodologies, and of the design of the API access control. It also includes user rights management and authorization validation.

Load testing

Load testing generally occurs after a specific unit or the whole codebase has been completed. This technique checks if the theoretical solutions work as planned. Load testing monitors the app's performance at both normal and peak conditions.

Runtime and error detection

This testing type is related to the actual running of the API — particularly with the universal results of utilizing the API codebase. This technique focuses on one of the below aspects: monitoring, execution errors, resource leaks, or error detection.

Penetration testing

Penetration testing is considered the second test in the auditing process. In this type, users with limited API knowledge will try to assess the threat vector from an outside perspective, which is about functions, resources, processes, or aim to the entire API and its components.

Fuzz testing

Fuzz testing is another step in the security audit process. In fuzz testing, a vast amount of random data (referred to as "noise" or "fuzz") will be input into the system to detect any forced crashes or negative behaviors. This technique tests the API’s limits to prepare for the "worst-case scenarios."

11 of 25

IMPORTANT NOTE

This presentation does not carry details of how the process is conducted in any specific organization – but it is a compilation of all the important practices that I have followed in the various teams I have worked with.

11

12 of 25

TOOLS THAT WERE COMPARED

COMPARISON CATEGORIES	POSTMAN	INSOMNIA	PAW
Authentication	No JWT Generation and AuthO	No JWT Generation and AuthO	No NTLM and others were working via Extensions
Environment	Environments and their variables are segregated per project	Environments can be segregated by workspace – no Variable Grouping	Environments and their variables are segregated per workspace but no color hints
Assertions/Automated Testing	Yes	No	No
Save response as “Example”	Yes	No	No
Plug-ins/Extensions	Does not allow users to create plugin	can create plugins	can create plugins
GraphQL	Yes, includes schema fetching.	Yes, includes schema fetching.	No

13 of 25

INTRODUCING�NEWMAN

13

A powerful command-line collection runner for Postman.

14 of 25

LET US GET STARTED

Make sure to install Node.js
Install Newman from npm globally on your system� npm install -g newman
Run any collection file from your file system.� newman run mycollection.json

OR

Pass a collection as a URL by sharing it. �newman run https://www.postman.com/collections/xxxxxxxxxxxxxxxxxxxx -e dev.json

14

Validation Testing

Validation testing occurs among the final steps and plays an essential role in the development process. It verifies the aspects of product, behavior, and efficiency. In other words, validation testing can be seen as an assurance of the correct development.

Functional testing

Includes testing particular functions in the codebase. These features are the representation of specific scenarios to make sure the API functions are handled well within the planned parameters.

UI testing

UI testing is defined as a test of the user interface for the API and other integral parts. UI testing focuses more on the interface which ties into the API rather than the API testing itself. Although UI testing is not a specific test of API in terms of codebase, this technique still provides an overview of the health, usability, and efficiency of the app’s front and back ends.

Security testing

This practice ensures the API implementation is secure from external threats. Security testing also includes additional steps such as validation of encryption methodologies, and of the design of the API access control. It also includes user rights management and authorization validation.

Load testing

Load testing generally occurs after a specific unit or the whole codebase has been completed. This technique checks if the theoretical solutions work as planned. Load testing monitors the app's performance at both normal and peak conditions.

Runtime and error detection

This testing type is related to the actual running of the API — particularly with the universal results of utilizing the API codebase. This technique focuses on one of the below aspects: monitoring, execution errors, resource leaks, or error detection.

Penetration testing

Penetration testing is considered the second test in the auditing process. In this type, users with limited API knowledge will try to assess the threat vector from an outside perspective, which is about functions, resources, processes, or aim to the entire API and its components.

Fuzz testing

Fuzz testing is another step in the security audit process. In fuzz testing, a vast amount of random data (referred to as "noise" or "fuzz") will be input into the system to detect any forced crashes or negative behaviors. This technique tests the API’s limits to prepare for the "worst-case scenarios."

15 of 25

HELPFUL FLAGS FOR CUSTOMIZATION

COMPARISON CATEGORIES	POSTMAN
-h, --help	Output usage information
-v, --version	Output the version number
--folder [folderName]	Specify a single folder to run from a collection.
-e, --environment [file\|URL]	Specify a Postman environment as a JSON [file]
-d, --iteration-data [file]	Specify a data file to use either json or csv
-g, --globals [file]	Specify a Postman globals file as JSON [file]
-n, --iteration-count [number]	Define the number of iterations to run
--delay-request [number]	Specify a delay (in ms) between requests [number]
--timeout-request [number]	Specify a request timeout (in ms) for a request
--bail	Stops the runner when a test case fails
-k, --insecure	Disable strict ssl
-x, --suppress-exit-code	Continue running tests even after a failure, but exit with code=0

16 of 25

TIPS TO KEEP IN MIND WHEN USING NEWMAN

16

Iterations to run the collection use the -n option�newman run collection.json -n 5
To use different set of data, such as variables for each iteration, use the –d�- Use JSON��newman run mycollection.json -d data.json�- Use CSV��

17 of 25

TIPS TO KEEP IN MIND WHEN USING NEWMAN

17

Use the JSON reporter and a file name to save the output into a file.

$ newman run mycollection.json --reporters cli,json --reporter-json-export outputfile.json

Newman also supports file uploads. For this to work correctly, upload the file in the relative location specified in the collection.�

18 of 25

RUN POSTMAN SCRIPTS WITH NEWMAN�DEMO

18

19 of 25

CI/CD WITH NEWMAN AND JENKINS

19

20 of 25

CI/CD WITH NEWMAN AND JENKINS

20

21 of 25

MANAGING MULTIPLE COLLECTIONS WITH DIFFERENT MAUTH PROXIES WITH NEWMAN

22 of 25

22

23 of 25

Thank you very much for your time

If you have any questions please don’t hesitate to contact me at:

www.thalayasingamchristina.wordpress.com
@ChristinaThalay
Linkedin : Christina Thalayasingam

23

24 of 25

24

REFERENCES

https://blog.testproject.io/2021/06/16/api-testing-101/

https://learning.postman.com/docs/running-collections/using-newman-cli/command-line-integration-with-newman/

https://learning.postman.com/

25 of 25

QUESTIONS

25

?