Software Testing�a Innovative Perception

Dr.Somnath Thigale

Head & Associate Professor - CSE Dept.

Innovation Ambassador

Chapter 2 :Types Of Testing

White-box testing

• White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) is a method of testing software that tests internal structures or workings of an application.
•  In white-box testing an internal perspective of the system, as well as programming skills, are used to design test cases. The tester chooses inputs to exercise paths through the code and determine the expected outputs. 

• White-box testing can be applied at the unit, integration and system levels of the software testing process. 

​

• It can test paths within a unit, paths between units during integration, and between subsystems during a system–level test. 

White-box test design techniques

• White-box test design techniques include the following code coverage criteria:
• Control flow testing
• Data flow testing
• Branch testing
• Statement coverage
• Decision coverage
• Modified condition/decision coverage
• Prime path testing
• Path testing

​

Classification of White Box Testing

Static Testing

• Static testing is the testing of the software work products manually, or with a set of tools, but they are not executed.
• It starts early in the Life cycle and so it is done during the verification process.
• It does not need computer as the testing of program is done without executing the program. For example:  reviewing, walk through, inspection, etc.

​

Advantages Of Static Testing

• Since static testing can start early in the life cycle, early feedback on quality issues can be established.
• By detecting defects at an early stage, rework costs are most often relatively low.
• Since rework effort is substantially reduced, development productivity figures are likely to increase.
• The evaluation by a team has the additional advantage that there is an exchange of information between the participants.
• Static tests contribute to an increased awareness of quality issues.

Disadvantages Of Static Testing

• Demand great amount of time when done manually
• Automated tools works with only few programming languages
• Automated tools may provide false positives and false negatives
• Automated tools only scan the code
• Automated tools cannot pinpoint weak points that may create troubles in run-time

​

Static Testing

1) Walkthrough�2) Inspection�3) Technical Review

​

Walkthrough�

• In walkthrough, author guides the review team via the document to fulfill the common understanding and collecting the feedback.
• Walkthrough is not a formal process.
• In walkthrough, a review team does not require to do detailed study before meeting while authors are already in the scope of preparation.
• Walkthrough is useful for higher-level documents i.e requirement specification and architectural documents.

​

Goals Of Walkthrough

• Make the document available for the stakeholders both outside and inside the software discipline for collecting the information about the topic under documentation.
• Describe and evaluate the content of the document.
• Study and discuss the validity of possible alternatives and proposed solutions.

​

Structured Walkthrough Participants:�

• Author - The Author of the document under review.
• Presenter - The presenter usually develops the agenda for the walkthrough and presents the output being reviewed.
• Moderator - The moderator facilitates the walkthrough session, ensures the walkthrough agenda is followed, and encourages all the reviewers to participate.
• Reviewers - The reviewers evaluate the document under test to determine if it is technically accurate.
• Scribe - The scribe is the recorder of the structured walkthrough outcomes who records the issues identified and any other technical comments, suggestions, and unresolved questions.

​

Benefits of Structured Walkthrough �

• Saves time and money as defects are found and rectified very early in the lifecycle.
• This provides value-added comments from reviewers with different technical backgrounds and experience.
• It notifies the project management team about the progress of the development process.
• It creates awareness about different development or maintenance methodologies which can provide a professional growth to participants.

​

 Inspection�

• The trained moderator guides the Inspection. It is most formal type of review.
• The reviewers are prepared and check the documents before the meeting.
• In Inspection, a separate preparation is achieved when the product is examined and defects are found. These defects are documented in issue log.
• In Inspection, moderator performs a formal follow-up by applying exit criteria.

​

Goals of Inspection

• Assist the author to improve the quality of the document under inspection.
• Efficiently and rapidly remove the defects.
• Generating the documents with higher level of quality and it helps to improve the product quality.
• It learns from the previous defects found and prohibits the occurrence of similar defects.
• Generate common understanding by interchanging information.

​

Technical Review�

• Technical review is a discussion meeting that focuses on technical content of the document. It is a less formal review.
• It is guided by a trained moderator or a technical expert.

​

Goals of technical review

• The goal is to evaluate the value of technical concept in the project environment.
• Build the consistency in the use and representation of the technical concepts.
• In early stages it ensures that the technical concepts are used correctly.
• Notify the participants regarding the technical content of the document.

​

Difference between walkthrough and inspection

Structural Testing

• The structural testing is the testing of the structure of the system or component.
• Structural testing is often referred to as ‘white box’ or ‘glass box’ or ‘clear-box testing’ because in structural testing we are interested in what is happening ‘inside the system/application’.
• In structural testing the testers are required to have the knowledge of the internal implementations of the code. Here the testers require knowledge of how the software is implemented, how it works.

​

• During structural testing the tester is concentrating on how the software does it. For example, a structural technique wants to know how loops in the software are working. Different test cases may be derived to exercise the loop once, twice, and many times.
• Structural testing can be used at all levels of testing. Developers use structural testing in component testing and component integration testing, especially where there is good tool support for code coverage. Structural testing is also used in system and acceptance testing, but the structures are different. For example, the coverage of menu options or major business transactions could be the structural element in system or acceptance testing.

​

Code Functional Testing

Control Flow Graph

• It is a graphical representation of the program depicting all the paths that may be transverse during the execution. It contains
• Basic Block or Node: It is a sequence of statements such that control can only have one entry point and can exit the block only when all the statements have been executed.
• Edge:It shows the control flow.

​

N1, N2, N3, N4 and N5 represents Basic Blocks or Nodes and E1, E2, E3, E4 and E5 represents Edges. We can also observe that N2, N3, N4 and N5 together represent an IF loop.

Statement Coverage:

• It aims to test all the statements present in the program
• Adequacy Criterion should be equal to 1 to ensure 100% coverage. It is a good measure of testing each part in terms of statements but it is not a good technique for testing the control flow.

​

Here all the statements in the program can be covered using two flow

1. E1->E2->E5
2. E1->E3

But using these two statements the flow from N3 to N4 is not tested i.e. E4. It does not provide branch coverage. N3 has 2 exit points depending upon the decision statement in N3 i.e. True and False, but since N4 has already been covered, the testing for a decision statement is missed out.

Branch Coverage:�

• It is also known as Decision coverage testing. It aims to test all the branches or edges at least once in the test suite or to test each branch from a decision point at least once. It provides solution for the problem faced in Statement coverage. 
• It is also known as Decision coverage testing. It aims to test all the branches or edges at least once in the test suite or to test each branch from a decision point at least once. It provides solution for the problem faced in Statement coverage. 

In this case, the 100% branch coverage can be provided as following

• E1->E2
• E3

But as we can see edge E1 can be tested for multiple conditions, but under branch testing the aim is to just test the edge for one condition and not for multiple conditions associated to it. The different conditions can be:

• A>2 and B>8 (A True and B False)
• A<2 and B<8 (A False and B True) etc.

Condition Coverage:�

• It aims to test individual conditions with possible different combination of Boolean input for the expression.
• It provides better coverage and the problem discussed under Branch coverage can be resolved 

However when compound conditions are involved the no. of test cases may increase exponentially, which is not desired.

Modified Condition/Decision coverage:

• It is a modification of condition coverage. Its adequacy criterion can be defined as N+1 test cases for N variables. The test cases are derived such that each variable independently affects the final output. So it provides coverage for different conditions without increasing the size of test case suit.

Path Coverage:�

• It aims to test the different path from entry to the exit of the program, which is a combination of different decisions taken in the sequence.
• The paths can be too many to be considered for testing, for ex, a loop can go on and on. It can be avoided using cyclomatic complexity, which help in finding out the redundant test cases.

​

[Cyclomatic Complexity is quantitative measure of no. of linearly independent paths through a program and is used to indicate complexity of a program].

​

​

So, cyclomatic complexity helps aiming to test all linearly independent paths in a program at least once. These were some of the test coverage under this Testing.

Cyclomatic complexity is a software metric used to measure the complexity of a program.

This metric measures independent paths through the program's source code. An independent path is defined as a path that has at least one edge which has not been traversed before in any other paths.

Cyclomatic complexity can be calculated with respect to functions, modules, methods or classes within a program.

IF a OR b

then procedure x

else procedure y

ENDIF

insertion_procedure (int a[], int p [], int N)

02. {

03.     int i,j,k;

04.     for (i=0; i<=N; i++) p[i] = i;

05.     for (i=2; i<=N; i++)

06.     {

07.         k = p[i];

08.         j = 1;

09.         while (a[p[j-1]] > a[k]) {p[j] = p[j-1]; j--}

10.         p[j] = k;

11.     }

12. }

01.insertion_procedure (int a[], int p [], int N)

02.  {

03. (1)    Int i,j,k;

04. (2)    for ((2a)i=0; (2b)i<=N; (2c)i++)

05. (3)        p[i] = i;

06. (4)    for ((4a)i=2; (4b)i<=N; (4c)i++)

07.        {

08. (5)       k=p[i];j=1;

09. (6)       while (a[p[j-1]] > a[k]) {

10. (7)           p[j] = p[j-1];

11. (8)           j--

12.           }

13. (9)          p[j] = k;

14.        }

Now, to calculate the cyclomatic complexity you use one of three methods:

1. Count the number of regions on the graph: 4
2. No. of predicates (red on graph) + 1 : 3 + 1 = 4
3. No of edges – no. of nodes + 2: 14 – 12 + 2 = 4

McCabe’s Cyclomatic Complexity

Mathematically, the cyclomatic complexity of a structured program is defined with reference to the control flow graph of the program, a directed graph containing the basic blocks of the program, with an edge between two basic blocks if control may pass from the first to the second. The complexity CC is then defined as

CC= E − N + 2P,

where

E = the number of edges of the graph.

N = the number of nodes of the graph.

P = the number of connected components.

​

The graph is strongly connected, and the cyclomatic complexity of the program is equal to the cyclomatic number of its graph which is defined as

CC = E − N + P.

​

For a single program (or subroutine or method), P is always equal to 1. So a simpler formula for a single subroutine is

CC = E − N + 2

​

Or it can be defined as

CC=D+1

Where D=Number of decision points in the graph

Flow graph notation for a program:

Example :

i = 0;

​

n=4; //N-Number of nodes present in the graph

​

while (i<n-1) do

​

j = i + 1;

​

while (j<n) do

​

if A[i]<A[j] then

​

swap(A[i], A[j]);

​

end do;

​

i=i+1;

​

end do;

Flow graph for this program will be

Computing mathematically,

​

CC = 9 - 7 + 2 = 4

CC= 3 + 1 = 4 (Condition nodes are 1,2 and 3 nodes)

Basis Set - A set of possible execution path of a program

1, 7

1, 2, 6, 1, 7

1, 2, 3, 4, 5, 2, 6, 1, 7

1, 2, 3, 5, 2, 6, 1, 7

Example :

• IF A = 10 THEN
• IF B > C THEN
• A = B
• ELSE
• A = C
• ENDIF
• ENDIF
• Print A
• Print B
• Print C

Advantages of Structural Testing:�

• Provides a more thorough testing of the software.
• Helps finding out defects at an early stage.
• Helps in eliminating dead code.
• Not time consuming as it is mostly automated.

​

Disadvantages of Structural Testing:�

• Requires knowledge of the code.
• Requires training in the tool used for testing
• It is expensive.

​

Black Box Testing

• Black Box Testing, also known as Behavioral Testing, is a software testing method in which the internal structure/ design/ implementation of the item being tested is not known to the tester. These tests can be functional or non-functional, though usually functional

This method attempts to find errors in the following categories:

​

• Incorrect or missing functions
• Interface errors
• Errors in data structures or external database access
• Behavior or performance errors
• Initialization and termination errors

EXAMPLE :

A tester, without knowledge of the internal structures of a website, tests the web pages by using a browser; providing inputs (clicks, keystrokes) and verifying the outputs against the expected outcome.

BLACK BOX TESTING ADVANTAGES

• Tests are done from a user’s point of view and will help in exposing discrepancies in the specifications.
• Tester need not know programming languages or how the software has been implemented.
• Tests can be conducted by a body independent from the developers, allowing for an objective perspective and the avoidance of developer-bias.
• Test cases can be designed as soon as the specifications are complete.

​

BLACK BOX TESTING DISADVANTAGES

• Only a small number of possible inputs can be tested and many program paths will be left untested.
• Without clear specifications, which is the situation in many projects, test cases will be difficult to design.
• Tests can be redundant if the software designer/ developer has already run a test case.
• Ever wondered why a soothsayer closes the eyes when foretelling events? So is almost the case in Black Box Testing.

​

Techniques For Black Box Testing

1.Requirement Based Testing

2.Positive & Negative Testing

3.Boundry Value Analysis

4.Decision Tables

5. Equivalence Partitioning

6.User Documentation Testing

7.Graph Based Testing

1.Requirement Based Testing�

• Requirements-based testing is a testing approach in which test cases, conditions and data are derived from requirements. It includes functional tests and also non-functional attributes such as performance, reliability or usability.

Stages in Requirements based Testing:

​

Defining Test Completion Criteria - Testing is completed only when all the functional and non-functional testing is complete.

Design Test Cases - A Test case has five parameters namely the initial state or precondition, data setup, the inputs, expected outcomes and actual outcomes.

Execute Tests - Execute the test cases against the system under test and document the results.

Verify Test Results - Verify if the expected and actual results match each other.

Verify Test Coverage - Verify if the tests cover both functional and non-functional aspects of the requirement.

Track and Manage Defects - Any defects detected during the testing process goes through the defect life cycle and are tracked to resolution. Defect Statistics are maintained which will give us the overall status of the project.

2.Positive & Negative Testing�

• Positive Testing is testing process where the system validated against the valid input data. In this testing tester always check for only valid set of values and check if a application behaves as expected with its expected inputs.
• Positive Testing always tries to prove that a given product and project always meets the requirements and specifications.

Example of Positive testing:�

• To test an application which contains a simple textbox to enter age and requirements say that it should take only integers values. 

Negative Testing is testing process where the system validated against the invalid input data. A negative test checks if a application behaves as expected with its negative inputs.

​

The main reason behind Negative testing is to check the stability of the software application against the influences of different variety of incorrect validation data set.

It should show a validation error message for all invalid inputs or system should not allow to enter a non integer values.

Positive & Negative Testing Scenarios�

Positive Testing Scenario

• Password textbox should accept 6 characters
• Password textbox should upto 20 characters
• Password textbox should accepts any value in between 6-20 chars length.
• Password textbox should accepts all numeric & alphabets values.

​

​

Negative Testing Scenario

​

• Password textbox should not accept less than 6 characters
• Password textbox should not exceeds more than 20 characters
• Password textbox should not accept special characters

​

3.Boundry Value Analysis�

• For the most part, errors are observed in the extreme ends of the input values, so these extreme values like start/end or lower/upper values are called Boundary values and analysis of these Boundary values is called “Boundary value analysis”. It is also sometimes known as ‘range checking’.
• Boundary value analysis is used to find the errors at boundaries of input domain rather than finding those errors in the center of input.

Example Of BVA

• Suppose you have very important tool at office, accepts valid User Name and Password field to work on that tool, and accepts minimum 8 characters and maximum 12 characters. Valid range 8-12, Invalid range 7 or less than 7 and Invalid range 13 or more than 13.

​

​

• Test Cases 1: Consider password length less than 8.
• Test Cases 2: Consider password of length exactly 8.
• Test Cases 3: Consider password of length between 9 and 11.
• Test Cases 4: Consider password of length exactly 12.
• Test Cases 5: Consider password of length more than 12.

​

Test cases for the application whose input box accepts numbers between 1-1000. Valid range 1-1000, Invalid range 0 and Invalid range 1001 or more.

• Test Cases 1: Consider test data exactly as the input boundaries of input domain i.e. values 1   and 1000.
• Test Cases 2: Consider test data with values just below the extreme edges of input domains i.e. values 0 and 999.
• Test Cases 3: Consider test data with values just above the extreme edges of input domain i.e. values 2 and 1001.

4.Decision Tables�

• A decision table is a good way to deal with different combination inputs with their associated outputs and also called cause-effect table.
• Used to determine the test scenarios for complex business logic.
• It helps testers to search the effects of combinations of different inputs
• Provides a regular way of stating complex business rules

Way to use decision tables in test designing

• Firstly; get to know a suitable function or subsystem that acts according to a combination of inputs or events.
• Taken system should be with fewer inputs or else combinations will become impossible. Always better to take maximum numbers of conditions, split them into subsets and use these subsets one at a time.
• After getting features that need to be combined, add them to a table showing all combinations of “Yes” and “No” for each of the feature.

Let’s take an example of a finance application, where users pay money – monthly Repayment or year wise (the term of loan). If user chooses both options, the system will create a negotiation between two. So, there are two conditions of the loan amount, mention in the given below table,

TABLE 1: Blank decision table

Next, recognize all of the combinations in “Yes” and “No” (In Table 2). In each column of two conditions mention “Yes” or “No”, user will get here four combinations (two to the power of the number of things to be combined). Note, if user has three things to combine, they will have eight combinations, with four things, there are 16, etc. Because of this, it’s always good to take small sets of combinations at once. To keep track on combinations, give alternate “Yes” and “No” on the bottom row, put two “Yes” and then two “No” on the row above the bottom row, etc., so the top row will have all “Yes” and then all “No” (Apply the same principle to all such tables).

TABLE 2: Decision table – Input combination

�

In the next step, recognize the exact outcome for each combination (In Table 3). In this example, user can enter one or both of the two fields. Each combination is sometimes referred to as a step.

TABLE 3: Decision table – Combinations and outcomes

At this time you didn’t think that what will happen when customer don’t enter anything in either of the two fields. The table has shown a combination that was not given in the specification for this example. This combination can result as an error message, so it is necessary to add another action (In Table 4). This will flash the strength this method to find out omissions and ambiguities in specifications.

TABLE 4: Decision table – Additional outcomes

We will provide you some other example that allows the customer to enter both repayment and term. This will change the outcome of our table, this will generate an error message if both are entered 

TABLE 5: Decision table – Changed outcomes

The final process of this method is to write test cases to use each of the four steps in our table.

Conditions                           Rule 1               Rule 2                 Rule 3             Rule 4�Repayment amount has           Y                        Y                         N                       N�been entered:�Term of loan has been             Y                         N                         Y                       N�entered: 

Actions/Outcomes:�Result:      Error                Process loan           Process              Error�message               amount                   term                message message

​

Advantage of decision table technique:

​

1. Any complex business flow can be easily converted into the test scenarios & test cases using this technique.
2. Such type of table are work iteratively, means the table created at the first iteration is used as input table for next tables. Such iteration can be carried out only if the initial table is unsatisfactory.
3. Simple to understand and everyone can use this method design the test scenarios & test cases.
4. It provide complete coverage of test cases which help to reduce the rework on writing test scenarios & test cases.
5. These tables guarantee that we consider every possible combination of condition values. This is known as its “completeness property”.

Credit card example:�Let’s take another example. If you are a new customer and you want to open a credit card account then there are three conditions first you will get a 15% discount on all your purchases today, second if you are an existing customer and you hold a loyalty card, you get a 10% discount and third if you have a coupon, you can get 20% off today (but it can’t be used with the ‘new customer’ discount). Discount amounts are added, if applicable. This is shown in Table 

5. Equivalence Partitioning�

• In this method, the input domain data is divided into different equivalence data classes. This method is typically used to reduce the total number of test cases to a finite set of testable test cases, still covering maximum requirements.

Let us consider a program that separates integers into positive or negative. And accepts any number between -5 and + 5.

The range of input integers can be split into  the following partitions :

• Negative Integers: Values between -5 and -1
• Zero (neither positive nor negative)
• Positive Integers: Values between  1 and  5
• Values >  5  (Invalid)
• Values <  -5 (Invalid)

​

1. Any Number greater than 10 entered in the reservation column (let say 11) is considered invalid.
2. Any Number less than 1 that is 0 or below, then it is considered invalid.
3. Numbers 1 to 10 are considered valid
4. Any 3 Digit Number say 100 is invalid.

• Ticket values 1 to 10 are considered valid & ticket is booked. While value 11 to 99 are considered invalid for reservation and error message will appear, "Only ten tickets may be ordered at one time."

6.User Documentation Testing

• It is a type of non-functional testing.
• Any written or pictorial information describing, defining, specifying, reporting, or certifying activities, requirements, procedures, or results’. Documentation is as important to a product’s success as the product itself. If the documentation is poor, non-existent, or wrong, it reflects on the quality of the product and the vendor.
• As per the IEEE Documentation describing plans for, or results of, the testing of a system or component, Types include test case specification, test incident report, test log, test plan, test procedure, test report. Hence the testing of all the above mentioned documents is known as documentation testing.
• This is one of the most cost effective approaches to testing. If the documentation is not right: there will be major and costly problems. The documentation can be tested in a number of different ways to many different degrees of complexity. These range from running the documents through a spelling and grammar checking device, to manually reviewing the documentation to remove any ambiguity or inconsistency.
• Documentation testing can start at the very beginning of the software process and hence save large amounts of money, since the earlier a defect is found the less it will cost to be fixed.

​

BENEFITS

•  Removes Uncertainties
•  Offer good training material to fresher’s
• Good Marketing Strategy
•  Good Coverage and Estimating the Project’s Deadline

7.Graph Based Testing�

• Graph based testing is useful  to represent a transaction or work flows. consider a example of a leave application by an employee. A typical leave application process can be visualized as being made up of the following steps.�

1. Employee desires leave

2. He applies leave application form

3. HR verify eligibility for leave

4. If it is eligible manager ensures feasibility

 If it is feasible manager approve leave

If it is not feasible manager rejects leave

5.If it is not eligible HR rejects leave

​