Software Testing�a Innovative Perception
Dr.Somnath Thigale
Head & Associate Professor - CSE Dept.
Innovation Ambassador
Chapter No 5�Defect Management
Different causes of software defects
Defect Classification
Defect Classification
1.Severity Wise
2.Work Product Wise
3.Type Of Error Wise
4.Status Wise
Defect Classification
Severity Wise:
Defect Classification
Work product wise:
Defect Classification
Type of Errors Wise:
Inadequate or sub optimal Design: Design features/approach needs additional inputs for it to be complete Design features described does not provide the best approach (optimal approach) towards the solution required
In correct Design: Wrong or inaccurate Design
Ambiguous Design: Design feature/approach is not clear to the reviewer. Also includes ambiguous use of words or unclear design features.
Boundary Conditions Neglected: Boundary conditions not addressed/incorrect
Interface Error: Internal or external to application interfacing error, Incorrect handling of passing parameters, Incorrect alignment, incorrect/misplaced fields/objects, un friendly window/screen positions
Logic Error: Missing or Inadequate or irrelevant or ambiguous functionality in source code
Message Error: Inadequate/ incorrect/ misleading or missing error messages in source code
Navigation Error: Navigation not coded correctly in source code
Performance Error: An error related to performance/optimality of the code
Missing Requirements: Implicit/Explicit requirements are missed/not documented during requirement phase
Inadequate Requirements: Requirement needs additional inputs for to be complete
Incorrect Requirements: Wrong or inaccurate requirements
Ambiguous Requirements: Requirement is not clear to the reviewer. Also includes ambiguous use of words – e.g. Like, such as, may be, could be, might etc.
Sequencing / Timing Error: Error due to incorrect/missing consideration to timeouts and improper/missing sequencing in source code.
Standards: Standards not followed like improper exception handling, use of E & D Formats and project related design/requirements/coding standards
System Error: Hardware and Operating System related error, Memory leak
Test Plan / Cases Error: Inadequate/ incorrect/ ambiguous or duplicate or missing - Test Plan/ Test Cases & Test Scripts, Incorrect/Incomplete test setup
Typographical Error: Spelling / Grammar mistake in documents/source code
Variable Declaration Error: Improper declaration / usage of variables, Type mismatch error in source code
Defect Classification
Status Wise:
Defect Management Process
The process of finding defects and reducing them at the lowest cost is called as Defect Management Process
Defect Prevention -- Implementation of techniques, methodology and standard processes to reduce the risk of defects.
Deliverable Baseline -- Establishment of milestones where deliverables will be considered complete and ready for further development work. When a deliverable is base lined, any further changes are controlled. Errors in a deliverable are not considered defects until after the deliverable is base lined.
Defect Discovery -- Identification and reporting of defects for development team acknowledgment. A defect is only termed discovered when it has been documented and acknowledged as a valid defect by the development team member(s) responsible for the component(s) in error.
Defect Resolution -- Work by the development team to prioritize, schedule and fix a defect, and document the resolution. This also includes notification back to the tester to ensure that the resolution is verified.
Process Improvement -- Identification and analysis of the process in which a defect originated to identify ways to improve the process to prevent future occurrences of similar defects. Also the validation process that should have identified the defect earlier is analyzed to determine ways to strengthen that process.
Management Reporting -- Analysis and reporting of defect information to assist management with risk management, process improvement and project management.
Defect Life Cycle
Defect Prevention Process
Identify Critical Risks -- Identify the critical risks facing the project or system. These are the types of defects that could jeopardize the successful construction, delivery and/or operation of the system.
Estimate Expected Impact -- For each critical risk, make an assessment of the financial impact if the risk becomes a problem.
Minimize Expected Impact -- Once the most important risks are identified try to eliminate each risk. For risks that cannot be eliminated, reduce the probability that the risk will become a problem and the financial impact should that happen.
The five general activities of defect prevention are:
Defects introduced during the requirements and design phase are not only more probable but also are more severe and more difficult to remove.
Front-end errors in requirements and design cannot be found and removed via testing, but instead need pre-test reviews and inspections.
Self-review is one of the most effective activity in uncovering the defects which may later be discovered by a testing team or directly by a customer.
A self-review of the code helps reduce the defects related to algorithm mplementations, incorrect logic or certain missing conditions.
Peer review is similar to self-review in terms of the objective – the only difference is that it is a peer (someone who understands the functionality of the code very well) who reviews the code
Effective defect tracking begins with a systematic process. A structured tracking process begins with initially logging the defects, investigating the defects, then providing the structure to resolve them. Defect analysis and reporting offer a powerful means to manage defects and defect depletion trends, hence, costs.
4. Root Cause Analysis and Preventive Measures Determination
After defects are logged and documented, the next step is to analyze them
Reducing the defects to improve the quality: The analysis should lead to implementing changes in processes that help prevent defects and ensure their early detection.
Applying local expertise: The people who really understand what went wrong are the people present when the defects were inserted – members of the software engineering team. They can give the best suggestions for how to avoid such defects in the future.
Targeting the systematic errors: There may be many errors or defects to be handled in such an analysis forum; however, some mistakes tend to be repeated. These systematic errors account for a large portion of the defects found in the typical software project.
Implementation is the toughest of all activities of defect prevention.
It requires total commitment from the development team and management.
A plan of action is made for deployment of the modification of the existing processes or introduction of the new ones with the consent of management and the team.
Defect Report Template
ID | Unique identifier given to the defect. (Usually Automated) |
Project | Project name. |
Product | Product name. |
Release Version | Release version of the product. (e.g. 1.2.3) |
Module | Specific module of the product where the defect was detected. |
Detected Build Version | Build version of the product where the defect was detected (e.g. 1.2.3.5) |
Summary | Summary of the defect. Keep this clear and concise. |
Description | Detailed description of the defect. Describe as much as possible but without repeating anything or using complex words. Keep it simple but comprehensive. |
Steps to Replicate | Step by step description of the way to reproduce the defect. Number the steps. |
Actual Result | The actual result you received when you followed the steps. |
Expected Results | The expected results. |
Attachments | Attach any additional information like screenshots and logs. |
Remarks | Any additional comments on the defect. |
Defect Severity | Severity of the Defect. |
Defect Priority | Priority of the Defect. |
Reported By | The name of the person who reported the defect. |
Assigned To | The name of the person that is assigned to analyze/fix the defect. |
Status | The status of the defect. |
Fixed Build Version | Build version of the product where the defect was fixed (e.g. 1.2.3.9) |
DEFECT REPORT TEMPLATE
Estimate Expected Impact Of A Defect
E = P * I, where:
P= probability of the risk becoming a problem and
I= Impact in dollars if the risk becomes a problem.
Once the expected impact of each risk is identified, the risks should be prioritized by the expected impact and the degree to which the expected impact can be reduced. While guess work will constitute a major role in producing these numbers, precision is not important. What will be important is to identify the risk, and determine the risk's order of magnitude. Large, complex systems will have many critical risks. Whatever can be done to reduce the probability of each individual critical risk becoming a problem to a very small number should be done. Doing this increases the probability of a successful project by increasing the probability that none of the critical risks will become a problem.
One should assume that an individual critical risk has a low probability of becoming a problem only when there is specific knowledge justifying why it is low. For example, the likelihood that an important requirement was missed may be high if developers have not involved users in the project. If users have actively participated in the requirements definition, and the new system is not a radical departure from an existing system or process, the likelihood may be low.
For example:
o An organization with a project of 2,500 function points and was about medium at defect discovery and removal would have 1,650 defects remaining after all defect removal and discovery activities.
o The calculation is 2,500 x 1.2 = 3,000 potential defects.
o The organization would be able to remove about 45% of the defects or 1,350 defects.
o The total potential defects (3,000) less the removed defects (1,350) equals the remaining defects of 1,650.
Techniques For Finding Defects
Reporting A Defect