Defect Tracking & Reporting: More Efficient with Automated Software Testing
The defect tracking lifecycle is a critical component of any software testing program. Documentation of defects needs to be thorough and all of the stakeholders need to understand the process, its implications and the results. This includes monitoring error status and implementing corrections.
Defect tracking tools can be purchased or developed in-house. Once tools are acquired, the defect tracking lifecycle must be instituted, documented and communicated. Testers need to communicate with engineers regarding the defect flow—from detection to resolution. Engineers must work with testers to determine when retesting is required. Once these and other questions are resolved, the process must be finalized. These procedures would benefit greatly from automation.
In establishing the software testing lifecycle, the details surrounding uncovered defects must be clearly documented by the test team. This will assist the development team’s defect correction activities. Defects also need to prioritized, so that high-priority defects are resolved first. Test engineers need to be on board with a defect walkthrough and included in reviewing reports.
Once identified defects are corrected and incorporated into a new software build, the focus switches to retesting, and this is indicated to the testers in the defect-tracking tool. Ideally, the fixed defect status will contain a description of the fix and list other areas of the system that might be affected by it.
Test teams need to perform defect reporting by following a clearly defined process, including:
- Analysis – How should testers evaluate unexpected system behavior, including suspected false positives and false negatives? Testers will need to have the diagnostic knowledge to determine the correctness of the test output.
- Defect entry – If the tester determines that the unexpected system behavior is a defect, this is entered into the defect-tracking tool by a member of the test team.
- Recurrence – How will the defect reporting process handle a recurring issue? Should this be recorded as a new defect, or should the previous defect be reopened?
- Closure – Once the defect has been corrected or ruled out, who has the authority to close out the defect? Usually, this should be the test manager.
Even after defects are resolved, we recommend keeping all defect records for historical purposes. One reason for this is that deleting the defect through the interface can interrupt the continuity of the numbering system. Testers and engineers may want previous defect information to inform future development.
We also recommend that defect reports are as detailed as possible and only one issue should be documented at a time. Even very similar defects should be documented separately. And partially corrected defects should not be closed as fixed.
Once defects have been corrected and tested to the satisfaction of the development team, the corrected code should be checked in using the software configuration management tool. At preset intervals or as critical defects are corrected, the team begins a new software build and, upon completion, gives that to the test team to initiate another cycle of testing.
Automated software testing tools, like ATRT: Test Manager, streamline the testing process and allow for automatic documentation of defects with minimal manual intervention.
Some information taken from: Dustin, Elfriede, Thom Garrett, and Bernie Gauf. Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality. Upper Saddle River, NJ: Addison-Wesley, 2009. This book was authored by three current IDT employees and is a comprehensive resource on AST. Blog content may also reflect interviews with the authors.