Metrics can aid in improving automated testing processes and tracking the status of such efforts. While we don’t recommend that metrics alone drive the automated software testing process, they can certainly enhance it. Carefully defined metrics offer objective, measurable factors by which to assess a testing effort. The better the testing effort is, the better the quality of the final product. This blog will focus on the importance of defect removal as it relates to software quality.
One of the more popular metrics for tracking quality through testing, if defect count is being used as a measure of quality, is Defect Removal Efficiency (DRE). This metric is not specific to automation, but is very useful when used in conjunction with automation efforts.
DRE is a metric used to determine the effectiveness of your defect removal efforts. It is also an indirect measurement of the quality of the product. The value of the DRE is calculated as a percentage. The higher the DRE percentage is, the higher the positive impact on the quality of the product. This is because it represents the timely identification and removal of defects at any particular phase.
- DRE = Defect Removal Efficiency
- DT = Number of defects found during testing
- DA = Number of defects found after delivery
The highest attainable value of DRE is “1” which equates to 100%. In practice we have found that an efficiency rating of 100% is not likely. DRE should be measured during the different development phases. If the DRE is low during analysis and design, it may indicate that more time should be spent improving the way formal technical reviews are conducted.
This calculation can be extended for released products as a measure of the number of defects in the product that were not caught during the product development or testing phases.
For more information on using metrics as a measure of quality during software testing contact IDT, consult our previous blogs, or read the complete article, Useful Automated Software Testing Metrics, upon which this blog is based.
Some information taken from: Dustin, Elfriede, Thom Garrett, and Bernie Gauf. Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality. Upper Saddle River, NJ: Addison-Wesley, 2009. This book was authored by three current IDT employees and is a comprehensive resource on AST. Blog content may also reflect interviews with the authors.