In almost every blog, we make the case that Automated Software Testing improves the quality of software programs. But before launching an Automated Software Testing effort, it is prudent to develop a preliminary estimate of the projected time and cost savings that would result.
In our experience, it almost always makes good business sense to implement automated software testing on applications that consistently undergo regression testing. These projects usually have a standard set of test procedures that is repeated with each delivery. Additionally, these same tests need to be run across multiple configurations. Here’s an example based on our experience with regression testing: a test scenario with multiple data inputs that previously took three days to execute manually was run in less than an hour after automation. And the project benefitted from this time reduction each time the test was rerun.
When calculating the savings—or potential savings—for an automation effort, it is most helpful to have records of the time and manpower involved in previous manual testing efforts. Because development teams don’t always retain this history, it is sometimes challenging to calculate the exact time savings once automation is introduced. When that is the case, the best alternative we have found is to meet with the test team and reconstruct the historical data as accurately as possible.
Test teams usually have a pretty good idea how many people were involved in previous test efforts, how long individual tests took to run, and how frequently tests were repeated. Based on this information, it is possible to develop estimates for the time and effort associated with manual testing. Once automation is introduced, and cost calculations are in hand, this information can be used to create a metric by which to evaluate similar future projects.
Some projects have mitigating factors that prevent initial time and cost savings. These may include: new test programs, less mature test processes, and teams of testers who are brand new to automation. If facing these or similar circumstances, the first few automated test runs may be more protracted than the future ones will be. It is also important to weigh whether or not automation is implemented with a reliable product or with a proven partner.
This is an overview on estimating the potential time and cost savings provided by automated software testing. Basically, it is a simplified Return on Investment (ROI) summary. A previous blog, Making the Business Case for Automated Software Testing, also addresses automation from a business perspective. Future blogs will go into more detail on this topic. Stay tuned!
Some information taken from: Dustin, Elfriede, Thom Garrett, and Bernie Gauf. Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality. Upper Saddle River, NJ: Addison-Wesley, 2009. This book was authored by three current IDT employees and is a comprehensive resource on AST. Blog content may also reflect interviews with the authors.