Is Changing your approach to Software Testing a Risky Business?
The benefits of implementing automated software testing solutions are abundant. However, in any discussion about making the business case for implementing automated software testing, one area that should not be overlooked is an honest evaluation of the potential risks. These would include any events, actions, or circumstances that might prevent an automated test program from being successfully implemented and executed. Potential barriers to success may include: lack of time, lack of skills, constantly changing requirements, budget issues, delayed arrival of necessary equipment, or postponement of the automated software installation.
When making the business case for a new automated software testing effort, it should include not only potential risks, but also strategies for addressing these issues, should they arise. Cost overruns, schedule slippage, and critical software errors are all potential areas of failure. A successful testing effort not only works, but meets business expectations on these and other fronts.
In our experience, most risk is caused by a few factors:
Testing budgets and schedules are often determined at the onset of a project and based on past projects or other evaluation techniques. Our experience and customer research has revealed that a thorough manual testing effort often takes up 50% or more of a project’s software development schedule. Even with automation, if the testing allowance is not generous enough, corners are often cut. Lack of complete testing coverage due to time and budget constraints can lead to project failure. As part of the business case, this issue should be addressed. Schedules should be adjusted or risk mitigation strategies should be developed.
Lack of adequate skills
Not all companies will have the in-house resources—in human capital and skills—to fully develop and implement a new automated software testing effort. This area should be closely evaluated. Several solutions are available, including: hiring more testing engineers, purchasing an automated testing product, and/or working with an experienced software testing partner like Innovative Defense Technologies. Limited availability of skilled testing engineers would be another potential risk to an automation effort.
Changing or difficult-to-automate requirements
Some existing system features that are not well-documented can be a potential problem area. Additionally, as development progresses, requirements often change or grow. For a requirement or feature to be successfully tested in an automated fashion, this information needs to be available. Functional or nonfunctional requirements that are difficult to test due to complexity or lack of documentation will also post a high risk to an automation effort.
Whenever new technology is implemented, there is the potential that a new automated testing tool will have to be either developed in-house or purchased. This can be considered a risk as well.
Any time a software-dependent business changes its approach, it is risky. Implementing an automated software testing strategy is no exception. However the benefits, as outlined in many previous blogs, are abundant. As long as the risks are honestly evaluated and planned for, it may be time for change.
Some information taken from: Dustin, Elfriede, Thom Garrett, and Bernie Gauf. Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality. Upper Saddle River, NJ: Addison-Wesley, 2009. This book was authored by three current IDT employees and is a comprehensive resource on AST. Blog content may also reflect interviews with the authors.