Current news headlines give us notable examples of large, complex software systems that could have benefited from additional testing prior to release. But even with best laid plans, when deadlines loom and customers are waiting, time has a way of evaporating. This is the third in a series of four blogs on this topic–beginning on September 30.
To insure that software systems are truly ready for release, engineers and developers must establish a strategy for thorough testing. This can be accomplished most efficiently through Automated Software Testing, which offers many advantages over manual testing.
In addition to increasing the depth and breadth of testing, we have found at least sixteen notable benefits of implementing Automated Software Testing. Our two previous blogs on this topic addressed the first eight benefits. Today’s blog contains four more.
Allows for Lights-Out Testing
Automated Software Testing enables test scripts to be set up to run at any specified time, including during lunch or after hours. Often called Lights-Out testing, running tests after hours enables testing without any user interaction. Test engineers can set up automated tests in advance; they can then be initiated by the test tool in a defined sequence at a specified time and run while testers are not present. When the testers return to work (or come back from their meetings or lunch), they can review the automated test execution output and conduct an analysis. Running tests at these times maximizes the use of the test lab and the testers’ time.
Improved Requirements Definition
Part of the Software Testing Lifecycle (STL) that can be automated is requirements management. Automation of requirements keeps an audit trail of any changes or updates, produces an automated Requirements Traceability Matrix (RTM) and links requirements to all artifacts, including test procedure pass/fail results and defects. This can insure that the SUT is tested against requirements and will meet expectations.
Improved Performance Testing
As inefficient as it may seem, some companies still gather performance or transaction timing data with a combination of manual test executions and stopwatches. Obviously, this method of capturing performance measures is labor-intensive and error-prone. Automated tools enable test engineers to capture performance statistics automatically. This frees the test engineer to do more intellectually challenging testing work.
Improved Stress and Endurance Testing
Stress testing involves running a system in high-volume scenarios and subjecting it to maximum loads to find out where and how often the system breaks. Also known as endurance testing, it alerts testers to system weaknesses that, left unchanged, will occur when the system is overloaded.
It is extremely difficult to manually stress-test a complex system intended for high-volume use. This is due to the inability to reproduce the large number of users and workstations required for it. Using a limited number of client workstations, automated test tools can simulate a large number of users (using virtual users) interacting with a system. The process begins by capturing user interactions and then running multiple instances of test scripts to simulate the projected user base.
Some automated test tools include a load simulator which enables high-volume scenarios for testing. Even these tests can run unaccompanied and will produce a concise log of results. Any system intended for high visibility and broad use would benefit from the extensive stress testing that automation can offer.
Implementing Automated Software Testing into your testing program offers many advantages. We have now elaborated on twelve of them. Check back on October 21 for part four in this series.
Some information taken from: Dustin, Elfriede, Thom Garrett, and Bernie Gauf. Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality. Upper Saddle River, NJ: Addison-Wesley, 2009. This book was authored by three current IDT employees and is a comprehensive resource on AST. Blog content may also reflect interviews with the authors.