One of the most important factors in implementing a successful Automated Software Testing (AST) effort is knowing and understanding all of the requirements. These include: your user requirements (What does the user want?); the system requirements and use cases (How will the user requirements be implemented?); the test requirements and mission threads (How will the system be tested?); and ultimately, the automated test requirements (Which tests need to be automated and how will they be automated?). Requirements are the blueprint which guides how your testing program is built, implemented and measured.
Requirements should be gathered and organized in a concise manner, as they will form the basis for effective AST planning. As with all information gathering, the accuracy of the information collected is generally proportional to the effort directed at the task. The project team will need to exercise discernment in prioritizing requirements and classifying them by levels of importance (i.e., high-risk versus low priority).
Various types of requirements need to be considered when designing an effective automated testing effort, including:
- AUT or SUT requirements
- Automation framework & tool requirements
- AST data requirements
- AST environment requirements
- AST process requirements
- AST resources, such as personnel, time lines and budgets
AUT or SUT Requirements
The requirements for the System Under Test (SUT) or the Application Under Test (AUT) come from a variety of areas and also depend on what the customer (internal or external) desires and has specified. In the best case scenario, the test automation team is involved in the process from the beginning and has a thorough understanding of the task and the business logic behind the requirements.
An ideal basis for the gathering of the AUT/SUT requirements comes in many forms and from many areas, including: architecture and design documents, project requirements, test procedures, and general information, as illustrated below.
Once this information is procured, the test team can drill down further to understand the task at hand, its scope, and its associated testing goals in the ways that follow.
Understanding the System
The test team must understand both the functional and nonfunctional (e.g., security, performance, etc.) requirements that the AUT/SUT must meet. Just reading a list of requirement statements like, The system shall…, hardly provides a complete picture, because functional flow and business logic may not be clear. To obtain a broader view, the test team should gather additional input regarding the system, review proposals discussing the problem to be solved, read product management case studies, understand the high-level business requirements and detailed system and design requirements.
Understanding the Scope
Next, it is important to understand the scope of the test requirements, as there are many different tests that can be performed, including: functional requirements testing, network testing, user interface testing, performance testing and security testing, among others.
Requirements definition is a crucial first step on the path towards test automation. Missing or misinterpreted requirements may result in delays, cost overruns, and customer dissatisfaction. An experienced automated software testing partner like Innovative Defense Technologies (IDT) can help define the scope and ROI of your testing effort. Please contact us for more information.
Part two of this blog, addressing AST framework, tool and data requirements, is posted here.
Some information taken from: Dustin, Elfriede, Thom Garrett, and Bernie Gauf. Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality. Upper Saddle River, NJ: Addison-Wesley, 2009. This book was authored by three current IDT employees and is a comprehensive resource on AST. Blog content may also reflect interviews with the authors.