At the beginning of an automated software testing effort, the project is either automating existing manual test procedures, starting a new automation effort from scratch, or some combination of both. Whichever the case, a percent automatable metric can be determined. Percent automatable can be defined as: of a set of given test cases, how many are automatable? This could be represented in the following equation:
ATC # of test cases automatable
PA (%) = ———– = ( ———————————————- )
TC # of total test cases
- PA = Percent Automatable
- ATC = # of test cases automatable
- TC = # of total test cases
In evaluating test cases to be developed, what is to be considered automatable and what is not to be considered automatable? Given enough ingenuity and resources, one can argue that almost anything can be automated. So where do you draw the line? Something that can be considered ‘not automatable’ for example, could be an application area that is still under design, not very stable, and much of it is in flux. In cases such as this, we should:
“Evaluate whether it make sense to automate.”
We would evaluate for example, given the set of automatable test cases, which ones would provide the biggest return on investment:
“Just because a test is automatable doesn’t necessary mean it should be automated.”
When going through the test case development process, determine what tests can be AND makes sense to automate. Prioritize your automated software testing effort based on your outcome. This metric can be used to summarize, for example, the % automatable of various projects or component within a project, and set the automation goal.
Some information taken from: Dustin, Elfriede, Thom Garrett, and Bernie Gauf. Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality. Upper Saddle River, NJ: Addison-Wesley, 2009. This book was authored by three current IDT employees and is a comprehensive resource on AST.