By Elfriede Dustin–Currently numerous automated software testing solutions are available on the market. Deciding which solution to choose is tough. After an initial high-level evaluation, some users will think most tools are created equal and choose their solution based on market share or may select a product based on cost.
Before you choose an Automated Software Testing solution based solely on such generic criteria, let’s take a closer look. First, it is important to understand the types of tests that ideally are automated. Next, verify that the automated solution you are considering supports automating these types of tests. Last, select a testing solution based on your testing domain. Verifying the solution meets this type of criteria will eventually provide the biggest return on your automated test investment.
Based on our experience, we know that moving automated testing to the left of the development lifecycle and starting with automating the unit tests (also called white box testing) during development is most efficient. The integration testing phase is a good time to add grey box tests and black box tests can complete the testing cycle.
Your automated software testing solution therefore ideally supports these types of tests:
- White box testing – Automating unit tests as part of your development effort will uncover the defects during the phase when it is still cheapest to fix the bugs. Verify the automated software testing solution allows integration of unit testing tools such as nunit or junit.
- Grey box testing – Automating GUI based and message based tests combined is one of the most powerful approaches to integration testing. Now you can detect defects at the GUI level and at the message level. Verify the automated software testing solution supports grey box testing.
- Black box testing– Automating the tests that are verifying the system as a black box without knowledge of the system internals most often focus on the GUI. Verify the automated software testing solution supports black box testing as part of your testing arsenal. Additionally, verify the following criteria:
- Does the tool support systems of system testing – can it run automated tests in parallel on a system of systems architecture
- Does the tool recognize all controls on the System under Test (SUT) – Verify the solution supports all GUI controls, even “generic” controls and doesn’t require some sort of generic GUI control mapping
- Does the tool support export/import of test artifacts – Being able to export and import automated tests is especially important in a collaborative and/or geographically disbursed testing environment
Additionally, your automated software testing solution should address the following concerns:
- Does the solution modify the system configuration? Make sure the automated software testing solution doesn’t have to be installed on the SUT thus modifying the SUT’s configuration; that is simply not acceptable to our customers
- Can the solution automate tests for a SUT running on Windows, Linux and Solaris or any other OS or platform? If you need to run automated tests on any other OS than Windows, verify that your solution supports it. Don’t settle for “analog” recording on any other OS than Windows. We highly recommend against automating any tests using “analog” recording based on x,y coordinates.
- Does it easily support a cloud environment? Make sure your solution doesn’t need to physically be installed on each SUT component in order to automate the tests, defeating the entire purpose of being able to access SUTs remotely and automate or execute the tests remotely in a cloud environment setting.
Given all the tool solutions and choices currently available, I highly recommend you look at this list of tool criteria before choosing any automated testing solution. I invite you to consider the solutions IDT develops based on patented ATRT technology–which meets these criteria and provides many additional features. Contact Innovative Defense Technologies (IDT) for more information.