Background:
This effort was focused on using automation and automated software testing strategies to support test procedures that are run on ships to verify system functionality. The software systems on board ships are increasingly complex. Additionally, the time and effort required for crew members to run test procedures continues to grow, while also demanding increasing levels of expertise. This case study will show the benefits of automation of shipboard test procedures in reducing crew workload while also decreasing the time the system is unavailable due to testing.
Strategy:
Innovative Defense Technologies (IDT) developed an automated test strategy based on a review of the current manual test procedures and methods used. The strategy identified the testing tasks that were repeated the most often, were the most time consuming, and also the most manpower intensive. The use of automated testing strategies and technologies provided an opportunity to increase the efficiency and the quality of testing of these complex processing systems.
Approach:
IDT utilized a structured system engineering approach to provide a phased implementation to automate the test procedures identified in the automated test strategy. The strategy also identified Automated Test and ReTest (ATRT): Test Manager as the best suited technology to meet the project requirements. The phases are as follows:
(1) System Engineering Phase
The System Engineering phase focused both on determining the most applicable existing test procedures for automation and determining the most efficient manner to design the test automation. This also included identifying common functions between tests to maximize the reuse of automated test actions between test cases.
(2) Integration Phase
During the integration phase ATRT: Test Manager was installed to verify operation with the system under test . Interfaces to target hardware and workstations were verified and a manual conduct of the test also conducted to validate the test procedure and system integrity. Once all integrity checks were made, the test procedures were imported into the ATRT: Test Manager.
(3) Automation Phase
During this phase the existing test procedures and associated requirements that had been imported or entered into ATRT Test Manager served as the basis for developing automation actions.
(4) Verification Phase
This phase allowed the engineer to use ATRT: Test Manager to verify all test cases and their test flows; ensuring they are error free and execute as expected.
(5) Optimization Phase
A phase was included to provide time for optimizing the automated test cases for test execution efficiency and to ensure any special conditions for the test were accounted for.
Conclusion:
Initial results clearly showed that automated software testing reduced the time and people required to run the shipboard the procedures. Another very strong case for applying automation to shipboard testing was not captured in the metrics is the ability to run the automated test identically every time it is executed. The cost of not following written, approved procedures every time would be difficult to capture; however, specific instances, such as human error, illustrated that the cost can be significant.
A reduction in time and manpower of 67% compared to the manual testing approach was able to be achieved following the strategy we outlined and deployed using automated software testing with ATRT: Test Manager.