Background:
This effort was focused on using automation and automated software testing strategies to support network interface test procedures to verify shipboard networks and configurations. The configuration included over 90 mission critical devices connected via a network of networks onboard the ship. The test comprised of node availability tests; port status; network path verification, security checks, and other status verifications. This case study will show the benefits of automation of shipboard test procedures in significantly reducing crew workload while also increasing the overall test coverage and quality. Furthermore, the use of automated software testing procedures enabled the test team to capture considerably more test results than when using previous manual testing methods.
Strategy:
An automated test strategy was developed based on a review of the current manual network test procedures and methods used. The strategy identified and focused on the network testing tasks that were repeated most often, were the most time consuming, and also the most manpower intensive. The high-level objectives were to:
- Reduce the number of people required to travel to shipyards/ports to conduct network tests
- Reduce days of testing (e.g. time ship needs to be in shipyard)
- Identify problems sooner and more precisely to expedite fix
Approach:
Consistent with IDT’s automation methodology, a structured system engineering approach was used to provide a phased implementation to automate the test procedures identified in the automated test strategy. The strategy also identified the ATRT: Test Manager as the best suited technology to meet the project requirements. The phases are as follows:
(1) System Engineering Phase
The System Engineering phase focused both on determining the most applicable existing test procedures for automation and determining the most efficient manner to design the test automation. Some of the steps involved included:
- Review Test Procedures
- Analyze Potential Issues
- Determine Automation Strategy
- Design Prototype
(2) Integration Phase
Integration phase verified operation with the system under test. In this case the system was the shipboard network interface. Once all integrity checks were made, the test procedures, functions, and prototypes were imported into the ATRT: Test Manager.
(3) Automation Phase
During this phase the existing test procedures and associated requirements that had been imported or entered into ATRT: Test Manager served as the basis for developing automation actions. The prototype from the Integration Phase proved to be very instrumental in allowing the engineer time to optimize the automation.
(4) Verification Phase
This phase verified all network interface connectivity. This also allowed the engineer to use ATRT: Test Manager to verify all test cases and their test flows; ensuring they are error free and execute as expected.
(5) Optimization Phase
The automated test cases were optimized for test execution efficiency and any special system conditions or test conditions necessary to support the effort.
An optimization phase was necessary to provide time for adjusting (if needed) the automated test cases for test execution efficiency and to ensure any special conditions for the test were accounted for.
Conclusion:
Results demonstrated that automated software testing reduced the time and people required to run the shipboard network interface test procedures. Furthermore, the use of ATRT: Test Manager provided a far more thorough detailed metrics reporting mechanism.
A reduction in time and manpower of 76% compared to the manual testing approach was able to be achieved following the strategy we outlined and deployed using automated software testing with ATRT: Test Manager.