By Phil Monte—Many commercial and government test programs place a heavy emphasis on the verification of application behavior from the perspective of the end-user. The approach taken to this verification is to apply testing against the end-user interface. For the past eight years, IDT has provided customers with an enterprise solution to automated GUI-based testing. Customers who have adopted this solution have revolutionized their test programs with productivity gains in excess of 75%.
While there is excellent value and return-on-investment when adopting an automated user interface test approach, IDT’s ATRT: Test Manager can also provide a deeper level of application verification and validation. This automated testing solution offers advanced message testing capabilities that allow test engineers and application developers to bring their automated verification and validation a level deeper than GUI-based testing.
An outline of the automated message testing capabilities integrated into ATRT: Test Manger is provided below.
Message Control Logic:
Message control logic provides the capability to create automated test cases that involve any of the supported message paradigms (Sockets-UDP/TCP, CORBA, DDS, SOAP, etc.). This allows for:
- Decision logic to be defined (state-based conditional etc.)
- Responses to incoming messages
- Sending messages at specified rates and times
- Message actions to communicate with the rest of the test flow
Message Data Manipulation:
Message data manipulation provides the capability to define message payloads and manipulate the payloads dynamically as follows:
- Import IDL
- Auto-generate data for message testing according to parameters such as range and precision
- Extract data from message payload for test case usage
- Facilitate automated creation of payloads using data from received messages or other sources within ATRT: Test Manager
- Set all or specific payload values during test execution
- Provide ability to add dynamic data to payloads such as timestamps
Message Analysis:
Message analysis provides the user with capabilities to verify actual message values against expected message values. This feature allows an entire message or just portions of a message to be analyzed as follows:
- Compare message values against statically defined values
- Manually entered
- Imported from external files
- Compare message values against dynamically generated values
- Values displayed on user interface (using OCR)
- Range and tolerance data
- Data provided via external data sources
- Data from received messages
- Provide different analysis types
- Exact match, tolerance match
- Verify message latency and periodicity
Message Reporting:
Message reporting provides the user with capabilities to view message analysis results and aid in filtering the raw data results into knowledge about the message analysis outcome, including:
- Chronological message report to view all messages transmitted in the context of a test case, interleaved by time rather than type
- Message instance view that displays the actual and expected results and provides filters and that allow the user to easily and quickly navigate through the sets of message data
Combining these messaging capabilities with a robust GUI-based test approach enables deeper verification and validation of software applications. It also provides the ability to develop inter-application/inter-system test development. Lastly, it allows for verification methods to migrate beyond a demonstration to more formal test and analysis.
If your application/system under test makes use of message communications, ATRT: Test Manager would be an invaluable tool to adopt in your test program. Ultimately, it will help lead to increased system reliability and customer confidence in your products.
For more information on ATRT: Test Manager and our other automated software testing solutions, please contact us.
Phil Monte is a Senior Systems Engineer at Innovative Defense Technologies (IDT). He spends most of his time focused on engineering automated test solutions for the Navy’s Submarine Programs.