The Role of Automated Testing in Continuous Integration

By Jeff Rose—Over the last few years, software companies have looked to continuous integration (CI) as a way to increase the quality of their products.  The practices of CI also enable organizations to get more from their development, test and quality assurance (QA) teams using their existing resources.  Truly making CI worth the effort requires adopting strong automated testing tools and methodologies as part of the CI package.

Implementing principles of continuous integration enables teams to deliver more reliable software by increasing their efficiency.  Running a build automatically each night can narrow the focus for build problems down to one day’s worth of commits.  Many teams using CI also run unit tests on code to further help in debugging code issues and to lessen the burden on developers.

Nightly Regression Tests: An Achievable Goal?

Using unit tests provides spot checks of code correctness, but a full regression test is required to ensure functionality and build quality.  Without adding additional resources, most teams will have difficulty achieving a nightly regression test every day of the week.  Attempting to do so could overload a test team just to run the tests, not to mention preparing reports for developers to assess problems. Performing a full regression test on every nightly build without adding resources is an achievable goal – if automation is used in the process.

Enter ATRT: Test Manager

IDT’s automated testing solution, ATRT: Test Manager, allows test teams to create robust regression tests that can be run against each new build in a fraction of the time it takes to run these tests manually.  Using ATRT: Test Manager, the same set of tests can be run each night, evaluating the same functionality for correctness each time they are run.  If a problem is found by the regression test, developers can run portions of the automated test to recreate the problem, identify a patch, and test their fix, without any ambiguity in the steps to reproduce the defect.  Reports are generated as soon as the test is completed, so no manual reporting time is required to evaluate test results.  Lastly, ATRT: Test Manager’s tests can be integrated into a suite of nightly tasks using a simple command line interface, enabling teams to kick off automated tests without any human intervention.

Continuous integration outlines valuable practices to reduce the number of escaped bugs in each release.  Automated testing plays an important role in successful CI applications, both in code coverage from unit tests and regression testing.  Including ATRT: Test Manager as a regression and QA testing  tool can augment your CI suite by providing daily reports from robust, consistent, and repeatable automated test procedures, all in less time and while using fewer test resources.

Jeff Rose is a Software Engineer at Innovative Defense Technologies (IDT).  He is a contributing member of the ATRT: Test Manager development team.