Using TackleTest-Unit to automatically create Java unit tests

The CIO organization at IBM manages complex enterprise IT systems, delivering and supporting the applications that run IBM’s critical business functions, equipping more than 250,000 IBMers with the tools and technologies they need to get their jobs done. The organization manages and maintains over 4,000 applications globally, across a complex hybrid-cloud environment that encompasses multiple computing architectures, from cloud-based to on-premises systems, as well as worldwide development, deployment, and management of systems with an always-on approach.

The CIO organization provides developer productivity tools, a key asset for IBM’s enormous developer community, which includes several thousand developers that create and maintain the IBM CIO portfolio. One of the main challenges facing these development teams is the need to modernize applications to take full advantage of the hybrid cloud. Before a team can begin moving an application to a new cloud environment, or begin refactoring an application to take advantage of hybrid cloud features, they need to determine whether the application, in its new environment or new form, still reliably fulfills the same business functions as the previous application.

The IBM CIO organization found that only 20% of CIO application teams used automated unit testing tools as part of their testing strategy, and many of CIO custom Java applications surveyed had outdated unit tests or inadequate unit test coverage.

To be successful in migrating and modernizing applications to IBM’s modern hybrid cloud platform, proper unit testing is critical. Having adequate automated test cases in place on the existing application makes the modernization process much more efficient and faster than relying on manual testing.

When asked why they don’t write automated unit tests, many developers indicated that writing test cases is time consuming. Developers are often under time constraints to deliver code quickly; writing or updating unit test cases often gets skipped or deferred until later. Often, later never comes and the unit test cases become more out of date.

This cycle of outdated tests increases overall costs because business users must spend additional time to validate the application changes. This takes them away from their business responsibilities. If they’re also time constrained, which is often the case, then the possibility exists for defects to escape into production.

Generate unit tests automatically

IBM’s solution is to generate unit tests automatically by evaluating the Java application classes. TackleTest-Unit automatically generates unit test cases for Java applications. It’s part of the open-source Konveyor project created by IBM Research and Red Hat to enable teams to modernize their applications and move them to containers.

Each test case generated by TackleTest-Unit focuses on exercising the visible methods of an application class. Test generation can be done on a specific set of application classes or on the entire application. These are some of TackleTest-Unit’s features:

• Supports automatic generation of test assertions: tests can be generated with or without assertions
• Combines different coverage metrics and generates coverage reports: the supported coverage metrics include conventional code coverage criteria, such as statement and branch coverage, as well as a novel type-based combinatorial coverage of Java methods
• Accommodates developer-written unit test suites: it can perform coverage-driven augmentation and directed test generation to enhance the coverage achieved by a given test suite
• Supports multiple build tools: Apache Maven, Gradle Build Tool, and Apache Ant
• Integrates two open-source test-generation tools, EvoSuite and Randoop

Tests can be generated using either of these tools as stand-alone or in combination with combinatorial test design (CTD) modeling, which generates test cases guided by a different coverage criterion that exercises methods based on combinations of different types of method parameters. The goal of CTD-guided test generation is to exercise a method with different combinations of subtypes of the declared types of the method parameters with as few test cases as possible.

Installation, configuration, and execution

You can install TackleTest-Unit for test generation either by using a pre-built Docker container or through a command line option. Once you’ve completed the installation, you’re now ready to proceed to the next step. There are three steps to use the TackleTest-Unit Generator:

Once you’ve completed the configuration file, you’re ready to generate the test cases. For a large application, test generation can run for hours. Here are example times for one of the enterprise Java applications from the IBM CIO organization:

• Classes: 387 and Methods: 1428
• Test plan size: 12,297
• Test cases: ~ 5400 test cases (covering 180 classes) generated in about one hour

There are three report types generated by TackleTest-Unit:

• CDT Coverage Report (if you selected ctd-amplified as the test generation strategy)
• Java Code Coverage (JaCoCo) Report
• JUnit Report with execution information and results

Integrate unit tests into the build pipeline

The unit tests generated are standard JUnit tests that are quick to integrate into your build process. Move the generated JUnit tests to the application repository and check them into GitHub. Having the unit tests integrated into your build process facilitates the identification of defects early when the developer is most familiar with code (shift-left) and when they are cheap to fix.

TackleTest-Unit is also a great method for validating applications when they’re moved from the existing environment to a new cloud environment. Simply generate the tests against the current environment and then run the tests against the new environment to validate successful migration.

With TackleTest-Unit teams can do the following:

• Generate your JUnit test cases in just a few hours. And this only happens once.
• Go from no unit test code coverage to 35% or higher code on Day 1
• Receive out-of-the-box test coverage reports that show your progress
• Easily integrate with existing Continuous Integration/Continuous Deployment (CI/CD) pipelines or the new CIO organization CI/CD pipeline