As my awareness around automated testing, and particularly TDD, has risen over the last few years I’ve been keeping my ear to the ground for new ideas and techniques I can experiment with.

I was recently at a meetup hosted by Equal Experts and there was a talk by Oliver Shaw on what “Doing TDD” is, and isn’t. The talk provided a great live demo of TDD exploring a fairly simple use-case but showing how natural the TDD process can be. The demo was done using Scala and he used Vim as his IDE which was impressive and very effective.

The relevance to this post is that he was using an SBT task (~test-quick) to continuously watch for file changes and automatically run the unit tests against them when they changed. This allowed him to remain focused on the code and the cycles involved in TDD rather than breaking flow through repeatedly having to manually run the tests.

I like the idea of this! Anything that can help to keep focus and make the process more efficient is worth a try.

What is continuous unit testing?⌗

Continuous unit testing is basically the idea that every time you make a change to your code, be that functional or test code, the tests that are associated with that change will automatically run. This is not be confused with Continuous Testing as part of a build pipeline.

Continuous unit testing tools or frameworks use the concept of a test watcher.

The test watcher runs in the background and watches for changes on the source files you are working on. When a file changes the test watcher triggers the execution of any related tests automatically.

The standard TDD cycle, as per Wikipedia, is:

  • Add a test
  • Run all tests to see if the new test fails
  • Write the code
  • Run tests
  • Refactor code
  • Repeat

Using a continuous unit testing approach, the cycle becomes:

  • Add a test (tests run automatically and see new test fail)
  • Write the code (tests run automatically and hopefully see all tests pass)
  • Refactor (tests run automatically and hopefully continue to see all tests pass)
  • Repeat

Each time the tests run automatically, any failures are brought to the developers attention to be dealt with immediately. A fix can be applied, or the change that caused the failure reverted, at which point the tests automatically run again until you eventually get to the position of all tests passing.

The continuous unit testing approach has reduced the number of steps (not including Repeat) in the TDD cycle from 5 to 3.

Some benefits of this are:

  • reduced context switching due to not having to manually run the tests
  • shorter cycles (feedback loops) between making a change and finding if a test has passed, or failed

There are various tools available depending on your language of choice. The natural choice for Scala is likely SBT as mentioned above. This can also be used for Java based projects however another option that integrates with my IDE of choice, IntelliJ, is Infinitest.

Infinitest⌗

Infinitest provides plugins for both Eclipse and IntelliJ. We will look at how it works with IntelliJ for the purposes of this post.

The project’s website describes what Infinitest does very nicely:

Each time a change is made on the source code, Infinitest runs all the tests that might fail because of these changes.

The documentation is pretty brief but from my experience it covers everything you need to know to get started and configure it in various ways.

IntelliJ Setup⌗

The first thing to do is install the plugin in IntelliJ. It can be found easily by searching the plugin repos.

Once the plugin is installed, it needs to be configured for your project. This is achieved by adding a facet to the project. Navigate to the Project Structure view in IntelliJ (⌘; on MacOS), select Facets and click the add (+) button. Select Infinitest from the list and associate it with the module(s) in your project that you want Infinitest to run against.

Infinitest facet

A new tool window should now be visible in the bottom panel of your IDE that indicates that Infinitest is watching for changes.

Infinitest window start

If configuring Infinitest for a brand new project the window will be empty as there won’t yet be any tests. When adding Infinitest to an existing project the window may also be empty as Infinitest will only react to changes when the files you are changing are compiled.

To support the continuous unit testing process, it makes sense to enable the “compile on save” feature in your IDE to ensure that the tests run automatically.

In IntelliJ this means enabling the feature to build the project automatically.

Let’s write a failing test and take a another look at the window.

@Test
public void thisFails() {
   assertEquals(1, 2);
}

On saving the test file, the Infinitest window automatically refreshes, we see red and details of the test failure.

Infinitest window failure

Fixing the test and saving causes the window to automatically refresh again and we see green.

Infinitest window passing

That’s it, you’re good to go with your continuous unit testing setup.

More on test failures⌗

When a test fails you obviously want to understand why and fix it as soon as possible.

Double clicking the failure in the Infinitest window will take you straight to the line of code that caused the test to fail, usually an assertion but possibly an exception.

In addition to this, Infinitest will highlight the line of code that is failing by adding its icon to the side panel. Hovering over the icon will display the reason the test failed. Nice.

Infinitest failure icon

Filtering tests⌗

In many applications, there will be a mix of tests of different styles such as unit, integration and load. In general, it is unlikely you will want to automatically want run integration or load tests every time a file changes that they depend on due to them being slow running or requiring more complex external setup.

Infinitest provides the ability to filter tests for this exact reason. Simply adding an infinitest.filters file to the root of your project and adding regular expressions (one per line) will filter out the tests that match.

A simple example to filter out integration tests, assuming they have names such as MyAppIT, is to create a file with the following content:

.*IT

Filters can also be applied to packages using a regex such as:

com\.nickebbitt\.it\..*

If using TestNG rather than JUnit there is also the option to filter tests using TestNG groups.

To summarise, if you are following a TDD approach and are looking to shorten the cycles, or even just writing unit tests and want quicker feedback, then following a continuous unit testing approach can help achieve this.

If you’re developing in Java and using IntelliJ, or Eclipse, then Infinitest is your friend. It’s simple to setup and provides exactly the features you need to start continuously unit testing on your projects.