Testing for exceptions in JUnit revised

In his recent post the author of fantastic mocking framework Mockito collected few rules about testing exceptions. What caught my attention is the advice to use JUnit rules (nomen est omen!) for testing exceptions. ExpectedException rule gathers advantages of both expected @Test attribute clarity and try-catch strictness. Here is the example:

public class DefaultFooServiceTest {

 private FooService fooService = new DefaultFooService();

 @Rule
 public ExpectedException exception = new ExpectedException();

 @Test
 public void shouldThrowNpeWhenNullName() throws Exception {
  //given
  String name = null;

  //when
  exception.expect(NullPointerException.class);
  fooService.echo(name);

  //then
 }

}

Szczepan claims that ExpectedException fits into given/when/then test template nicely. I disagree! Look at the code snippet above – what is the most natural place to put assertions on exception being thrown? From the obvious reasons it must be the last line before the line that actually throws the exceptions. So you have a choice to put assertion as the last statement in given block or as first in when block. You are right, this is how this test should look like in an ideal world:

@Test
public void shouldThrowNpeWhenNullName() throws Exception {
 //given
 String name = null;

 //when
 fooService.echo(name);

 //then
 exception.expect(NullPointerException.class);
}

No we’re talking! There’s just this tiny problem with example above – we expect code in when block to throw an exception and put assertions afterwards in then block. See the problem? If we could somehow transparently catch the exception, store it somewhere, return normally and let assertions to run against it... With AOP it is actually pretty easy, but I wanted to implement this feature using pure Java and with JUnit framework. First, I will introduce @UnderTest marker annotation:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface UnderTest {
}

Now put this annotation above the field in your test case corresponding to class under test:

@UnderTest
private FooService fooService = new DefaultFooService();

Although the annotation brings some value itself, marking which object is actually being tested, it is not meant for documentation and test readability, I am going to use it together with some Java reflection. I mentioned that AOP would solve our problems. JUnit 4.7 ships with AOP-like mechanism called rules. By writing a rule (ExpectedException is an example of a JUnit built-in rule) you simply create an interceptor around every test method. With this interceptor you can, for instance, run test method in separate thread, do some setup and cleanup, etc. My custom rule will do two things:

• wrap class under test in Java proxy to catch every exception, store it and return normally
• verify thrown exception against assertions introduced in then block

Skeleton of the rule code is as follows:

package com.blogspot.nurkiewicz.junit.exceptionassert;

public class ExceptionAssert implements MethodRule {

 @Override
 public Statement apply(Statement base, FrameworkMethod method, Object testCase) {
  this.testCase = testCase;
  return new ExceptionAssertStatement(base);
 }

 private class ExceptionAssertStatement extends Statement {

  private final Statement base;

  private Throwable exceptionThrownFromClassUnderTest;
  private Field underTestField;

  public ExceptionAssertStatement(Statement base) {
   this.base = base;
   underTestField = findClassUnderTestField(testCase);
  }

  @Override
  public void evaluate() throws Throwable {
   final Object originalClassUnderTest = wrapClassUnderTest(testCase);
   try {
    base.evaluate();
   } finally {
    setUnderTestField(originalClassUnderTest);
   }
   verifyException();
  }

}

ExceptionAssertStatement is an example of Decorator pattern: it takes original Statement instance (representing test method execution) and replaces it with wrapped (decorated) custom class, also implementing Statement. ExceptionAssertStatement adds some additional logic and calls original’s statement evaluate() method. This additional logic is pretty straightforward:

• first, take class under test and wrap it (wrapping again!) in a proxy
• execute the test method (evaluate())
• (revert to original class under test)
• when test method exits, verify exception throw from class under test (if any)

The last interesting piece is the proxy wrapping class under test itself:

private Object wrapWithProxy(final Object classUnderTest) {
 return Proxy.newProxyInstance(classUnderTest.getClass().getClassLoader(), new Class[]{underTestField.getType()}, new InvocationHandler() {
  @Override
  public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
   try {
    return method.invoke(classUnderTest, args);
   } catch (InvocationTargetException e) {
    exceptionThrownFromClassUnderTest = e.getCause();
    return null;
   }
  }
 });
}

Nothing fancy: if class under test throws an exception, store it somewhere and return normally. Not that hard. Enough of the internals, let’s look at our brand new rule in action:

public class DefaultFooServiceTest {

 @UnderTest
 private FooService fooService = new DefaultFooService();

 @Rule
 public ExceptionAssert exception = new ExceptionAssert();

 @Test
 public void shouldReturnHelloString() throws Exception {
  //given
  String name = "Tomek";

  //when
  final String result = fooService.echo(name);

  //then
  assertEquals("Hello, Tomek!", result);
 }

 @Test
 public void shouldThrowNpeWhenNullName() throws Exception {
  //given
  String name = null;

  //when
  fooService.echo(name);

  //then
  exception.expect(NullPointerException.class);
 }

 @Test
 public void shouldThrowIllegalArgumentWhenNameJohn() throws Exception {
  //given
  String name = "John";

  //when
  fooService.echo(name);

  //then
  exception.expect(IllegalArgumentException.class)
    .expectMessage("Name: 'John' is not allowed");
 }

}

First test method does not expect any exception to be thrown – it if will, test will fail. Second test expects NullPointerException. Please note that if the exception will be thrown from any other line than fooService.echo() (or fooService.echo() won’t throw NullPointerException), test will fail. The last test shows that you can also assert exception message as well.

Few things are still missing in 0.0.1 "version", mainly proxying classes (CGLIB, anyone?); also, some syntactic sugar together with DSL-like (FEST-like) assertions could be introduced:

@Test
public void shouldThrowIllegalArgumentWhenNameJohn() throws Exception {
 //given
 String name = "John";

 //when
 fooService.echo(name);

 //then
 expect(IllegalArgumentException.class)
   .withMessage("Name: 'John' is not allowed");
}

Also I had to copy&paste big parts of JUnit’s ExpectedException, as most obvious inheritance was not possible due to private constructor. But still, take a look at ExceptionAssert rule and see for yourself how readable exception testing can be. As always, source code can be cloned and downloaded from my GitHub account. Any comments and contributions are welcome!