Mocking Best Practices

When writing unit tests you have to use mocks or stubs for dependant objects. Very often it is convenient to use a mocking library (like RhinoMocks for .NET or jMock for Java). Just using a mocking library does not guarantee to get readable and maintainable unit tests. It make sense to have a set of rules which are guiding developers when writing unit tests with mock objects. I compiled a list of such best practices for mock objects ("mocking" rules). I use the term mock object for a objects verifying expectations about calls to themselves. Test stubs are only used for feeding inputs into the system under test.

Rule 1: Try to avoid using mock objects and prefer state verification over behaviour verification if it is possible.

If you can verify the outcome of an method by checking the return value or by checking the state of the system under test, this is the preferable method, because it is simpler and makes your test more independent from the implementation details. You may need test stubs to feed indirect inputs into the system under test.

Rule 2: Apply the Law of Demeter („no train wrecks”) to your code you want to unit test as much as possible.

Testing code like "employee.GetDepartment().GetManager().GetOffice().GetAddress().GetZip()" is much harder than "employee.GetManagersOfficeZipCode()".

Avoiding "train wrecks" reduces the number of mocks and stubs you need and therefore improves readability and maintainability for your tests.

Rule 3: Use a minimum number of mock objects per test, preferable is only one mock object.

Concentrate on one aspect per test. A reader can identify the most important part more easily. In one test you may check the call to one depended-on component (DOC) and in another test you will check another DOC. You may need additional test stubs to feed indirect inputs into the system under test.

Rule 4: Define a minimum number of expectations as possible.

It’s easier for the reader to see what is important. Tests are less brittle when code changes. Test what code does, not how.

Rule 5: Use Command-Query Separation (Side-Effect-Free Functions) for your code. Don't define expectations on mock objects for queries, only for commands.

Divide all object's methods of your code into two sharply separated categories:

• Queries: Return a result and do not change the observable state of the system (are free of side effects).
• Commands: Change the state of a system but do not return a value.

Queries deliver input for tests, commands are output. Only check the output (what)

Rule 6: At the borderline between your own code towards foreign code, it may be wise to not stub or mock the foreign code directly.

Very often it is better to create an own interface and implement a small layer of adapter code. Test this small layer with integration tests including the foreign code. It is much easier then to stub or mock your own adapter interface when you test your other own code. Examples for foreign code are database access code (like ADO.NET, JDBC in Java, Hibernate, NHibernate), active directory access, network access, and so on.

References:

• Martin Fowler: "Mocks Aren't Stubs"
• Jeremy Miller: "Best and Worst Practices for Mock Objects"
• Steve Freeman, Nat Pryce: "Growing Object-Oriented Software, Guided by Tests"
• Roy Osherove: "Art of Unit Testing"

Testing Java Console Applications

Currently I'm reading the great book "Growing Object-Oriented Software, Guided by Tests" by Steve Freeman and Nat Pryce. They encourage us to drive software development in the large with an outer loop of end-to-end acceptance tests and in the small with an inner loop of unit tests. While implementing just for fun the Nine Men's Morris board game with a simple console user interface, I tried to get a feeling for "test guided growing of software" as it is described in the book.

During that exercise I needed to find a solution to control the input towards and the output from the console. The following example source code shows one possible solution.

I started with an acceptance test. I choose to implement it with JUnit, but Fitnesse tool would have been an alternative.

public class NineMensMorrisAcceptanceTests {

  private ApplicationRunner application = new ApplicationRunner();
  
  @Test
  public void applicationAsksForUserMoveAndThenMakesOwnMove()
  {
    application.startGame();
    application.hasDisplayed("Nine Men's Morris");
    application.hasDisplayed("Please enter spot to place piece:");
    application.userEnters("1\r\n");
    application.hasDisplayed("Computer places piece on spot: 2");
  }
}

The ApplicationRunner class starts the console application in a new thread and acquires control over the input and output streams. Luckily Java has such a well designed IO system, which allows easy test set up. The game application writes to the console via System.out and reads from the console via System.in:

public class ApplicationRunner {

  private PipedOutputStream pipedOutputStream;
  private PipedInputStream pipedInputStream;
  private ByteArrayOutputStream outputStream;

  public ApplicationRunner(){
    pipedOutputStream = new PipedOutputStream();
    pipedInputStream = new PipedInputStream(pipedOutputStream);
    System.setIn(pipedInputStream);

    outputStream = new ByteArrayOutputStream();
    System.setOut(new PrintStream(outputStream));
  }
 
  public void startGame() {
    Thread thread = new Thread("Test Application"){
      @Override public void run(){Console.main(null);}
    };
    thread.setDaemon(true);
    thread.start();
  }

  public void hasDisplayed(String text) {
    boolean displayed = false; int tries = 20;
    while(tries>0 && !displayed){
      Thread.sleep(100);
      displayed = outputStream.toString().contains(text) ? true : false;
      tries--;
    }
    if (!displayed){
      throw new AssertionError("Missing text in output: " + text);
    }
  }

  public void userEnters(String userInput) {
    pipedOutputStream.write(userInput.getBytes());
  }
}

The Console.main() method set ups and starts the console application:

public static void main(String[] args) {
  ConsoleGameUI consoleGameUI = new ConsoleGameUI();
  GameController controller = new GameController(
    consoleGameUI, 
    new Engine(),
    new MoveGenerator());
  consoleGameUI.init(new InputParser(),controller);
  controller.start();
}

When we develop the ConsoleGameUI class, we will write some unit tests. There we can use also the hijacked streams to control inputs and outputs. Because this time the test runs synchronously we can use a ByteArrayInputStream instead of PipedInputStream to supply the user input to the system under test:

public class ConsoleGameUITests {
 
  // Class under test
  ConsoleGameUI consoleGameUI;

  private String userInput;
  private ByteArrayInputStream inputStream;
  private ByteArrayOutputStream outputStream;
  ...
 
  @Before public void setUp() {
    userInput = "some input from user";
    inputStream = new ByteArrayInputStream(userInput.getBytes());
    outputStream = new ByteArrayOutputStream();
    System.setIn(inputStream);
    System.setOut(new PrintStream(outputStream));
    ... 
    consoleGameUI = new ConsoleGameUI();
    consoleGameUI.init(inputParserMock, gameControllerMock);
  }
 
  @Test public void shouldPromptTheUserToEnterSpotToPlaceAPiece(){
    consoleGameUI.askUserForMove(Turn.PLACE_WHITE);
    assertTrue(outputStream.toString().contains("Please enter spot to place piece:"));
  }

  @Test public void shouldPromptTheUserToEnterSpotsToSlidePiece(){
    consoleGameUI.askUserForMove(Turn.SLIDE_WHITE);
    assertTrue(outputStream.toString().contains("Please enter spots to slide piece:"));
  }

  @Test public void shouldReadInputAndCallParser()
  {
    context.checking(new Expectations() {{
      oneOf(inputParserMock).Parse(userInput);
      ...  
    }});
    consoleGameUI.askUserForMove(Turn.PLACE_WHITE);
    context.assertIsSatisfied();
  }
  ...
}

In the ConsoleGameUI class we use System.out and System.in:

public class ConsoleGameUI implements GameUI {

  private final Scanner scanner;
  private IInputParser parser;
  private IGameController gameController;
 
  public ConsoleGameUI(){
    this.scanner = new Scanner(System.in);
  }
 
  public void init(IInputParser parser, IGameController gameController){...}
 
  @Override
  public MoveRequest askUserForMove(Turn turn) {
    switch(turn){
    case PLACE_WHITE:
      System.out.print("Please enter spot to place piece:"); 
      break;
    case SLIDE_WHITE:
      System.out.print("Please enter spots to slide piece:");
      break;
    ...  
    String line = scanner.nextLine();
    MoveRequest request = parser.Parse(line);
    return request;
  }
  ...
}

Rhino Mocks Arrange / Act / Assert (AAA) Syntax Quick Reference

Recently I held a training session about Test-Driven Development(TDD) for .NET developers. A important part in this training was about Mocking, which is essential when you apply TDD to the real world. I presented several examples with Rhino Mocks and used the brilliant AAA syntax of Rhino Mocks. As supporting material for the practical exercises of the participants I missed a quick reference or an API documentation for this syntax style. Based on Ayende's article Rhino Mocks 3.5 and the Rhino Mocks 3.3 Quick Reference I created a new document with code examples:

Rhino Mocks AAA Syntax Quick Reference on Google Docs Rhino Mocks AAA Syntax Quick Reference on Scribd

'Design By Contract' Metrics

We are currently introducing 'Design By Contract' to a software development group of about 60 developers, which are developing different components. We started by defining 'Design By Contract' policies for C# and Java. It is quite challenging to manage this change effort.

One piece of the change strategy is to measure the progress. We are counting the number of classes and the number of contract assertions (Preconditions, post conditions and invariants). So we have two statistics:

1. Absolut number of contract assertions per component
2. Average number of contract assertions per class per component

The metrics tell us whether contracts are "at all" be used. We want to increase the code quality with contracts. If we see a team not implementing any contracts or only very few, we can support the team with training and consulting.

The metrics are published on a regular basis and serve as a means for motivation.

The limitation of thise metrics is, that they do not tell whether a component has enough contracts, so its understandability, maintainability and so on is best supported with 'Design By Contract'. Quality of contracts is not covered by the metrics.

Example for parsing C# code with the NRefactory library

NRefactory is part of the open source IDE SharpDevelop for the .NET platform. NRefactory is a parser library for C# and VB. It can create an Abstract Syntax Tree (AST) that represents all constructs that are available in C# or VB. This AST can be used to analyze source code or to modify and generate code again.

You can download the SharpDevelop IDE, install it and then find the ICSharpCode.NRefactory.dll in the bin folder of the installation. Or you download the SharpDevelop source code and compile the dll yourself.

The example below shows how to parse C# source code files, generate an AST and then using the AST to create metrics about the number of classes and the number of Code Contracts.

using System;
using System.IO;
using System.Diagnostics.Contracts;
using ICSharpCode.NRefactory;
namespace ContractCounter
{
  class Program
  {
    public static void Main(string[] args)
    {
      TextReader reader = File.OpenText("Program.cs");
      using (IParser parser = ParserFactory.CreateParser(SupportedLanguage.CSharp, reader))
      {
        parser.Parse();
        if (parser.Errors.Count <= 0)
        {
          // Here we will use the parser.CompilationUnit(AST)
          ...
        }
        else
        {
          Console.WriteLine("Parse error: " + parser.Errors.ErrorOutput);
        }
      }
      Console.Write("Press any key to continue . . . ");
      Console.ReadKey(true);
    }
  }
}

To traverse the AST we can use the Visitor pattern (see [Gamma et.al:Design Patterns]. We implement a new visitor 'CounterVisitor' for our purposes. We can inherit from the predefined 'AbstractAstVisitor'. In NRefactory visitors are responsible for traversing the AST by themselfs so we have to call the children when we are visiting certain node types:

using System.Diagnostics.Contracts;
using ICSharpCode.NRefactory.Ast;
using ICSharpCode.NRefactory.Visitors;
namespace ContractCounter
{
  public class CounterVisitor : AbstractAstVisitor
  {
    public override object VisitCompilationUnit(CompilationUnit compilationUnit, object data)
    {
      Contract.Requires(compilationUnit != null);
      
      // Visit children (E.g. TypeDcelarion objects)
      compilationUnit.AcceptChildren(this, data);
      
      return null;
    }

    public override object VisitTypeDeclaration(TypeDeclaration typeDeclaration, object data)
    {
      Contract.Requires(typeDeclaration != null);
      
      // Is this a class but not a test fixture?
      if (IsClass(typeDeclaration) && !HasTestFixtureAttribute(typeDeclaration))
      {
        classCount++;
      }

      // Visit children (E.g. MethodDeclarion objects)
      typeDeclaration.AcceptChildren(this, data);
      
      return null;
    }

    public override object VisitMethodDeclaration(MethodDeclaration methodDeclaration, object data)
    {
      Contract.Requires(methodDeclaration != null);

      // Visit the body block statement of method declaration
      methodDeclaration.Body.AcceptVisitor(this, null);

      return null;
    }

    public override object VisitBlockStatement(BlockStatement blockStatement, object data)
    {
      Contract.Requires(blockStatement != null);

      // Visit children of block statement (E.g. several ExpressionStatement objects)
      blockStatement.AcceptChildren(this, data);

      return null;
    }

    public override object VisitExpressionStatement(ExpressionStatement expressionStatement, object data)
    {
      Contract.Requires(expressionStatement != null);

      // Visit the expression of the expression statement (E.g InnvocationExpression)
      expressionStatement.Expression.AcceptVisitor(this, null);

      return null;
    }

    public override object VisitInvocationExpression(InvocationExpression invocationExpression, object data)
    {
      Contract.Requires(invocationExpression != null);

      // Visit the target object of the invocation expression (E.g MemberReferenceExpression)
      invocationExpression.TargetObject.AcceptVisitor(this, null);
      return null;
    }

    public override object VisitMemberReferenceExpression(MemberReferenceExpression memberReferenceExpression, object data)
    {
      Contract.Requires(memberReferenceExpression != null);

      IdentifierExpression identifierExpression = memberReferenceExpression.TargetObject as IdentifierExpression;

      // Is this a call to Contract.Requires(), Contract.Ensures() or Contract.Invariant()?
      if ( identifierExpression != null &&
          identifierExpression.Identifier == "Contract" &&
          (memberReferenceExpression.MemberName == "Requires" ||
           memberReferenceExpression.MemberName == "Ensures" ||
           memberReferenceExpression.MemberName == "Invariant") )
      {
        assertionCount++;
      }

      return null;
    }

    public int ClassCount {
      get { return classCount; }
    }

    public int AssertionCount
    {
      get { return assertionCount; }
    }

    #region private members
    private int classCount;
    private int assertionCount;

    static private bool IsClass(TypeDeclaration typeDeclaration)
    {
      return typeDeclaration.Type == ClassType.Class;
    }

    static private bool HasTestFixtureAttribute(TypeDeclaration typeDeclaration)
    {
      bool hasTestFixtureAttribute = false;
      foreach (AttributeSection section in typeDeclaration.Attributes) {
        foreach (Attribute attribute in section.Attributes) {
          if (attribute.Name == "TestFixture") {
            hasTestFixtureAttribute = true;
            break;
          }
        }
      }
      return hasTestFixtureAttribute;
    }
    #endregion
  }
}

The actual counting takes place in the VisitTypeDeclaration() and the VisitMemberReferenceExpression() methods. All other methods are just neccesary for traversing the tree. We now have to start the vistor to traverse the AST in the Main() method:

...
// Here we will use the parser.CompilationUnit(AST)
CounterVisitor visitor = new CounterVisitor();
parser.CompilationUnit.AcceptVisitor(visitor, null);
Console.WriteLine("The file contains " + visitor.ClassCount + " class(es)");
Console.WriteLine("The file contains " + visitor.AssertionCount + " contract(s)");
...

For exploring the structure of the NRefactory AST you can use the NRefactoryDemo application, which is part of the SharpDevelop source code. You can enter source code and let the application create the according AST:

Stubs vs Mocks with Rhino Mocks

Recently I was asked: "What is the difference between Stubs and Mocks?" Martin Fowler has written a good article about that topic "Mocks Aren't Stubs". Also in the Rhino Mocks Wiki there is is comprehensive explanation.

Besides the theoretical aspect you have to decide whether to use

MockRepository.GenerateMock or

MockRepository.GenerateStub

when you use the "Arrange, Act, Assert" syntax of Rhino Mocks. To find out the detailed differences I wrote some tests, which are all green. I defined expected behaviour or stubbed behaviour with different syntax combinations. The behaviour is not called in any test. Only in the tests with the "ExpectedException" attribute RhinoMocks throws an exception:

[Test]
[(typeof(ExpectationViolationException))]
public void VerifyAllExpectationsWithExpectMethodOnMock()
{
   IList listMock = MockRepository.GenerateMock<IList>();
   listMock.Expect(x => x.Count).Return(10);

   listMock.VerifyAllExpectations();
}

[Test]
public void VerifyAllExpectationsWithExpectMethodOnStub()
{
   IList listStub = MockRepository.GenerateStub<IList>();
   listStub.Expect(x => x.Count).Return(10);

   listStub.VerifyAllExpectations();
}

[Test]
public void VerifyAllExpectationsWithStubMethodOnMock()
{
   IList listMock = MockRepository.GenerateMock<IList>();
   listMock.Stub(x => x.Count).Return(10);

   listMock.VerifyAllExpectations();
}

[Test]
public void VerifyAllExpectationsWithStubMethodOnStub()
{
   IList listStub = MockRepository.GenerateStub<IList>();
   listStub.Stub(x => x.Count).Return(10);

   listStub.VerifyAllExpectations();
}

[Test]
[ExpectedException(typeof(ExpectationViolationException))]
public void AssertWasCalledWithMock()
{
   IList listMock = MockRepository.GenerateMock<IList>();

   listMock.AssertWasCalled(x => { int temp = x.Count; });
}

[Test]
[ExpectedException(typeof(ExpectationViolationException))]
public void AssertWasCalledWithStub()
{
   IList listStub = MockRepository.GenerateStub<IList>();

   listStub.AssertWasCalled(x => { int temp = x.Count; });
}

The conclusion from these tests are: If you need a mocking behaviour in Rhino Mocks that verifies your code under test, you have to use one of these two combinations of methods:

1. GenerateMock with Expect(...) and VerifyAllExpectations()
2. GenerateMock/GenerateStub with AssertWasCalled(...)

LightContracts is a simple, small and lightweight library supporting 'Design by Contract'

Recently I created a project on Codeplex.com. It is called LightContracts and is a simple, small and lightweight library supporting 'Design by Contract' style of programming. It let you specify simple pre- and postconditions with a fluent API, without using an heavy AOP framework. It is developed in C#. In his famous book "Object-Oriented Software Construction" Bertrand Meyer described a design technique called "Design by Contract" (DBC) which can improve software quality dramatically. His programming language Eiffel supports this design technique inherently. This valuable technique is universal and widely accepted and can be used regardless in which programming language you are developing. There are several tools available to use "Design By Contract" on the .NET platform ( E.g. ContractDriven.NET or ContractN). Recently Microsoft announced that its Code Contracts projects would be included as part of .Net 4.0.

So why should I use LightContracts?

All mentioned projects are quite heavyweight. They are using aspect-oriented programming (AOP) frameworks to inject code, require to inherit from a base class or need special tools and compiler settings. My observation is that 90% of all assertions are very simple and do not justify the use of heavy tools. LightContracts is a small DLL which can simply be referenced by any .NET project, without installing additional software. It has a fluent API which allows reading pre- and postconditions nearly like natural language. This allows simply copying these assertions into the method comments.