'Design By Contract' Metrics

We are currently introducing 'Design By Contract' to a software development group of about 60 developers, which are developing different components. We started by defining 'Design By Contract' policies for C# and Java. It is quite challenging to manage this change effort.

One piece of the change strategy is to measure the progress. We are counting the number of classes and the number of contract assertions (Preconditions, post conditions and invariants). So we have two statistics:

1. Absolut number of contract assertions per component
2. Average number of contract assertions per class per component

The metrics tell us whether contracts are "at all" be used. We want to increase the code quality with contracts. If we see a team not implementing any contracts or only very few, we can support the team with training and consulting.

The metrics are published on a regular basis and serve as a means for motivation.

The limitation of thise metrics is, that they do not tell whether a component has enough contracts, so its understandability, maintainability and so on is best supported with 'Design By Contract'. Quality of contracts is not covered by the metrics.

Example for parsing C# code with the NRefactory library

NRefactory is part of the open source IDE SharpDevelop for the .NET platform. NRefactory is a parser library for C# and VB. It can create an Abstract Syntax Tree (AST) that represents all constructs that are available in C# or VB. This AST can be used to analyze source code or to modify and generate code again.

You can download the SharpDevelop IDE, install it and then find the ICSharpCode.NRefactory.dll in the bin folder of the installation. Or you download the SharpDevelop source code and compile the dll yourself.

The example below shows how to parse C# source code files, generate an AST and then using the AST to create metrics about the number of classes and the number of Code Contracts.

using System;
using System.IO;
using System.Diagnostics.Contracts;
using ICSharpCode.NRefactory;
namespace ContractCounter
{
  class Program
  {
    public static void Main(string[] args)
    {
      TextReader reader = File.OpenText("Program.cs");
      using (IParser parser = ParserFactory.CreateParser(SupportedLanguage.CSharp, reader))
      {
        parser.Parse();
        if (parser.Errors.Count <= 0)
        {
          // Here we will use the parser.CompilationUnit(AST)
          ...
        }
        else
        {
          Console.WriteLine("Parse error: " + parser.Errors.ErrorOutput);
        }
      }
      Console.Write("Press any key to continue . . . ");
      Console.ReadKey(true);
    }
  }
}

To traverse the AST we can use the Visitor pattern (see [Gamma et.al:Design Patterns]. We implement a new visitor 'CounterVisitor' for our purposes. We can inherit from the predefined 'AbstractAstVisitor'. In NRefactory visitors are responsible for traversing the AST by themselfs so we have to call the children when we are visiting certain node types:

using System.Diagnostics.Contracts;
using ICSharpCode.NRefactory.Ast;
using ICSharpCode.NRefactory.Visitors;
namespace ContractCounter
{
  public class CounterVisitor : AbstractAstVisitor
  {
    public override object VisitCompilationUnit(CompilationUnit compilationUnit, object data)
    {
      Contract.Requires(compilationUnit != null);
      
      // Visit children (E.g. TypeDcelarion objects)
      compilationUnit.AcceptChildren(this, data);
      
      return null;
    }

    public override object VisitTypeDeclaration(TypeDeclaration typeDeclaration, object data)
    {
      Contract.Requires(typeDeclaration != null);
      
      // Is this a class but not a test fixture?
      if (IsClass(typeDeclaration) && !HasTestFixtureAttribute(typeDeclaration))
      {
        classCount++;
      }

      // Visit children (E.g. MethodDeclarion objects)
      typeDeclaration.AcceptChildren(this, data);
      
      return null;
    }

    public override object VisitMethodDeclaration(MethodDeclaration methodDeclaration, object data)
    {
      Contract.Requires(methodDeclaration != null);

      // Visit the body block statement of method declaration
      methodDeclaration.Body.AcceptVisitor(this, null);

      return null;
    }

    public override object VisitBlockStatement(BlockStatement blockStatement, object data)
    {
      Contract.Requires(blockStatement != null);

      // Visit children of block statement (E.g. several ExpressionStatement objects)
      blockStatement.AcceptChildren(this, data);

      return null;
    }

    public override object VisitExpressionStatement(ExpressionStatement expressionStatement, object data)
    {
      Contract.Requires(expressionStatement != null);

      // Visit the expression of the expression statement (E.g InnvocationExpression)
      expressionStatement.Expression.AcceptVisitor(this, null);

      return null;
    }

    public override object VisitInvocationExpression(InvocationExpression invocationExpression, object data)
    {
      Contract.Requires(invocationExpression != null);

      // Visit the target object of the invocation expression (E.g MemberReferenceExpression)
      invocationExpression.TargetObject.AcceptVisitor(this, null);
      return null;
    }

    public override object VisitMemberReferenceExpression(MemberReferenceExpression memberReferenceExpression, object data)
    {
      Contract.Requires(memberReferenceExpression != null);

      IdentifierExpression identifierExpression = memberReferenceExpression.TargetObject as IdentifierExpression;

      // Is this a call to Contract.Requires(), Contract.Ensures() or Contract.Invariant()?
      if ( identifierExpression != null &&
          identifierExpression.Identifier == "Contract" &&
          (memberReferenceExpression.MemberName == "Requires" ||
           memberReferenceExpression.MemberName == "Ensures" ||
           memberReferenceExpression.MemberName == "Invariant") )
      {
        assertionCount++;
      }

      return null;
    }

    public int ClassCount {
      get { return classCount; }
    }

    public int AssertionCount
    {
      get { return assertionCount; }
    }

    #region private members
    private int classCount;
    private int assertionCount;

    static private bool IsClass(TypeDeclaration typeDeclaration)
    {
      return typeDeclaration.Type == ClassType.Class;
    }

    static private bool HasTestFixtureAttribute(TypeDeclaration typeDeclaration)
    {
      bool hasTestFixtureAttribute = false;
      foreach (AttributeSection section in typeDeclaration.Attributes) {
        foreach (Attribute attribute in section.Attributes) {
          if (attribute.Name == "TestFixture") {
            hasTestFixtureAttribute = true;
            break;
          }
        }
      }
      return hasTestFixtureAttribute;
    }
    #endregion
  }
}

The actual counting takes place in the VisitTypeDeclaration() and the VisitMemberReferenceExpression() methods. All other methods are just neccesary for traversing the tree. We now have to start the vistor to traverse the AST in the Main() method:

...
// Here we will use the parser.CompilationUnit(AST)
CounterVisitor visitor = new CounterVisitor();
parser.CompilationUnit.AcceptVisitor(visitor, null);
Console.WriteLine("The file contains " + visitor.ClassCount + " class(es)");
Console.WriteLine("The file contains " + visitor.AssertionCount + " contract(s)");
...

For exploring the structure of the NRefactory AST you can use the NRefactoryDemo application, which is part of the SharpDevelop source code. You can enter source code and let the application create the according AST:

Stubs vs Mocks with Rhino Mocks

Recently I was asked: "What is the difference between Stubs and Mocks?" Martin Fowler has written a good article about that topic "Mocks Aren't Stubs". Also in the Rhino Mocks Wiki there is is comprehensive explanation.

Besides the theoretical aspect you have to decide whether to use

MockRepository.GenerateMock or

MockRepository.GenerateStub

when you use the "Arrange, Act, Assert" syntax of Rhino Mocks. To find out the detailed differences I wrote some tests, which are all green. I defined expected behaviour or stubbed behaviour with different syntax combinations. The behaviour is not called in any test. Only in the tests with the "ExpectedException" attribute RhinoMocks throws an exception:

[Test]
[(typeof(ExpectationViolationException))]
public void VerifyAllExpectationsWithExpectMethodOnMock()
{
   IList listMock = MockRepository.GenerateMock<IList>();
   listMock.Expect(x => x.Count).Return(10);

   listMock.VerifyAllExpectations();
}

[Test]
public void VerifyAllExpectationsWithExpectMethodOnStub()
{
   IList listStub = MockRepository.GenerateStub<IList>();
   listStub.Expect(x => x.Count).Return(10);

   listStub.VerifyAllExpectations();
}

[Test]
public void VerifyAllExpectationsWithStubMethodOnMock()
{
   IList listMock = MockRepository.GenerateMock<IList>();
   listMock.Stub(x => x.Count).Return(10);

   listMock.VerifyAllExpectations();
}

[Test]
public void VerifyAllExpectationsWithStubMethodOnStub()
{
   IList listStub = MockRepository.GenerateStub<IList>();
   listStub.Stub(x => x.Count).Return(10);

   listStub.VerifyAllExpectations();
}

[Test]
[ExpectedException(typeof(ExpectationViolationException))]
public void AssertWasCalledWithMock()
{
   IList listMock = MockRepository.GenerateMock<IList>();

   listMock.AssertWasCalled(x => { int temp = x.Count; });
}

[Test]
[ExpectedException(typeof(ExpectationViolationException))]
public void AssertWasCalledWithStub()
{
   IList listStub = MockRepository.GenerateStub<IList>();

   listStub.AssertWasCalled(x => { int temp = x.Count; });
}

The conclusion from these tests are: If you need a mocking behaviour in Rhino Mocks that verifies your code under test, you have to use one of these two combinations of methods:

1. GenerateMock with Expect(...) and VerifyAllExpectations()
2. GenerateMock/GenerateStub with AssertWasCalled(...)

LightContracts is a simple, small and lightweight library supporting 'Design by Contract'

Recently I created a project on Codeplex.com. It is called LightContracts and is a simple, small and lightweight library supporting 'Design by Contract' style of programming. It let you specify simple pre- and postconditions with a fluent API, without using an heavy AOP framework. It is developed in C#. In his famous book "Object-Oriented Software Construction" Bertrand Meyer described a design technique called "Design by Contract" (DBC) which can improve software quality dramatically. His programming language Eiffel supports this design technique inherently. This valuable technique is universal and widely accepted and can be used regardless in which programming language you are developing. There are several tools available to use "Design By Contract" on the .NET platform ( E.g. ContractDriven.NET or ContractN). Recently Microsoft announced that its Code Contracts projects would be included as part of .Net 4.0.

So why should I use LightContracts?

All mentioned projects are quite heavyweight. They are using aspect-oriented programming (AOP) frameworks to inject code, require to inherit from a base class or need special tools and compiler settings. My observation is that 90% of all assertions are very simple and do not justify the use of heavy tools. LightContracts is a small DLL which can simply be referenced by any .NET project, without installing additional software. It has a fluent API which allows reading pre- and postconditions nearly like natural language. This allows simply copying these assertions into the method comments.

What is Design by Contract?

This is the first article in a series which I started after I reasoned about "Why is Design by Contract not common practice in Software Enineering?".

Design by Contract is a technique to specify the behavior of a class. It helps to communicate what effects methods will have and what the methods expect before they can be executed. Design by Contract means that the caller of a class and the class itself make a contract, which is described as a set of assertions called Preconditions, Postconditions and Invariants.

So the interface and the implementation of a class is enhanced with additional assertions. Preconditions define obligations for the caller of a method, which have to be satisfied before the method is called. Postconditions will guarantee the outcome of the method. Invariants apply to the class as a whole and define conditions which are valid at the end of each method call.

I show an example in C# for preconditions and postconditions. Preconditions are commonly describes with the term 'require' and postconditions with the term 'ensure'. The property PrinterNames has the postcondition that the delivered list is not null. The method RemovePrinter has three preconditions:

public class Printers
{
    private List<string> names;
    
    public Printers()
    {
        names = new List<string>();
        names.Add("printer1");
        names.Add("printer2");
    }
    
    /// <summary>
    /// List of printer names.
    /// Assertion.Ensure(names != null,"Result is not null");
    /// </summary>
    public List<string> PrinterNames
    {
        get
        {
            Assertion.Ensure(names != null,"Result is not null");
            return names;
        }
    }
    
    /// <summary>
    /// Remove a printer
    /// Assertion.Require(PrinterNames().Count > 0,"There is at least one printer");
    /// Assertion.Require(printerIndex >= 0,"printerIndex is not negative");
    /// Assertion.Require(printerIndex lessThan PrinterNames().Count ,"printerIndex is in range");
    /// </summary>
    public void RemovePrinter(int printerIndex)
    {
        Assertion.Require(PrinterNames.Count > 0,"There is at least one printer");
        Assertion.Require(printerIndex >= 0,"printerIndex is not negative");
        Assertion.Require(printerIndex < PrinterNames.Count ,"printerIndex is in range");
    
        names.RemoveAt(printerIndex);
    }

    ...
}

For further information about Design by Contract please refer to the Eiffel web site or the book Object-Oriented Software Construction

In the next article I will try to answer the question: What are the benefits of Design by Contract?

Why is Design by Contract not common practice in Software Enineering

Concerned with a lot of source code reviews over the last years, I have recognized that Design by Contract is not a common practice in Software Engineering. Nearly all C++, Visual Basic, Java and C# source code I saw was designed and written without applying Design by Contract. Thinking about the reasons for programmers not to use Design by Contract I made a list with questions they may have:

• What is Design by Contract?
• What are the benefits of Design by Contract?
• Do I need to use Eiffel as my programming language to apply Design by Contract?
• Do I have to spend a lot of effort for Design by Contract?
• In which situations should I use Design by Contract?
• Does Design by Contract conflict with other design and implementation techniques like TDD and DDD?

If you have no or wrong answers for these questions you will probably not use Design by Contract in your day to day programming work. I will try to answer these questions in a series of articles and hope it will convince more programmers that Design by Contract is an elegant technique to improve software quality.

Preconditions, Postconditions: Design by Contract for C#

Bertrand Meyer has described a design technique called "Design by Contract" (DBC) in his book Object-Oriented Software Construction. He also added support for this technique in his programming language Eiffel. Also other languages do not have a native support for this valuable technique, we nevertheless can apply this technique in all other languages. What we need is the possibility to define and execute assertions at runtime and we have to document the pre-, and postconditions in the header of our methods. For C# we only need a small helper class (comments are omitted):

public static class Assertion
{
    public static void Require(bool precondition, string conditionDescription)
    {
        if (!precondition)
        {
            throw new AssertException(ExceptionDescription("Precondition", conditionDescription));
        }
    }

    public static void Require(bool precondition, string descriptionFormat, params object[] descriptionParameters)
    {
        if (!precondition)
        {
            throw new AssertException(ExceptionDescription("Precondition", string.Format(CultureInfo.InvariantCulture, descriptionFormat, descriptionParameters)));
        }
    }

    public static void RequireIsNotNull(object toBeTested,string objectName)
    {
        if (toBeTested==null)
        {
            throw new AssertException(ExceptionDescription("Precondition", objectName + " is not null"));
        }
    }

    public static void Ensure( bool postcondition, string conditionDescription )
    {
        if ( ! postcondition )
        {
            throw new AssertException(ExceptionDescription("Postcondition",conditionDescription));
        }
    }

    public static void Ensure( bool postcondition, string descriptionFormat, params object[] descriptionParameters )
    {
        if ( ! postcondition )
        {
            throw new AssertException(ExceptionDescription("Postcondition",string.Format( CultureInfo.InvariantCulture, descriptionFormat, descriptionParameters ) ) );
        }
    }


    public static void Check( bool condition, string conditionDescription )
    {
        if ( ! condition )
        {
            throw new AssertException(ExceptionDescription("Condition",conditionDescription));
        }
    }

    public static void Check( bool condition, string descriptionFormat, params object[] descriptionParameters )
    {
        if ( ! condition )
        {
            throw new AssertException(ExceptionDescription("Condition",string.Format( CultureInfo.InvariantCulture, descriptionFormat, descriptionParameters ) ) );
        }
    }


    //
    // Private methods
    //

    private static string ExceptionDescription(string assertionType, string description)
    {
        return string.Format(CultureInfo.InvariantCulture, "{0} failed. The expectation was '{1}', but this is false.", assertionType, description);
    }

}

As important as checking the assertions at runtime is to allow a client of our class to read the preconditions and postconditions without inspecting the implementation of our methods. The simplest solution is to copy the assertions into then method comments:

public class PrinterDescription
{
    private XmlDocument printerXml;

    /// <summary>
    /// Load the descripton from a xml file
    /// Assertion.RequireIsNotNull(printerDescriptionPath, "printerDescriptionPath");
    /// Assertion.Require(File.Exists(printerDescriptionPath), "File printerDescriptionPath exists");
    /// Assertion.Ensure(IsLoaded, "IsLoaded");
    /// </summary>
    public void Load(string printerDescriptionPath)
    {
        Assertion.RequireIsNotNull(printerDescriptionPath, 
            "printerDescriptionPath");
        Assertion.Require(File.Exists(printerDescriptionPath), 
            "File printerDescriptionPath exists");

        printerXml = new XmlDocument();
        printerXml.Load(printerDescriptionPath);

        Assertion.Ensure(IsLoaded, "IsLoaded");
    }

    /// <summary>
    /// Is the description loaded?
    /// </summary>
    public bool IsLoaded
    {
        get
        {
            return printerXml != null;
        }
    }

    /// <summary>
    /// Name of the printer
    /// Assertion.Require(IsLoaded, "IsLoaded");
    /// </summary>
    public string Name
    {
        get
        {
            Assertion.Require(IsLoaded, "IsLoaded");
            XPathNavigator nameNode = printerXml.CreateNavigator().SelectSingleNode("//PrinterName");
            return nameNode.Value;
        }
    }

}