'Design By Contract' Metrics

We are currently introducing 'Design By Contract' to a software development group of about 60 developers, which are developing different components. We started by defining 'Design By Contract' policies for C# and Java. It is quite challenging to manage this change effort.

One piece of the change strategy is to measure the progress. We are counting the number of classes and the number of contract assertions (Preconditions, post conditions and invariants). So we have two statistics:

1. Absolut number of contract assertions per component
2. Average number of contract assertions per class per component

The metrics tell us whether contracts are "at all" be used. We want to increase the code quality with contracts. If we see a team not implementing any contracts or only very few, we can support the team with training and consulting.

The metrics are published on a regular basis and serve as a means for motivation.

The limitation of thise metrics is, that they do not tell whether a component has enough contracts, so its understandability, maintainability and so on is best supported with 'Design By Contract'. Quality of contracts is not covered by the metrics.

Example for parsing C# code with the NRefactory library

NRefactory is part of the open source IDE SharpDevelop for the .NET platform. NRefactory is a parser library for C# and VB. It can create an Abstract Syntax Tree (AST) that represents all constructs that are available in C# or VB. This AST can be used to analyze source code or to modify and generate code again.

You can download the SharpDevelop IDE, install it and then find the ICSharpCode.NRefactory.dll in the bin folder of the installation. Or you download the SharpDevelop source code and compile the dll yourself.

The example below shows how to parse C# source code files, generate an AST and then using the AST to create metrics about the number of classes and the number of Code Contracts.

using System;
using System.IO;
using System.Diagnostics.Contracts;
using ICSharpCode.NRefactory;
namespace ContractCounter
{
  class Program
  {
    public static void Main(string[] args)
    {
      TextReader reader = File.OpenText("Program.cs");
      using (IParser parser = ParserFactory.CreateParser(SupportedLanguage.CSharp, reader))
      {
        parser.Parse();
        if (parser.Errors.Count <= 0)
        {
          // Here we will use the parser.CompilationUnit(AST)
          ...
        }
        else
        {
          Console.WriteLine("Parse error: " + parser.Errors.ErrorOutput);
        }
      }
      Console.Write("Press any key to continue . . . ");
      Console.ReadKey(true);
    }
  }
}

To traverse the AST we can use the Visitor pattern (see [Gamma et.al:Design Patterns]. We implement a new visitor 'CounterVisitor' for our purposes. We can inherit from the predefined 'AbstractAstVisitor'. In NRefactory visitors are responsible for traversing the AST by themselfs so we have to call the children when we are visiting certain node types:

using System.Diagnostics.Contracts;
using ICSharpCode.NRefactory.Ast;
using ICSharpCode.NRefactory.Visitors;
namespace ContractCounter
{
  public class CounterVisitor : AbstractAstVisitor
  {
    public override object VisitCompilationUnit(CompilationUnit compilationUnit, object data)
    {
      Contract.Requires(compilationUnit != null);
      
      // Visit children (E.g. TypeDcelarion objects)
      compilationUnit.AcceptChildren(this, data);
      
      return null;
    }

    public override object VisitTypeDeclaration(TypeDeclaration typeDeclaration, object data)
    {
      Contract.Requires(typeDeclaration != null);
      
      // Is this a class but not a test fixture?
      if (IsClass(typeDeclaration) && !HasTestFixtureAttribute(typeDeclaration))
      {
        classCount++;
      }

      // Visit children (E.g. MethodDeclarion objects)
      typeDeclaration.AcceptChildren(this, data);
      
      return null;
    }

    public override object VisitMethodDeclaration(MethodDeclaration methodDeclaration, object data)
    {
      Contract.Requires(methodDeclaration != null);

      // Visit the body block statement of method declaration
      methodDeclaration.Body.AcceptVisitor(this, null);

      return null;
    }

    public override object VisitBlockStatement(BlockStatement blockStatement, object data)
    {
      Contract.Requires(blockStatement != null);

      // Visit children of block statement (E.g. several ExpressionStatement objects)
      blockStatement.AcceptChildren(this, data);

      return null;
    }

    public override object VisitExpressionStatement(ExpressionStatement expressionStatement, object data)
    {
      Contract.Requires(expressionStatement != null);

      // Visit the expression of the expression statement (E.g InnvocationExpression)
      expressionStatement.Expression.AcceptVisitor(this, null);

      return null;
    }

    public override object VisitInvocationExpression(InvocationExpression invocationExpression, object data)
    {
      Contract.Requires(invocationExpression != null);

      // Visit the target object of the invocation expression (E.g MemberReferenceExpression)
      invocationExpression.TargetObject.AcceptVisitor(this, null);
      return null;
    }

    public override object VisitMemberReferenceExpression(MemberReferenceExpression memberReferenceExpression, object data)
    {
      Contract.Requires(memberReferenceExpression != null);

      IdentifierExpression identifierExpression = memberReferenceExpression.TargetObject as IdentifierExpression;

      // Is this a call to Contract.Requires(), Contract.Ensures() or Contract.Invariant()?
      if ( identifierExpression != null &&
          identifierExpression.Identifier == "Contract" &&
          (memberReferenceExpression.MemberName == "Requires" ||
           memberReferenceExpression.MemberName == "Ensures" ||
           memberReferenceExpression.MemberName == "Invariant") )
      {
        assertionCount++;
      }

      return null;
    }

    public int ClassCount {
      get { return classCount; }
    }

    public int AssertionCount
    {
      get { return assertionCount; }
    }

    #region private members
    private int classCount;
    private int assertionCount;

    static private bool IsClass(TypeDeclaration typeDeclaration)
    {
      return typeDeclaration.Type == ClassType.Class;
    }

    static private bool HasTestFixtureAttribute(TypeDeclaration typeDeclaration)
    {
      bool hasTestFixtureAttribute = false;
      foreach (AttributeSection section in typeDeclaration.Attributes) {
        foreach (Attribute attribute in section.Attributes) {
          if (attribute.Name == "TestFixture") {
            hasTestFixtureAttribute = true;
            break;
          }
        }
      }
      return hasTestFixtureAttribute;
    }
    #endregion
  }
}

The actual counting takes place in the VisitTypeDeclaration() and the VisitMemberReferenceExpression() methods. All other methods are just neccesary for traversing the tree. We now have to start the vistor to traverse the AST in the Main() method:

...
// Here we will use the parser.CompilationUnit(AST)
CounterVisitor visitor = new CounterVisitor();
parser.CompilationUnit.AcceptVisitor(visitor, null);
Console.WriteLine("The file contains " + visitor.ClassCount + " class(es)");
Console.WriteLine("The file contains " + visitor.AssertionCount + " contract(s)");
...

For exploring the structure of the NRefactory AST you can use the NRefactoryDemo application, which is part of the SharpDevelop source code. You can enter source code and let the application create the according AST: