Unit testing code with timers

Attempts to unit test code which uses a timer most often lead to tests like:

[Test]
public void SlowAndFragileTest()
{
    // Setup
    dataProviderMock = MockRepository.
                             GenerateMock<IDataProvider>();
    systemUnderTest = new PollCommand(dataProviderMock);

    // Act
    systemUnderTest.StartPoll();

    // Make sure the timer has been executed at least
    // once. The action is started after 10 seconds.
    Thread.Sleep(11000);

    // Assert
    dataProviderMock .AssertWasCalled(
                         x=>x.Retrieve());
}

This testing approach is problematic. The test always takes 11 seconds, which is far to slow for a unit test. Depending on the timer used it is not guaranteed at what exact point in time the action is triggered. The action itself also takes time, so we can't be 100% sure that the action is finished when we test reaches the assert statement.

Solution - Test code with timers in one thread

We can separate the timer mechanics from our code and use a special timer which executes the action on the current thread. We can also tell this timer how much time has passed by:

[Test]
public void ShouldRetrieveDataEvery10Seconds()
{
    // Arrange
    timerStub = new DeterministicTimer();
    dataProviderMock = MockRepository.
                             GenerateMock<IDataProvider>();
    systemUnderTest = new PollCommand(
                             timerStub, 
                             dataProviderMock);

    // Act
    systemUnderTest.StartPoll();
    // Now execute the timer actions on
    // the current thread.
    timerStub.TickSeconds(30);

    // Assert
    dataProviderMock.AssertWasCalled(
                        x=>x.Retrieve(),
                        y=>y.Repeat.Times(3));
}

This test is very fast and takes fractions of a second. This test is also robust because every code is executed on the current thread.
Our PollComand class we are testing now looks very simple:

private readonly ITimer timer;

public void StartPoll()
{
    timer.StartTimer(
             RetrieveData, 
             new TimeSpan(0, 0, 0, 10));
}
private void RetrieveData()
{
    retrievedDate = dataProvider.Retrieve();
}

The DeterministicTimer is straight forward:

public class DeterministicTimer : ITimer
{
 private Action action;
 private TimeSpan intervall;

 private bool isStarted;
 private TimeSpan elapsedTime;

 #region ITimer methods
 public void StartTimer(Action action, TimeSpan intervall)
 {
     isStarted = true;
     this.action = action;
     this.intervall = intervall;
     elapsedTime = new TimeSpan();
 }

 public bool IsStarted()
 {
     return isStarted;
 }

 public void StopTimer()
 {
     isStarted = false;
 }
 #endregion

 public void TickSeconds(int seconds)
 {
     TimeSpan newElapsedTime = elapsedTime + new TimeSpan(0, 0, 0, seconds);
     if (isStarted)
     {
       long executionCountBefore = elapsedTime.Ticks / intervall.Ticks;
       long executionCountAfter = newElapsedTime.Ticks / intervall.Ticks;

       for (int i = 0; i < executionCountAfter - executionCountBefore; i++)
       {
         action();
       }
     }
     elapsedTime = newElapsedTime;
 }
}

See also my MSDN Magazine article for general remarks about unit testing multithreaded code.

Unit testing multi-threaded code

Attempts to unit test multi-threaded code most often lead to tests like:

[Test]
public void FragileAndSlowTest()
{
    // Setup
    SystemUnderTest sut = new SystemUnderTest();
     
    // Start new thread
    sut.DoAsync(); 
 
     // Make sure the thread has finished execution
    Thread.Sleep(2000);
 
    // Assert
    ...
}

This testing approach has two disadvantages:

1. From time to time this test fails when the thread is not started and executed within two seconds. This can happen for example when the PC is busy with other activities. We can call these kinds of tests fragile. 
2. This test takes alway at least two seconds to run. This is far to slow for a unit test.

Solution 1 - Call functionality without multi-threading code 

We can execute the method which is executed in the new thread directly excluding the concurrency aspect. Either we make the method 'public' or we extract the functionality of the method into a separate class and then call that method directly from the test:

[Test]
public void TestWithoutTouchingThreadingCode()
{
    // Setup
    SystemUnderTest sut = new SystemUnderTest();
     
    // Directly call synchronous method 
    // which is internally called by DoAsync()
    sut.Do(); 
 
    // Assert
    ...
}

Advantages:

• We are forced to separate concerns in the code (functionality and concurrency) leading to better maintainability. 

Disadvantages:

• We have to make the private method Do() public for the purpose of testing. In production we would never call the method from outside so we are violating the information hiding principle. 
• The DoAsync() method is not executed in any test. If we have a more complex concurrency behaviour than in the example like starting an additional thread when the first thread is finished, we will also not test this behaviour. 

Solution 2 - Test multi-threading code in one thread

We can replace the thread starting object with a test stub, which executes the Do() method in the same thread as the unit test. The example shows C# code using tasks from the .NET Framework Class Library. The default TaskScheduler can be replaced with our own implementation called DeterministicTaskScheduler. The test executes all code in the current thread:

[Test]
public void TestMultiThreadingCodeSynchronously()
{
    // Arrange
    DeterministicTaskScheduler taskScheduler =
        new DeterministicTaskScheduler();
    SystemUnderTest sut = 
        new SystemUnderTest(taskScheduler);
     
    // Execute multi-threaded code and 
    // return immediately.
    sut.DoAsync();

    // Now execute the new task on
    // the current thread.
    taskScheduler.RunTasksUntilIdle();
 
    // Assert
    ...
}

See this blog post for a detailed description of the DoAsync() implementation and the DeterministicTaskScheduler.
The concept also works very well for code using timers:

[Test]
public void ShouldRetrieveDataEverySecond()
{
    // Arrange
    timerStub = new DeterministicTimer();
    dataProviderStub = MockRepository.GenerateStub<IDataProvider>();
    systemUnderTest = new PollCommand(timerStub, dataProviderStub);

    // Act
    systemUnderTest.StartPoll();
    // Now execute the timer actions on
    // the current thread.
    timerStub.TickSeconds(3);

    // Assert
    dataProviderStub.AssertWasCalled(x=>x.Retrieve(),y=>y.Repeat.Times(3));
}

Advantages:

• We are executing the DoAsync() method in our test.
• Our multi-threading code can still use directly the concurrency methods of the .NET Framework Class Library like Task.Factory.StartNew()  

Disadvantages:

• We need to prepare our code to inject the thread starting object. 

See also my MSDN Magazine article for general remarks about unit testing multithreaded code.