Unit testing multi-threaded code

Attempts to unit test multi-threaded code most often lead to tests like:

[Test]
public void FragileAndSlowTest()
{
    // Setup
    SystemUnderTest sut = new SystemUnderTest();
     
    // Start new thread
    sut.DoAsync(); 
 
     // Make sure the thread has finished execution
    Thread.Sleep(2000);
 
    // Assert
    ...
}

This testing approach has two disadvantages:

1. From time to time this test fails when the thread is not started and executed within two seconds. This can happen for example when the PC is busy with other activities. We can call these kinds of tests fragile. 
2. This test takes alway at least two seconds to run. This is far to slow for a unit test.

Solution 1 - Call functionality without multi-threading code 

We can execute the method which is executed in the new thread directly excluding the concurrency aspect. Either we make the method 'public' or we extract the functionality of the method into a separate class and then call that method directly from the test:

[Test]
public void TestWithoutTouchingThreadingCode()
{
    // Setup
    SystemUnderTest sut = new SystemUnderTest();
     
    // Directly call synchronous method 
    // which is internally called by DoAsync()
    sut.Do(); 
 
    // Assert
    ...
}

Advantages:

• We are forced to separate concerns in the code (functionality and concurrency) leading to better maintainability. 

Disadvantages:

• We have to make the private method Do() public for the purpose of testing. In production we would never call the method from outside so we are violating the information hiding principle. 
• The DoAsync() method is not executed in any test. If we have a more complex concurrency behaviour than in the example like starting an additional thread when the first thread is finished, we will also not test this behaviour. 

Solution 2 - Test multi-threading code in one thread

We can replace the thread starting object with a test stub, which executes the Do() method in the same thread as the unit test. The example shows C# code using tasks from the .NET Framework Class Library. The default TaskScheduler can be replaced with our own implementation called DeterministicTaskScheduler. The test executes all code in the current thread:

[Test]
public void TestMultiThreadingCodeSynchronously()
{
    // Arrange
    DeterministicTaskScheduler taskScheduler =
        new DeterministicTaskScheduler();
    SystemUnderTest sut = 
        new SystemUnderTest(taskScheduler);
     
    // Execute multi-threaded code and 
    // return immediately.
    sut.DoAsync();

    // Now execute the new task on
    // the current thread.
    taskScheduler.RunTasksUntilIdle();
 
    // Assert
    ...
}

See this blog post for a detailed description of the DoAsync() implementation and the DeterministicTaskScheduler.
The concept also works very well for code using timers:

[Test]
public void ShouldRetrieveDataEverySecond()
{
    // Arrange
    timerStub = new DeterministicTimer();
    dataProviderStub = MockRepository.GenerateStub<IDataProvider>();
    systemUnderTest = new PollCommand(timerStub, dataProviderStub);

    // Act
    systemUnderTest.StartPoll();
    // Now execute the timer actions on
    // the current thread.
    timerStub.TickSeconds(3);

    // Assert
    dataProviderStub.AssertWasCalled(x=>x.Retrieve(),y=>y.Repeat.Times(3));
}

Advantages:

• We are executing the DoAsync() method in our test.
• Our multi-threading code can still use directly the concurrency methods of the .NET Framework Class Library like Task.Factory.StartNew()  

Disadvantages:

• We need to prepare our code to inject the thread starting object. 

See also my MSDN Magazine article for general remarks about unit testing multithreaded code.