Sunday, December 29, 2013

Unit testing c# code which starts new threads via tasks

Unit testing multithreaded code seems sometimes impossible. We expect Unit Tests to run fast and to deliver predictable results. Tests which are executing code that spawns new threads are very often slow and fragile. These tests are not Unit Tests. Unit Tests are tests running our code in isolation from other parts of the system. Another part of the system are the classes providing access to the operating system multithreading capabilities. We use these classes to start new threads.
To test the functional part of our code with unit tests we have to exclude the concurrency and synchronization aspects. These aspects are system-wide concerns and have to be tested later separately with integration tests. Integration tests can for example put the system under load to uncover synchronisation problems between multiple threads.
But first we have to assure that the functional part of our code works as expected. We have to equip our code to be able to isolate the concurrency and synchronization aspects. Then we can write Unit Tests which are executing our functionality on the same thread as the unit test.
So for Unit Tests in conjunction with multithreaded code we can summarize:
  • Multithreaded code has to be equipped for isolating concurrency and synchronization aspects from functional aspects. 
  • Unit Tests can only test the functional aspect of multithreaded code. They can not test concurrency and synchronization aspects of the code, because these are concerns outreaching a single unit.
The jMock Team has described an brilliant approach for Java in the 'jMock Cookbook: Test Multithreaded Code'. This concept can also be applied to C# code using Tasks of the .NET Framework Class Libarary. The default TaskScheduler can be replaced with our own implementation called DeterministicTaskScheduler to test our code synchronously in a deterministic way. Unit tests are then able to run the tasks on their own thread:

[Test]
public void ShouldUpdateTheStatusWhenDoAsync()
{
    // Arrange 
    DeterministicTaskScheduler taskScheduler = new DeterministicTaskScheduler();
    SystemUnderTest sut = new SystemUnderTest(taskScheduler);
    Assert.AreEqual("Nothing done", sut.Status);
    
    // Act
    sut.DoAsync(); // starts a new task and returns immediately
    Assert.AreEqual("Starting task", sut.Status);
    // Now execute the new task
    taskScheduler.RunTasksUntilIdle();

    // Assert
    Assert.AreEqual("Task done", sut.Status);
}

Our code which starts a new task has to provide dependency injection capability for the TaskScheduler. We start a new task in DoAsync() via Task.Factory.StartNew():
public class SystemUnderTest
{
 private string status;
 private TaskScheduler taskScheduler;

 public SystemUnderTest(TaskScheduler taskScheduler)
 {
  status = "Nothing done";
  this.taskScheduler = taskScheduler;
 }

 public string Status {get { return status;}}

 public void DoAsync()
 {
  status = "Starting task";
  Task.Factory.StartNew(
      DoWork1, 
      new CancellationToken(), 
      TaskCreationOptions.None, 
      taskScheduler);
 }

 private void DoWork1()
 {
  // Do some work here
  status = "Task done";
 }
}

The DeterministicTaskScheduler to be used in unit tests:
    /// 
    /// TaskScheduker for executing tasks on the same thread that calls RunTasksUntilIdle() or RunPendingTasks() 
    /// 
    public class DeterministicTaskScheduler : TaskScheduler
    {
        private List<Task> scheduledTasks = new List<Task>(); 

        #region TaskScheduler methods

        protected override void QueueTask(Task task)
        {
            scheduledTasks.Add(task);
        }

        protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
        { 
            scheduledTasks.Add(task);
            return false;
        }

        protected override IEnumerable<Task> GetScheduledTasks()
        {
            return scheduledTasks;
        }

        public override int MaximumConcurrencyLevel { get { return 1; } }

        #endregion

        /// 
        /// Executes the scheduled Tasks synchronously on the current thread.
        /// If those tasks schedule new tasks they will also be executed
        /// until no pending tasks are left.
        /// 
        public void RunTasksUntilIdle()
        {
            while (scheduledTasks.Any())
            {
                this.RunPendingTasks();
            }
        }

        /// 
        /// Executes the scheduled Tasks synchronously on the current thread.
        /// If those tasks schedule new tasks they will only be executed 
        /// with the next call to RunTasksUntilIdle() or RunPendingTasks(). 
        /// 
        public void RunPendingTasks()
        {
            foreach (var task in scheduledTasks.ToArray())
            {
                this.TryExecuteTask(task);
                scheduledTasks.Remove(task);
            }
        }
    }

In the client production code we can inject a real TaskScheduler from the current context to our class:
SystemUnderTest sut = new SystemUnderTest(
                      TaskScheduler.FromCurrentSynchronizationContext());
sut.DoAsync();

See also my MSDN Magazine article for general remarks about unit testing multithreaded code.

Sunday, September 15, 2013

Example for a responsive console application with async/await vs threads

There are mainly two reasons for using asynchronous programming:
  1. Utilize your CPU cores by parallelizing CPU intensive work.
  2. Keep your application responsive while long running activities are in progress. These activities may be CPU intensive work or are waiting for a response from the web, lan or database (IO bound work)   
Responsiveness is not only required for applications with an user interface but also for background services and applications. Background services should answer additional request even when they are currently busy with something else. On a user interface a user wants to at least be able to cancel a long running activity.

The following simple example demonstrate asynchronous programming with .NET 4.5 and async/await with an console application. The two long running activities (DoAction1() and DoAction2()) with CPU bound work are executed in parallel. The user can exit the application even before the computation and printing of the result has finished:
class Program
{
 static void Main()
 {
     var program = new Program();

     program.PrintResultAsync();

     Console.WriteLine("Press <return> to exit ...");
     Console.ReadLine();
 }

 public int DoAction1()
 {
     int result = 0;
     for (int i = 0; i < 5; i++)
     {
       Console.WriteLine("Action1 "+ result);
       Thread.Sleep(250);
       result++;
     }
     return result;
 }

 public int DoAction2()
 {
     int result = 0;
     for (int i = 0; i < 5; i++)
     {
       Console.WriteLine("Action2 " + result);
       Thread.Sleep(250);
       result--;
     }
     return result;
 }


 public async void PrintResultAsync()
 {
     Tuple<int, int> r = await ComputeResultAsync();
     Console.WriteLine("Result " + r.Item1 + " + " + r.Item2 + " = " + (r.Item1 + r.Item2));
 }

 private async Task<Tuple<int, int>> ComputeResultAsync()
 {
     var t1 = Task<int>.Factory.StartNew(this.DoAction1);
     var t2 = Task<int>.Factory.StartNew(this.DoAction2);
     return new Tuple<int, int>(await t1, await t2);
 }
}

The application has the output on the console below:
We can compare the previous async/await implementation with an equal implementation using plain threads. The thread based implementation is much longer and requires more overhead like additional member variables:
class Program
{
 static void Main()
 {
     var program = new Program();

     program.PrintResultWithThreads();

     Console.WriteLine("Press <return> to exit ...");
     Console.ReadLine();
 }

 public int DoAction1()
 {
     int result = 0;
     for (int i = 0; i < 5; i++)
     {
       Console.WriteLine("Action1 "+ result);
       Thread.Sleep(250);
       result++;
     }
     return result;
 }

 public int DoAction2()
 {
     int result = 0;
     for (int i = 0; i < 5; i++)
     {
       Console.WriteLine("Action2 " + result);
       Thread.Sleep(250);
       result--;
     }
     return result;
 }

 private int r1;
 private int r2;
 private ManualResetEvent e1 = new ManualResetEvent(false);
 private ManualResetEvent e2 = new ManualResetEvent(false);
 private WaitHandle[] manualEvents;

 public void PrintResultWithThreads()
 {
     var printThread = new Thread(
       () =>
       {
         e1 = new ManualResetEvent(false);
         e2 = new ManualResetEvent(false);
         manualEvents = new WaitHandle[] { e1, e2 };
         ComputeResultsWithThreads();
         WaitHandle.WaitAll(manualEvents);
         Console.WriteLine("Result: " + r1 + " + " + r2 + " = " + (r1 + r2));
       });
     printThread.IsBackground = true;
     printThread.Start();
 }

 private void ComputeResultsWithThreads()
 {
     var t1 = new Thread(
       () =>
       {
         this.r1 = this.DoAction1();
         this.e1.Set();
       });
     t1.IsBackground = true;
     t1.Start();
     var t2 = new Thread(
       () =>
       {
         this.r2 = this.DoAction2();
         this.e2.Set();
       });
     t2.IsBackground = true;
     t2.Start();
 }
}

Sunday, January 13, 2013

Introducing Continuous Integration: A Success Story

In his book “The Clean Coder” Robert C. Martin tells several anecdotes from his professional live as a software developer to explain the attitude of a professional developer. That reminds me of a story when I started my current job.
It was around 2009 when I started to work in an organization with seven software development teams each with the size of approximately ten people. The teams delivered every two month new software versions which are then manually integrated to a running system on a medical device hardware. The integration took around additional four to six weeks. I couldn't believe that software was created with this “stone age” process. In the worst case the developers got feedback for their features from the fully integrated system only after three month. Oh my good, why do I started to work there? I talked to my boss about my concerns and he admitted that we have to change something. So I was appointed as a project manager. The core project team had two members: Matt, a very experienced software consultant and me.
We called the project “DailyBuild”. So our goal was to speed up the integration cycle time from ninety days to one. Yes, there where a lot of obstacles on our way. For example the teams used different source code control systems, we perceived the number of used build scripting tools as nearly infinite and the total software build time was about eight hours. Many problems we had to solve were related to the organizational structure of the development department. And of course if you want to change the way of working of people, a lot of them say: “Why should we change, we have always worked like this”. The only good thing was that I knew all the time that the project could not fail. I knew that the benefits would be overwhelming. The agile movement was already around several years and thousands of software teams had already adopted Continuous Integration.

After 4 month we had the first successful automated build including a simple smoke test of the integrated system on a virtual machine. It took some more weeks to finish things. Today, no one in the development department can imagine to work any more without the “DailyBuild” process in place. Yes we made it!