Preventing Unexpected Message Pumping in Third Party Libraries - An example using DirectShow

• Download source code - 102 KB

Introduction

This project demonstrates a problem inherent in the design of windowing applications and a simple solution to the problem. The problem is that when you call a third party library, there is no guarantee that the library will not call Application.DoEvents() or in some other way cause pumping of your application's message queue. This is normally not a problem, but it can set up a race condition in your code.

For example, assume the user has rapidly clicked a button several times in succession. Your code is busy processing the first button click and your application is in an inconsistent state. You probably assumed (justifiably) that throughout your button event handler, you were quite safe from being preempted by another button click. You are after all using a single threaded model with an event queue that serialises user input. In the button click event handler, you then make a call out to some third party code. That third party code, for whatever reason, calls Application.DoEvents() or pumps messages in native code. Suddenly, the next button click event is fired, and very quickly throws an exception due to the inconsistent state left by the partly run event handler of the first click.

Background

I ran into this problem while working on a C# WinForms application that uses DirectShow to do video recording. The problem was found by a tester who rapidly clicked on various control buttons and caused a null ref exception. The diagnosis of the problem is recorded in a UseNet thread here.

I considered other solutions to the problem such as dropping all user input and timer messages during the processing of a user input or timer message. This solution would prevent the reentrancy, but dropping messages could cause other problems.

Solution

To prevent the nesting of event handlers, we need to prevent the message loop in the third party code - the DirectShow code in this case - from dequeing and processing messages destined for windows in your application. The solution I used to do this was to start a second thread with its own message loop. This code fragment is from the Main() method of the application:

SynchronizationContext context = null;
bool started = false;

ThreadStart startRoutine =
    delegate
    {
        // Set the sync context on the thread
        context = new WindowsFormsSynchronizationContext(); 
        SynchronizationContext.SetSynchronizationContext(context);

        // Run a dedicated message loop with no forms
        Application.Run();
    };

// Start up the DS runner
Thread runner = new Thread(startRoutine);
runner.Start();

Note that a synchronization context is created and set on the new thread. This is to allow easy inter-thread messaging using SynchronizationContext.Send(). Simply place the calls to the third party library (the one that pumps messages) in a SendOrPostCallback delegate and Send() it to the other thread.

SendOrPostCallback a =
    delegate
    {
        //Call code that pumps messages here
        ......
    };

context.Send(a, null);

One detail is that a WindowsFormsSynchronizationContext is used, not a plain SynchronizationContext. The problem is that SynchronizationContext.Send() actually executes the SendOrPostCallback on the calling thread! The WindowsFormsSynchronizationContext.Send() executes it correctly on the target thread.

Running the Demo

The demo application has two buttons. The first button runs a DirectShow call on the main thread after first queuing a message on the main thread's message queue. When the DirectShow call is made, the queued message is processed, nested inside the button click.

The second button runs the DirectShow call synchronously on a dedicated thread. This means that the queued message is not processed until after the button click handler returns.

Points of Interest

It should be noted that the solution to the problem demonstrated here is not production ready. If you use it, you will want to package it up in a helper class somehow. You'll also need to consider what happens to any exceptions thrown in the second thread (hint: if you use Send(), they will come back to the main thread; if you use Post(), they'll need to be handled on the second thread).

Disclaimer

I do not consider myself an expert in either DirectShow or in the details of Windows programming, be it Win32 or .NET. This article is submitted in the hope that it will help other people avoid the same situation, but also in the hope that more expert programmers will suggest other solutions.