The Practical Guide to Multithreading - Part 1

Ajay Vijayvargiya

4.89/5 (121 votes)

Nov 18, 2009

CPOL

17 min read

314924

10434

Learn from this guide how and when - as well as when not - to use multithreading.

Introduction

This article does not give an introduction to multithreading, processes, processors, etc. Nor does it gives the syntax details of threading functions. It only presents the practical approach to multi-threaded programming. If you do not know about multithreading, then read some articles on the web that show the syntax, how you write console applications, and how to use two threads simultaneously writing on the console. Learn about how one finishes before the other and things like that. Here, I am only concerned about the practical approach. Like, "How can you cancel or pause the processing?" (one of the simplest!).

Expected Audience

Knows the basics of multithreading
Knows GUI programming in either MFC or .NET Framework

The example code and images in this article are presented in both C++/C# and MFC/.NET GUIs. The project of your choice is available to download.

The Practical Example 1

You are doing some lengthy processing, and displaying a progress-bar along with some text that reflects the processing status.

And here is the code to do the processing:

private void btnProcess_Click(object sender, EventArgs e)
{
    int nStart = Convert.ToInt32(txtStart.Text);
    int nEnd = Convert.ToInt32(txtEnd.Text);

    prgProcess.Minimum=nStart;
    prgProcess.Maximum=nEnd;
    for (int nValue = nStart; nValue <= nEnd; nValue++)
    {
        lblStatus.Text = "Processing item: "+ Convert.ToString(nValue);
        prgProcess.Value = nValue;
    }
}

As you can see, it seems to perform well, but not as expected. The status text isn't being updated (download and run the code). Furthermore, if you enter a high value like in the following image and then try to fiddle with the window, the window does not respond! You may see something like what is shown below (depending on your OS):

So, how do you solve this problem? The simplest and one of the worst solutions is to call Application.DoEvents (in C#). MFC/Windows API programmers can call the PeekMessage and DispatchMessage sequence for the same, but it is not shown here for simplicity:

for (int nValue = nStart; nValue <= nEnd; nValue++)
{
    lblStatus.Text = "Processing item: "+ Convert.ToString(nValue);
    prgProcess.Value = nValue;

    // Let Windows (application) process user actions (and other messages)
    Application.DoEvents();

}

In this code, you can see it works flawlessly!

So then, why do you need multithreading if this code solves the problem? Well, there are a lot of issues!

Let's start with a simple one. You can re-click the button to start processing (and even change the start/end values). This will move the progress bar backwards or forwards depending on the values chosen. It would also be random!

So what? Just disable all of the controls until the processing is done, and then re-enable controls!

In the initial stages of this kind of programming, it would work most of the time, thus another thread would not be needed. But, when the routine (function) is performing some advanced tasks—like reading from file, encrypting/decrypting contents of a file, validating the words in a file, sending them over the Internet, or just copying to another file—this would not work. You may need to call DoEvents (PeekMessage/DispatchMessage) multiple times:

ReadFile();
DoEvents();
VerifyContents();
DoEvents();
DecryptFile();
DoEvents();
bool bDuplicate = AreWordsDuplicate();
DoEvents();

if(bDuplicate)
{
    ProcessDuplicate();
    DoEvents();
}
else
{
    ...
}
...

As you can see from the above code, calling DoEvents at multiple phases of a routine is overkill. It will definitely slow down the processing. Also, you need to keep track of putting DoEvents calls properly. What if the user has clicked 'Cancel' (assume you left it enabled)? You would then need to have some global, class-level variable to check if the user has canceled, and this must be done after each call to DoEvents.

At times, you also might not be able to disable controls, and a set of controls might trigger another event. For example, if you have 'CopyAllFiles', which is a long routine, and you put a set of properly balanced DoEvents and 'HasUserPressedCancel' calls. But, what if the user presses 'DeleteFile', 'RenameFile', or something like that? You cannot, absolutely cannot, control user actions. Furthermore, you are restricting users to use your application efficiently. Just because you want to avoid multithreading?

Another problem! (Yes, I know there are readers who are justified with MT, but I must put it for completeness!)

Calling DoEvents can cause a stack overflow exception to occur! This would crash the application. You might wonder why that would happen. Well, DoEvents searches for the events for the window (events can be initiated by the user or by the system). If it finds an event (in Windows terms, it is the Windows-message), it calls the respective event/message-handler (function), which might also be calling DoEvents, which might also call something else. This process might go on, and on, and... duh! Exception! Your application is down!

Period. End of story. Do not use DoEvents, or an equivalent variant in other language/framework.

Let's get our hands dirty with multithreading!

You might be extremely willing to see how threads are created, managed. The actual code? Hold on your breath for a while. The following paragraph is very important! Read.

Windows and Threads are different concepts, but for now, assume they are the same. A Windows window may process the message synchronously or asynchronously.

Synchronously: When you notify a window to process the message/event, and do not return back until the message is processed. In Windows API/MFC, you notify the window (receiver of message) by using the SendMessage API. SendMessage will not return until the message is processed by the window. In .NET, you achieve the same by using the Control.Invoke method.
Asynchronously: You notify the window, but do not wait to see if the message is processed by the window. You achieve the same by using the PostMessage API. The PostMessage API will put the specified message into the target window's message queue (receiver, who is going to handle the message). You do not manage the message-queue, it is managed by Windows OS. In .NET, you use the Control.BeginInvoke method for the same.

Managing Threads

The following table lists out the common operations with threads and how you achieve them using different programming elements. This list does not include communicating with a thread while it is running, nor does it list thread-security, thread-priority etc. POSIX threads are not used in this article, but are listed here for completeness.

* The C# compiler would warn about these methods that we should use other techniques. And we would, as I would describe later!
** These are non-portable APIs, not officially supported by POSIX.
# Forceful thread termination is not recommended. Rarely, we may need to do that. We'll discuss it later.
^ sleep, or usleep functions suspends the entire process!
^^ pthread_kill with '0' signal is not appropriate. Another thread with the same ID might have been created. We need to use the pthread_key_* set of functions. Describing them is absolutely out of the scope of this article.

Adding in favor of gracefully ending threads: it is always recommended to let the thread function (the root function of the thread, like the 'main' function for the process) return and end the thread properly. Calling the respective functions (as listed) for End-Thread-Gracefully has some issues. A few of them could be: open file may not be closed, C++ class-objects would not be destroyed via destructor, thread-synchronization object would be inconsistent (later on this), and other issues.

I assume a basic understanding of threads from the reader, but I will still mention a few points:

A thread runs in the process address space. The process has at least one thread. The first thread created on application startup is the primary thread.
Each thread has its own (function) call stack. Even if you create multiple threads of the same name (i.e., the same function/thread-routine), they would have different call stacks.
Threads may have one of few 'states', like Running, Waiting for I/O, Blocked etc.
A thread that causes an exception, like divide by zero, or a null-pointer assignment, which is not handled by the thread, would take the entire process down!

Okay! Here is the code to create threads. The simplest approaches are chosen.

MFC

UINT Example1Thread(void*);

void CExample1_Dlg::OnBnClickedOk()
{
    AfxBeginThread(Example1Thread, this);

    // OnOK();
}

UINT Example1Thread(void *pParam)
{
    CExample1_Dlg* pThis= (CCExample1_Dlg*)pParam;

    // Use pThis
    // Perform processing

    return 0;
}

Please note the syntax of 'Example1Thread'. AfxBeginThread requires the function-pointer, which returns 'UINT' and takes 'void*'. The void* argument content is the same that we pass as the second argument to AfxBeginThread. Since we know the exact type, we further typecast it to a C++ class type. It must be noted that AfxBeginThread/CreateThread requests the Operating System to create the thread and eventually run it. The thread creation function does not wait for the thread-routine to finish (otherwise, there is no meaning for multithreading!).

C# / .NET

// using System.Threading;
private void btnStartThreaded_Click(object sender, EventArgs e)
{
    Thread MyThread = new Thread(ProcessRoutine);
    MyThread.Start();
}
void ProcessRoutine()
{
    // Do processing here.
    // We better make this method 'static'
    // But we'll take those things later.
    // Here we can use all member of class directly (since we have 'this')
}

Fine! We now put the code in 'ProcessRoutine' which was in 'btnProcess_Click' (see the first code snippet above). When we run (with F5), Visual Studio launches a new world for us! It interrupts the debugging session, and shows the Cross Thread Access exception:

If you run it without debugging (i.e., using Ctrl+F5), the program would run and would work as expected. But the program may crash down at any time. A similar thing would happen for MFC based applications. The debugger may show some assert at some remote location, showing some alien code of the MFC library. Remember that the MFC window classes are not thread safe.

Why does it happen?

You know that you cannot open the same file in write-mode simultaneously and change its contents (ignore advanced ways to do that, if you know!). Files, windows, sockets, data-buffers, screens (and other devices), database tables etc., are all objects. These objects, mostly, prevent simultaneous modifications to them. The reason is simple: Consistency! In general, you perform modifications to objects in the following pattern:

Wait for the object to become free for modification
Open object to modify
Modify the contents of the object
Close the object so that others can open it

You may list some types of objects (like database tables) that allow simultaneous modifications. Well, they internally use multithreading concepts, which you would get to know soon!

What it has to do with the current problem in hand

The windows (or say controls) are created by some thread, generally by the primary thread of the application, but that is not necessary. That thread holds the 'write' access to modify the contents. We are trying to modify the contents using another thread, which violates the rule of 'consistency' we discussed previously. We also cannot use a 4-step process (wait, open, modify, close) to amend the contents of the window (object). Closing the window object would literally mean destroying the window (control). Please note that modifying the window-object does not mean changing the text/caption only, but includes modifying any of its properties (like color, shading, font, size, orientation, visibility etc.) also.

So, what can be done?

We can send or post a message to the window! If you can recall correctly, sending the message (via SendMessage/Invoke) is a synchronous operation. It means you notify a window and wait for the target window to finish the operation. For example, you can send the WM_SETTEXT message from another window to set the text of the target window. Don't get confused over this weird name (WM_SETTEXT) or on the actual usage of SendMessage. Things will come out of the blur as we move on! Calling SendMessage this way is same as calling SetWindowText (see code).

UINT Example1Thread(void *pParam)
{
    CExample1_Dlg* pThis= (CExample1_Dlg*)pParam;

    pThis->SendMessage(WM_SETTEXT, 0,(LPARAM)L"New Caption!");

    // eq. to ::SendMessage(pThis->m_hWnd, WM_SETTEXT, 0,(LPARAM)L"New Caption!");

        // pThis->SetWindowText("New Caption!");
        // SetWindowText internally calls SendMessage,
        // which sends WM_SETTEXT message to target window.

    return 0;
}

Calling SendMessage from any thread to any window is safe! But it's not true with MFC objects, and not with .NET control classes. Be aware. They represent complex class-objects, and are not simple handles. Okay, I won't elaborate more about this here, but would continue to our multithreading session. :)

Continuing with the problem...

Now, let us update the progress bar and status-text in a properly threaded manner:

// Delegates - same as function pointers.
// First line declares type, second declares variable
delegate void DelegateType(int x);
DelegateType TheDelegate;

// We store the Start and End values in class-variable,
// so that 'thread' actually does only processing.
int StartFrom, EndTo;
private void btnStartThreaded_Click(object sender, EventArgs e)
{
    // Set the delegate.
    TheDelegate = MessageHandler;

    StartFrom = Convert.ToInt32(txtStart.Text);
    EndTo = Convert.ToInt32(txtEnd.Text);

    prgProcess.Minimum = StartFrom;
    prgProcess.Maximum = EndTo;

    // Disable button, so that user cannot start again.
    btnStartThreaded.Enabled = false;

    // Setup thread and start!
    Thread MyThread = new Thread(ProcessRoutine);
    MyThread.Start();
}

// This is delegated function, runs in the primary-thread
// (i.e. the thread that owns the Form!)
void MessageHandler(int nProgress)
{
    lblStatus.Text = "Processing item: " + Convert.ToString(nProgress);
    prgProcess.Value = nProgress;
}

void ProcessRoutine()
{
    for (int nValue = StartFrom; nValue <= EndTo; nValue++)
    {
        // Only actual delegates be called with Invoke, and not functions!
        this.Invoke(this.TheDelegate, nValue);
    }
}

I request you to study about delegates. In short, delegates are like function-pointers in C/C++. You set some delegate-variable to the appropriate type of function. Further, you call Invoke or BeginInvoke, that would eventually call the function in the context of the thread to which the control belongs. Remember all controls (forms, comboboxes, progress bars etc.) are derived from System.Control). In the sample above, I have disabled the button so that the user cannot start processing again.

The MFC approach

UINT Example1Thread(void*);

void CExample1_Dlg::OnBnClickedOk()
{
    // Get start and end values
    StartFrom = GetDlgItemInt(IDC_START);
    EndTo = GetDlgItemInt(IDC_END);

    // Set progress bar range
    ProgressBar.SetRange(StartFrom, EndTo);

    // Disable button
    GetDlgItem(IDOK)->EnableWindow(FALSE);

    // Start off the thread
    AfxBeginThread(Example1Thread, this);
}

// The thread message handler (WM_MY_THREAD_MESSAGE)
// -- #define WM_MY_THREAD_MESSAGE    WM_APP+100
// -- ON_MESSAGE(WM_MY_THREAD_MESSAGE, OnThreadMessage)
LRESULT CExample1_Dlg::OnThreadMessage(WPARAM wParam, LPARAM)
{
    int nProgress= (int)wParam;

    // update progress bar
    ProgressBar.SetPos(nProgress);

    CString strStatus;
    strStatus.Format("Processing item: %d", nProgress);

    // update status text
    StatusText.SetWindowText(strStatus);

    return 0;
}

UINT Example1Thread(void *pParam)
{
    CExample1_Dlg* pThis= (CExample1_Dlg*)pParam;

    for (int nValue = pThis->StartFrom; nValue <= pThis->EndTo;  ++nValue)
        pThis->SendMessage(WM_MY_THREAD_MESSAGE, nValue);

    return 0;
}

The thread calls SendMessage with the WM_MY_THREAD_MESSAGE message (custom message). This, in turn, calls OnThreadMessage in the primary thread's context, and thus updating the controls is thread-safe!

(For MFC/WinAPI programmers: This example may be a starting point for you to understand and use SendMessage, ON_MESSAGE, and defining/handling user-messages; and you might find it easy. I request you to read some material and try some stuff yourself!)

Here is what it looks like:

The last note before we move on to two more practical examples. The approach chosen here (in both C++ and C#) is not appropriate.

Firstly, we are accessing integer variables into another thread. We can pass these variables to a thread before the thread starts, which I will elaborate in the next part.
Secondly, we are still calling SendMessage/Invoke instead of PostMessage/BeginInvoke, which actually means the target thread must finish before the sender thread can continue further. (More on this in the next section.)
Thirdly, after the thread finishes, we are not letting the user to start processing again, since we disabled the button and did not enable it.
Fourthly, while the thread is running, the user is able to close the window, and that may raise an exception and crash the process!

Now, move on to where we would handle these four issues, along with allowing the user to Cancel (practical example 2), and allowing the user to Pause/Resume too (example 3)!

Practical Example 2

Now, you need something like this (after processing has been started):

For this, you need to notify the worker thread (the thread having a loop) using one of the synchronization mechanisms. There are a set of synchronization primitives, and a set of APIs and classes to use the respective primitives. In this article, I will only discuss one primitive: Event. There are two types of events: Manual Reset and Auto Reset, and (in this article) I will only cover Manual Reset events.

Please don't get confused between the Windows event mechanism and the thread-synchronization event primitive. Both are different entities!

Thread Synchronization

Events and other thread-sync primitives can have one of two states: Signaled or Non-signaled. Consider them as traffic signals, in the following manner:

When a sync-object is in signaled state, you can use the resource. Like when a traffic-signal is Green, you can use the road (resource) to travel.
When a sync-object is in non-signaled state, you cannot use the resource, you (the thread) have to wait for the signal to become free (signaled) before you can use it. Obviously, you got it right - when the traffic-signal is Red, you have to wait for the signal to become Green before you can move your vehicle!

When the thread is waiting for a sync-object to become signaled, it is said to be blocked. The following diagram shows a typical usage on how a thread waits on a sync-object, uses a resource, and then releases the sync-object:

The method names shown above in green are not actual, but only indicative. It must also be noted that the synchronization-object is not the resource you will be using. It is just a mechanism to protect unsafe simultaneous access to resources.

Continuing with Events

Events, one of the thread synchronization objects/primitives, are amongst the simplest sync-objects. They are used to raise an event that an event has occurred, so that other thread can perform some action. Threads can set the event (signaling), reset the event (non-signaling), and can wait for an event to be signaled.

The following table shows the thread primitive in different frameworks:

What / Where	Windows API	MFC Library	.NET Framework
Event data-type	`HANDLE`	`CEvent` class	`ManualResetEvent`, `AutoResetEvent`
Creating events*	`CreateEvent`	`CEvent` constructor	new `ManualResetEvent`, new `AutoResetEvent`
Waiting for events	`WaitForSingleObject`	`CEvent::Lock` method	`WaitOne` method
Raising events (signaling)	`SetEvent`	`CEvent::SetEvent` method	`Set` method
Set event to non-signaled	`ResetEvent`	`CEvent::ResetEvent` method	`Reset` method

* You can create events in either signaled or non-signaled mode.

Clearly understand that acquiring the sync-object successfully using 'Wait' implicitly means locking (non-signaling) the sync-object. You need not call 'Reset' to lock it. You call 'Set' to set it free (signaled) for other threads. Yes, yes, I know you need a practical example!

All this for allowing the user to 'Cancel'?

Yes! We just need to create a manual-reset event. Set it to non-signaled (i.e., no event is raised yet!). When the user clicks on Cancel, we set the event status to signaled (event raised!).

How would the other thread know? Well, on each iteration, the thread will 'wait' for the event, to test if it has become signaled. The wait timeout would exactly be zero! That means, wait should return immediately! The return value would tell us if we got the event (signal) or not. If we get the signal, we return from the thread! (Yeepeee! The thread can exit now!)

Here is the code (finally!):

ManualResetEvent CancelEvent = new ManualResetEvent(false);
Thread MyThread;

private void btnStartThreaded_Click(object sender, EventArgs e)
{
    // Other content is same as in previous example.
    // Here 'MyThread' is moved to class-level. See comments below this code.
    MyThread = new Thread(ProcessRoutine);
    MyThread.Start();
}

private void btnCancelButton_Click(object sender, EventArgs e)
{
    // Raise (signal) the event
    CancelEvent.Set();

    // Wait for the thread to finish
    // (i.e. let it process the signal request, and return)
    MyThread.Join();    // Wait for thread to exit!

    // Notify the user (see comments after code)
    lblStatus.Text = "Cancelled!";
}

void ProcessRoutine()
{
    for (int nValue = StartFrom; nValue <= EndTo; nValue++)
    {
        // Check on each iteration if event was raised (signaled)
        // that implicitly means if user requested to 'Cancel'
        if (CancelEvent.WaitOne(0, false) == true)
        {
            return;
        }

        this.BeginInvoke(this.TheDelegate, nValue);

        // Simulate processing, so we are sleeping for 200 milliseconds (.2s)
        Thread.Sleep(200);
    }
}

Comments about the code:

MyThread is moved to class level, so that we can wait for thread termination in btnCancelButton_Click (Thread.Join).
Sleep and BeginInvoke are needed so that the user can actually click the 'Cancel' button. If you have run Example-1 correctly, you'd have realized the form was not quite responsive.

More about BeginInvoke

(Applicable to the PostMessage API too): Sending the message (synchronously) means the sender thread will block until the target finishes. We need two threads to work parallel/concurrently in a literal sense. Notifying the UI thread (primary thread) using Invoke/SendMessage was actually causing the two threads to run serially. This way, the primary thread was also busy - and thus, the user would not be able to click 'Cancel' properly. Using 'Sleep', we can let the worker-thread to simulate the actual work, and let the primary-thread also be free for sometime. This way, the user may be able to click Cancel. But, still, both threads would not be running parallel. So I chose to use BeginInvoke. See the image after the C++ code.

As a last note: BeginInvoke/PostMessage posts messages to the target window, and there could be a delay by the target window/thread to process it. This means, the sender thread may exit completely before the target consumes it. That eventually means, Thread.Join (or WaitForSingleObject, below) may return before the Posted message was processed by the window. That is the reason "Cancelled" may not appear always!

Tall order, I know I know! Multi-threading needs more of practice than reading stuff like this (but read mine!). Try changing the code the way you like, and you'd know more of the bits and pieces!

Here is the code for Visual C++ programmers:

void CExample2_Dlg::OnBnClickedCancel()
{
    GetDlgItem(ID_CANCEL)->EnableWindow(FALSE);

    // Let thread know about this event
    CancelEvent.SetEvent();

    // Wait till target thread responds, and exists
    WaitForSingleObject(ptrThread->m_hThread,INFINITE);

    // update status text
    StatusText.SetWindowText("Canceled!");
}

static UINT Example2Thread(void*);

void CExample2_Dlg::OnBnClickedOk()
{
    // Get start and end values
    StartFrom = GetDlgItemInt(IDC_START);
    EndTo = GetDlgItemInt(IDC_END);

    // Set progress bar range
    ProgressBar.SetRange(StartFrom, EndTo);

    // Disable button
    GetDlgItem(IDOK)->EnableWindow(FALSE);
    GetDlgItem(ID_CANCEL)->EnableWindow(TRUE);

    // Start off the thread
    ptrThread = AfxBeginThread(Example2Thread, this);
}

UINT Example2Thread(void *pParam)
{
    CExample2_Dlg* pThis= (CExample2_Dlg*)pParam;

    for (int nValue = pThis->StartFrom; nValue <= pThis->EndTo;  ++nValue)
    {
        if( pThis->CancelEvent.Lock(0) )
            return -1;

        pThis->PostMessage(WM_MY_THREAD_MESSAGE, nValue);

        Sleep(200);
    }

    return 0;
}

Here is the graphical representation of the synchronous and asynchronous communication (Synchronous and Synchronization are two different words with different meanings!!)

Does that mean we should always use asynchronous message-passing? Absolutely not! There are instances where you need to perform serial communication, i.e., wait for the target thread/window to process your request. For example, in a file-copy program, you must let the user know the correct file name being copied. Using asnyc mode, you may mislead the user that the old file is still being copied, because of a delay in processing messages by the target thread/window. And lastly, (one or more) filename(s) may not be displayed to the user at all, since your file-copy-thread has already finished copying! The user might restart copying all the files again, and eventually after sometime, stop using your application! ;)

Practical Example 3

You need to allow the user to Pause/Cancel processing along with Cancel:

We can use either of two approaches:

Using thread-suspend and thread-resume features, to Pause and Resume the processing, respectively.
Using a proper synchronization mechanism.

The first approach is straightforward, which is not recommended:

bool IsThreadPaused;
// IsThreadPaused = false; (before Thread.Start)
private void btnPause_Click(object sender, EventArgs e)
{
    // Bad approach!
    if (!IsThreadPaused)
    {
        IsThreadPaused = true;
        MyThread.Suspend();
        btnPause.Text = "Resume";

        lblStatus.Text = "Paused!";

        // Disallow Cancel
        btnCancelButton.Enabled = false;
    }
    else
    {
        IsThreadPaused = false;
        MyThread.Resume();
        btnPause.Text = "Pause";

        btnCancelButton.Enabled = true;
    }
}

And the code for MFC guys:

void CExample3_Dlg::OnBnClickedPause()
{
    if( ! m_bIsThreadSuspended )
    {
        m_bIsThreadSuspended = true;
        ptrThread->SuspendThread();

        GetDlgItem(ID_CANCEL)->EnableWindow(FALSE);

        m_cPauseButton.SetWindowText("Resume");

        StatusText.SetWindowText("Paused!");
    }
    else
    {
        m_bIsThreadSuspended = false;
        GetDlgItem(ID_CANCEL)->EnableWindow();
        m_cPauseButton.SetWindowText("Pause");
        ptrThread->ResumeThread();
    }
}

I will cover the second approach to pause/resume thread in the next part of this series. If you can find out how to do that, I encourage you to do so!

Show Closing Time!

The code supplied in the article (and in downloadable files) is not perfect. The UI design is bad, and does not cope with how the user may actually use the application. Furthermore, error checking is not done, at almost all places. And things like that. My whole concentration was on the article and giving a naive multithreading programmer an idea of how and when to use multithreading. I tried to make the article as simple as possible so that everyone can get a clear picture about multithreading.

If you've doubts, drop in comments; if you don't like something, please do let me know. If you've ideas or questions that can assist me in writing better articles, please hit a few keys!

Next Part (April 5, 2010): The Practical Guide to Multithreading - Part 2.

Happy threading!