An Introduction to Jibu for .NET

What is Jibu?

Jibu is a library for .NET 2.0 and above which acts as a wrapper around the .NET framework multithreading functionality. It aims to abstract the programmer away from the low-level details of creating & using threads, and how to implement safe inter-thread communication. This frees the developer to focus on the actual logic they wish to parallelize, rather than how to make the multithreading work properly.

The Jibu home-page is at http://www.axon7.com. Free versions for Java & .NET are available to download, you can also access support forums there.

As well as helping to reduce development time by saving the programmer from having to write tricky multi-threading functionality, Jibu can significantly reduce the time spent debugging the code when bugs are found. Multi-threaded applications are notorious for bugs which are hard to isolate and reproduce, and as multicore systems with true parallel thread execution become the norm, these bugs become even more likely. On a single-core system, incorrect code can often function without problems since multiple threads don't actually run simultaneously (instead the operating system quickly switches between threads, giving each a short slice of time), but on a multicore system threads must be used correctly. Fixing bugs is one of the hardest parts of the development process to estimate, and tracking down threading bugs can be very time consuming - apart from the inherent difficulty of debugging threaded applications, few developers are truly knowledgeable on the correct way to use threads safely. Using Jibu, the developer can be much more confident that the multithreading code is correct. Many developers may have implemented their own threading libraries and/or utilities, but these can rarely be considered mature and given a great deal of trust - if a strange bug appears in your multithreaded application, how confident would you be that the multithreading support classes you wrote aren't to blame? By contrast, Jibu is thoroughly tested both by the developers who created it and by all the teams who use it - the more a library is used without problems, the more it can be trusted.

The methodology for using Jibu is quite different than the way threading is normally handled by the programmer. When writing a multithreaded application, the programmer traditionally thinks in terms of threads, critical sections, mutexes, semaphores, atomic actions and so on**. Many multithreaded libraries (like those provided by Java API and the C++ Boost libraries) simply wrap these constructs in utility classes, but Jibu treats things differently. Jibu is oriented around what you're trying to do, and as far as possible it wants you to forget about things like threads. There is no JibuThread class for instance. Instead, Jibu is a task-based API. You program in terms of tasks which need to be run, and Jibu takes charge of executing these tasks and allowing them to communicate. Jibu manages the low level decisions about when to create threads, how they safely interact with shared data, and so on. To see how all this works, we'll move on to look at the constructs around which Jibu is based.

** Please note that while substantial experience with traditional multi-threaded programming is not required, a basic theoretical understanding of what multithreading is will be assumed. In addition, terminology used in traditional concurrent programming (e.g words like mutex, semaphore, atomic), will sometimes be used, so a quick reference on such things might be handy. Failing that, you could just use Wikipedia like everyone else!

Key Concepts

As already mentioned, Jibu doesn't work in terms of threads, critical sections and other fundamentals of concurrent programming, it abstracts these into a task-based way of thinking. The library is based around 4 fundamental concepts: Task, Parallel, MailBox & Channel. Note that we are discussing concepts, not classes, at this point. There are far more than 4 classes!

Task

A Task is probably the easiest concept to grasp. A Task is simply a unit of work which can be run, for instance you might have a task to read some data from a file, or perform a mathematical calculation. For those familiar with Java, a Task is roughly equivalent to a Runnable. Jibu manages tasks which you create, and when a task is executed, a single thread will be used to run the task. If you want to execute 10 tasks, Jibu will use multiple threads to run the tasks simultaneously.

The core of Jibu is it's Task scheduler. This is what manages the creation and management of threads, and allocates Tasks to threads. It's important to note that the scheduler doesn't simply create a new thread for each Task. Under most circumstances, Jibu will aim to keep the number of threads equal to the number of cores of the system running the application.

Parallel

The Jibu Parallel construct is used to control the execution of multiple Tasks. A Task can be executed asynchronously but a multithreaded application often needs the ability to wait for several Tasks to complete. Parallel provides functionality to pause the main application's execution until a specified set of Tasks have completed.

Mailbox & Channel

These two constructs are both centered around allowing Tasks to communicate. There are many situations where an application/algorithm contains several totally unconnected parts, which can each be simply run as a separate Task. But there are many cases where an application has distinct parts which can run simultaneously, but need to read and write the same data. Anyone familiar with concurrent programming should know that there are specific problems which arise when you don't take precautions in this kind of situation. I don't want to teach concurrent programming here, but basically if one thread starts updating some data, another thread can read it partway through the update. Standard concurrent programming has constructs such as mutexes, critical sections and semaphores, which the programmer must use to safeguard the way data is accessed and modified.

In Jibu, we don't need to think at this level. The Mailbox and Channel concepts allow tasks to pass data around and guarantee it is done in a safe manner.

Every Task has its own Mailbox, which only it can read from. One Task may send data to another's Mailbox, but it cannot read from it. A Mailbox is used when one Task needs to pass information directly to another Task.

If there are more than 2 Tasks involved, then the Channel construct may be more useful. A Channel allows all Tasks which are linked to it to read and write data. A Task may push an object into the Channel. This remains in the Channel until a Task reads from it - at this point the object is pulled out of the Channel. In other words, data written by one Task can be retrieved by any other Task - but it cannot be read by multiple Tasks, as the act of reading removes it from the Channel. So if one Task wants to pass the same data to multiple other Tasks, it is currently recommended that the sending thread sends the data to each of the recipients' Mailboxes**

**At least this is the case in Jibu 1.0. Future versions are planned to add increased functionality for communication and data-sharing between multiple Tasks.

Using Jibu

We've so far looked at why Jibu exists, what it does, and the ideas on which it is built. Now it's time to see how the concepts translate into classes and take a look at the Jibu library itself.

Setting up a Jibu Application

This tutorial assumes you have a working knowledge of C#, using classes, inheritance and so on. If you're not very experienced then one thing you might not know is how to tell your application to use the Jibu library. Luckily though this is very simple**. In your Visual C# project, you should have the Solution Explorer window:

If you right-click on References and select Add Reference you should get a popup like this:

If you change to the Browse tab, you now need to find the Jibu.dll file, which is what contains the Jibu library. By default, it will be in a location similar to this:

Simply select jibu.dll and click OK. You should now have added Jibu as a reference and it should appear in the list of references in the solution explorer:

**This project was written using Visual C# Express 2005 against Jibu 1.0 RC2; there could be slight changes with different versions.

Core Jibu Classes

Before going on to look at the sample application, a brief overview of some of the key Jibu classes & interfaces will be given. This short look is not meant to replace the excellent class documentation. In fact I recommend that you view the class docs as essential reading. As well as a much more detailed description of each method, the docs are liberally sprinkled with useful code samples so you can see how the designers meant the library to be used.

Manager

The Manager class controls initialization of Jibu. Although it's essential to any Jibu application, we actually use this class very little. Manager's methods and properties are all static; this class is never instantiated. All that is needed is to ensure that Manager.Initialize() is called before using any other Jibu classes or functionality.

Task, Async & Future

The abstract Task class is probably the most important in the whole library. All tasks which you want to run in a multithreaded application will be written as custom subclasses of either Async or Future, which are direct subclasses of Task. Task, Async and Future are all abstract classes, they provide the functionality to run your code as a Jibu Task.

When you write your own Task, you must choose whether to base it on Async or Future. Async, as the name suggests, provides you an easy way to run your task asynchronously. Calling the start method will add your task object to the Jibu scheduler, which will execute the task, while normal program flow continues - in other words Async.Start does not block. You can later cause the main program flow to pause until the task has completed using the Async.WaitFor method.

Future is used to create asynchronous tasks as well, the difference is that Future allows a return value to be obtained from the task after it completes, through the Result() method.

Both Async and Future provide the Start method, which adds the Task to the Jibu scheduler, as well as the Cancel method, which can be used to terminate a task's execution prematurely. To create your custom task, the single method you must implement is the Run() method. This will be called by the scheduler after the Task is scheduled for execution by calling Start().

The Task class also provides the Mailbox API, primarily a Send method which allows data to be sent to another Task, and a Receive method which reads data sent by another Task.

Channel, ChannelReader & ChannelWriter

The Channel templated class embodies the Channel concept discussed earlier, which allows multiple Tasks to share data in a safe manner. It's simplest to think of a Channel as a stack or queue (you can choose either) into which objects of the template type may be pushed and pulled. Channels normally have a maximum size, if a channel is not full then a call to Write will enter data into the Channel and return immediately, otherwise the call will block until the channel has space to accept another item. When a Task calls Read, it pops the next item from the channel - this method blocks until an item is present to be removed from the Channel.

Associated with the Channel class are ChannelReader and ChannelWriter, which wrap the reading and writing of the underlying Channel. These might seem slightly pointless initially, but their existence means you can control read/write access to a Channel. For instance, pass a ChannelReader to a method and that method may read from the Channel but not write to it.

An important concept with Channels is the ability to poison a Channel (through the Poison method on a Channel, ChannelReader or ChannelWriter). Once a Channel is poisoned, any attempt to read to or write from this Channel will fail with an exception being thrown. Often, we may have a Task whose Run method is essentially a loop, reading from a Channel and processing the read data. By poisoning the Channel, we can cause this task to exit in a controlled manner instead of having to forcibly terminate it.

Parallel

The Parallel class is a utility class, providing static methods which implement simple parallel functionality. For instance, Parallel.Run accepts an Array of Tasks and blocks until all these Tasks have completed. There are also some very handy loop parallelism methods, For & Foreach, which essentially allow a for loop's iterations to be shared over multiple cores/cpus.

The Bar Sample Application

Accompanying this tutorial is the BarSample project. This project provides a simple model of a bar, where customers place orders and are served by waiters. A number of waiters and customers are generated, each customer decides at random intervals to place an order at the bar and is served by one of the waiters. An order can consist of multiple drinks of different types; each order is dealt with by a single waiter.

In terms of Jibu concepts, the application breaks down into the following functional units:

1. Each waiter and each customer is a Task.
2. There is a single Channel.
3. Customers place orders by writing to the Channel.
4. Waiters read orders from the Channel and serve the requested drinks.
5. When a waiter has served all drinks in a customer's order, they post a message to that customer's Mailbox.
6. Each customer will place a certain number of orders, when a customer's last order is completed that customer Task will end.
7. When all customer Tasks have ended, the Waiter tasks automatically terminate also.

Now, let's look at the code...

BarSampleApp.cs

This is the main class of the application. Assuming you've any knowledge of C#, it should be obvious that the static Main method is called when the application begins, creates a new BarSampleApp instance, and runs it.

There are only two methods in this class we need to look at... firstly the constructor:

private BarSampleApp()
{
    /*
     * Initialize the Jibu manager with default stack size.
     * Manager is a static class
     */
    Manager.Initialize();
}

The important thing in any Jibu application is to initialize Jibu, using Jibu.Manager.Initialize(). As the comment suggests, it is possible to set the stack size for Jibu in this method, but by passing no parameter we're choosing the default of 1Mb.

Having initialized Jibu, all the actual functionality of the application is provided in the Run method. At only around 30 lines, this is short enough we can analyze it line by line...

public void Run()
{
    CustomerTask[] customers = new CustomerTask[5];
    WaiterTask[] waiters = new WaiterTask[2];

As already described, the Bar sample models a number of customers and waiters, each represented by a Task. Here we see that the numbers of customers and waiters is hard-coded to 5 and 2 respectively. At this point, we know approximately the functionality that each task type provides, but not how it works. For now that's fine... we'll cover how the application hangs together before moving on to examine how the two custom Task classes work.

Note that of course we've not actually created any Tasks yet, just arrays to hold them.

Channel<Order> queueChannel = new Channel<Order>();

Our application uses a single Channel to provide a simple consumer/provider communication model. Customers may write Orders to the queue and Waiters may read them from the queue to process them. Note that the Channel is templated with the type of data that will be passed - namely the Order class. I'm not going to write anything about the Order class, it is just a trivial container used to note how many of each drink type are needed for each order placed.

for (int i = 0; i < waiters.Length; ++i)
{
    waiters[i] = new WaiterTask(queueChannel.ChannelReader, i + 1);
    waiters[i].Start();
}

Next, we actually create each WaiterTask. This class takes a ChannelReader and an id as construction parameters. As mentioned previously, using the ChannelReader means we can enforce the application logic, that a waiter can read an order but cannot place one.

Note that we start each WaiterTask after it is created. There is no problem with this because the WaiterTask simply starts and waits for orders to appear in the Channel... so at the end of this loop each waiter is now standing around waiting for thirsty customers to arrive.

for (int i = 0; i < customers.Length; ++i)
{
    customers[i] = new CustomerTask(queueChannel.ChannelWriter, i + 1);
}

In a very similar way, we now create our CustomerTasks. There are two differences to note... firstly we don't start the CustomerTasks yet. In Jibu, a Task can be created but is not passed to the scheduler until its Start method is called. Secondly, CustomerTasks accept a ChannelWriter as a construction parameter. Customers place orders, so they are allowed to write to the Channel but not read from it.

Parallel.Run(customers);

Remember, before this line the WaiterTasks are already started, running in threads controlled by the Jibu scheduler, but the CustomerTasks have not yet started. Parallel.Run is equally happy to accept Tasks which have or have not started - if they haven't been started it will start them. It then waits for all the CustomerTasks to complete, which is the same as saying that it waits until CustomerTask.Run exits on all customers. The method does not return until this happens. Looking at the summary of what the application does, this means that by the time this method call returns, the last customer has received his last order, and the application is nearly over. The actual ordering and serving of drinks is performed in the interactions between running CustomerTask and WaiterTask objects, communicating through the Channel, as we will see shortly.

queueChannel.Poison();

At this point, we know all the CustomerTasks have completed, because Parallel.Run has returned. However, the waiters are still patiently waiting for more orders - but since all the customers have left the bar it's time to send the waiters home. By poisoning the Channel, we will cause the waiters to encounter a PoisonException next time they attempt to read from the Channel. And the WaiterTask class is written so that when this happens, the task will end. So by poisoning the Channel, we have a convenient way to make the WaiterTasks end in a safe and controlled way.

Parallel.Run(waiters);

However, the waiters won't immediately know about the poisoning. By adding this line, we ensure that the application doesn't get any further until all WaiterTasks have actually terminated.

WaiterTask.cs

This class is a subclass of Async, and is used to model a waiter. As we've already seen, the WaiterTask receives a ChannelReader to the constructor, which is used to read orders from the Channel. It also receives an id which is simply used to identify the waiter in console messages.

The only method we need to look at is the Run method. Remember, Jibu calls this method after the Task's Start method is called.

As is obvious, the Run method loops endlessly until a PoisonException is thrown. This will happen when the Channel is poisoned, at which point the Run method, and therefore the WaiterTask, will end. Before this happens, the Run method runs a loop:

Order order = channelReader.Read();

ChannelReader will block if there is no Order in the Channel - in other words this method will not return until that time. The Task will wait, and Jibu will make sure that it's CPU use is minimal until an Order is ready to be read.

The next few lines simply extract the order information and model serving the required drinks, one at a time, with a delay between each.

Send(id, order.getCustomer().Address);
Timer.SleepFor(1000);

After serving all the drinks, a message is sent to the mailbox of the customer who placed the order. Note that we send a message using the Address of the Task, not the Task itself. In order to send a message to a Task, its address must be known.

After this, the loop pauses for second (simply to make it easier to follow the progress of the application) before starting over again.

CustomerTask.cs

CustomerTask is another subclass of Async. It also runs a loop but instead of each iteration being about reading and processing an order, the Task places a new order each iteration, until a maximum number of orders have been replaced, when it exits.

while (roundsBought<=maxRounds)
{
    int countdown = rand.Next(10, 20);
    while(countdown--> 0)
    {
        Timer.SleepFor(1000);
    }

At creation, maxRounds is randomly generated. This represents the number of orders the customer will make. At the start of each iteration, the task pauses for a random period, between 10-20s. Note the use of the Jibu.Timer class to provide Sleep functionality.

Order order = new Order(this);
...
channelWriter.Write(order);

A new Order is now created and populated. Notice that the Order maintains a refernce to the customer making the order, this allows the WaiterTask to send a mailbox message to the CustomerTask after servingthe order.

After constructing the Order object, it is written to the Channel. Because we didn't specify a buffer size for the Channel when it was created, the call to Write will block until the Order is read from the channe by a Waiter.

int val = Receive<int>();
Console.WriteLine("Customer #" + id + " got his drinks from waiter #"+val+"\n");

The call to Receive is used to read an int from the CustomerTask's mailbox. This method blocks, so it will not return until a message is posted to the customer. As already seen, this message is posted by the waiter after processing the customer's order.