Object Prevalence With Bamboo Prevalence

• Download demo project - 20 Kb
• Download source - 41 Kb

Why should I read this article?

Let me think… How about these claims:

• Your data should always live in memory.
• You don’t need an RDBMS to store your data.
• You can retrieve and process your data faster than with an RDBMS and SQL.
• Bamboo Prevalence is really, really cool.
• It’s so easy to use, that you’ll be able to use it right after reading this article.

Ok! I’m in… What is it?

Bamboo Prevalence (http://sourceforge.net/projects/bbooprevalence/) is an opensource .NET implementation of the Prevayler concept that you can read more about on http://www.prevayler.com/.

The idea behind prevayler is to keep all your data in memory. Prevayler has mechanisms that allow you to persist data without the use of a RDBMS, thus, it can be seen as a RDBMS killer. Instead of creating a relational data model, you’ll be creating objects. Only objects and no mapping code to or from a relational database. Just the clean objectmodel. Doesn't that sound wonderful?

Some of the main benefits are, as I will show here, is that:

• Bamboo Prevalence has a low startup cost. Both in the way of human capacity; since it is easy to learn and to get started with, and in the economical way, since there are no license costs for it…
• It lets us design our object model without regard to how we would model our problem in a relational database.

We'll go through these in the following sections.

The idea...

So the main idea is to skip the RDBMS and keep all objects in memory. Doesn’t that sound easy enough? Working with all our data in memory sounds like a great idea. That allows for extremely fast access to our data, and makes it easy to model our data object-oriented.

My memory is short, and I must admit that after years of doing relational databases, I started to think that I was working quite OO, also with data from relational databases. I wasn’t. Very often I found myself modeling my classes and objects around the notion of how the data were stored in the relational database, and what SQL statements I would use to retrieve or modify my data.

So basically, we’re just creating our object while the program runs. We’re going to need a way to store our data, so that it isn’t lost when we need to stop our application, we reboot the server or our application crashes. The Microsoft .NET Framework makes it easy for us to store our objects to disk.

formatter = new BinaryFormatter();
FileStream fstream = new FileStream("filename", FileMode.Create);
formatter.Serialize(fstream, object_to_serialize);
fstream.Close();

That sounds nice, doesn’t it? We can create our objects and store them to disk. It’s just as easy to retrieve it (deserialize) from disk into memory.

formatter = new BinaryFormatter();
FileStream fstream = new FileStream("filename", FileMode.Open);
formatter.Serialize(fstream, deserialized_object);

The only requirement to make this work is that the type(s) being serialized and deserialized either are marked with the SerializableAttribute or that they implement the ISerializable interface.

Well, enough about that… Let’s go back and look at the main idea. To get into the heart of Bamboo Prevalence, we need to take a look at and discover what’s missing.

So what are we missing? We are missing one of the main points when we’re working with RDBMSs. When working with relational databases, we tend to model our application around the database, meaning we store the data to the database once we create it. At any time, if our application crashes, or the server looses power, we will have a database containing all the data we created until our application stopped.

The before mentioned simple model of storing our object or object tree doesn’t work that way. At least not unless we do it every time we do an update to the data. This approach is quite expensive in terms of CPU and disk I/O, particularly when the amount of data grows.

We need a better solution.

Looking at MS SQL Server, it has two mechanisms that allow it to either change part of the file containing data (storing a page to disk) or just writing the changes to the transaction log as they are applied to the data.

Bamboo Prevalence gives us one of these solutions for free. We get the transaction log, or as it’s called in Prevaylance; the command log.

Bamboo Prevalence intercepts methods we call on an object, and writes information on what method is called, and the parameters to a commandlog file. This log can be used to restart the system, and recreate all the data. Bamboo Prevalence will simply start by playing back all logged method calls running them in sequence. After that we should have the exact same data as we had before we stopped the system.

Lets put this to work

We'll will demonstrate the above by creating a very small application, lets call it ContactManager. This will be a very simple system to store some info about our contacts, but complex enough to show the usability.

The specifications or user stories

• A register of contacts.
• A contact contains a name, a postal address, a phonenumber and an email address.
• It must be possible to add a contact. It must be possible to retrieve a stored contact.
• We expect to store 100.000 contacts

Since we are going to store all contacts in memory, memory can be a limiting issue on whether or not we are able to use object prevalence. So we need to find out how much memory we need.

The numbers...

Lets say a contact on average take up 250 bytes of data. (That breaks down to 25 characters for name, 50 for postal address, 25 for email address and 10 for phone number. That makes a total of 110 Unicode characters or 220 bytes. I added 30 additional bytes overhead for references and other object internals.)

We must be able to store 100.000 contacts, each using 250 bytes. We expect our data to use 25MB memory.

Even if we miss the number of contacts by a factor of 10, and the contact objects double in size, we’re going to be within the memory limits on a modern server.

The architecture (without bamboo, skeleton code only)

A Contact class that contains the data for a contact.

public class Contact
{
    ...
}

A Hashtable represents our datastore. It contains all our Contact objects.

public ArrayList mContacts = new ArrayList();

public class ContactManager
{
    public Contact CreateContact(...)
    {
        ...
    }

    public Contacts[] GetAllContacts(...)
    {
        ...
    }

    ...
}

A Contact object can be retrieved from the ContactManager class by an identifier. This identifier will be the same as the name of the contact, and will be used as the key in the hashtable. The contact will be retrieved when the lookup identifier value matches exactly the identifier string of the stored contact.

The system must also have some sort of UI. Since the logic in this system is presented as a class library, you can choose what kind UI you’re going to make for your system. A web application, Windows forms application or just a simple command line application will work equally well with this code.

In our example, the system is a simple single user Windows forms application. The code for the Windows forms application is included in the downloadable source.

Creating an object model (full code)

The Contact class is quite simple. It is the storage for all information about one contact, and has no methods.

public class Contact
{
    private string mName;
    private string mPostalAddress;
    private string mEmailAddress;
    private string mPhoneNumber;

    public string Name{get{return mName;}}
    public string PostalAddress{get{return mPostalAddress;}}
    public string EmailAddress{get{return mEmailAddress;}}
    public string PhoneNumber{get{return mPhoneNumber;}}

    public Contact(string name, string postalAddress, 
        string emailAddress, string phoneNumber)
    {
        mName = name;
        mPostalAddress = postalAddress;
        mEmailAddress = emailAddress;
        mPhoneNumber = phoneNumber;
    }
}

The ContactManager class is a singleton. This means that there can be only one instance of the class. The details of the singleton pattern are shown in the #region Singleton implementation block.

using System;
using System.Collections;

public sealed class ContactManager
{
    #region Singleton implementation
    public readonly ContactManager Instance = new ContactManager();
    private ContactManager()
    {
        //The constructor is made private, to make sure that 
        //no new instances of it can be instantiated outside
        //of this class.
    }
    #endregion

    public event EventHandler ContactAdded;
    private ArrayList mContacts = new ArrayList();

    //This method shows a simple implementation of the Factory pattern
    public Contact CreateContact(string name, string postalAddress, 
           string emailAddress, string phoneNumber)
    {
        Contact contact = new Contact(name, postalAddress, 
             emailAddress, phoneNumber);
        mContacts.Add(mIdGenerator, contact);

        //Raise event
        ContactAdded(this, null);

        return contact;
    }

    public Contact[] GetAllContacts()
    {
        return (Contact[])mContacts.ToArray(typeof(Contact));
    }
}

The full source code for this and the Windows Form application to test it, is in the version 1 folder included in the sourcecode download.

The ease of extending the object model to use Bamboo Prevalence

When running the WinForms application, we see that the model works, but our data is lost when we stop and restart our application. This is where Bamboo Prevalence comes in.

We’re going to add logging functionality to the ContactManager class using Bamboo Prevalence.

Of course, before we can use Bamboo Prevalence, we need to add a reference to it from our project. You’ll find the assembly Bamboo.Prevalence.dll in the "bin" folder where you extracted the "Bamboo.Prevalence.1.4.1.zip" file. (I've extracted it to the "c:\program files\bamboo prevalence\" folder.)

Next, We’re going to change some of the code in ContactManager, to make it prevalent.

using System;
using System.Collections;
using System.IO;
using Bamboo.Prevalence;
using Bamboo.Prevalence.Attributes;

[Serializable]
public sealed class ContactManager : MarshalByRefObject
{
    #region Singleton and prevayler implementation
    static private readonly PrevalenceEngine mEngine;
    static public readonly ContactManager Instance;

    static ContactManager()
    {
        string path = Path.Combine(Environment.CurrentDirectory, "Data");
        mEngine = PrevalenceActivator.CreateTransparentEngine(
            typeof(ContactManager), path);
        Instance = mEngine.PrevalentSystem as ContactManager;
    }

    private ContactManager()
    {
    }
    #endregion

    public event EventHandler ContactAdded;
    private ArrayList mContacts = new ArrayList();

    public Contact CreateContact(string name, string postalAddress, 
         string emailAddress, string phoneNumber)
    {
        Contact contact = new Contact(name, postalAddress, 
             emailAddress, phoneNumber);
        mContacts.Add(contact);
        
        //Raise event
        ContactAdded(this, null);

        return contact;
    }

    [Query]
    public Contact[] GetAllContacts()
    {
        return (Contact[])mContacts.ToArray(typeof(Contact));
    }
}

To make ContactManager a prevelence root class, it must be made serializable and inherit from MarshalByRefObject. Notice the changes in the #region Singleton and prevayler implementation block to see how we implement Bamboo Prevalence on top of our existing objectmodel.

In addition to making the ContactManager class the prevalence root class, we’ve added the QueryAttribute to the RetrieveContact method. The RetrieveContact method will never change any data, and it is not necessary to record it in the commandlog. At least not for the sake of persisting our actual data. The Query attribute marks that use of a method will not be logged, and should be applied to all methods in the prevalent root class that don’t modify data.

The Contact class also needs some modification. Since we are going to save it to disk now, we need it to be serializable.

using System;

[Serializable]
public class Contact
{
    ...
}

The full source code for this and the Windows Form application to test it, is in the version 2 folder included in the sourcecode download.

Now we can test the code again. If you run the code in version 2, you will see that these small changes makes the same object model persistent. You can start and stop the application as you like without loosing any data.

Is this it?

Well, almost. At least until you realize that at one point in time the commandlog will be getting very large, and playing it back is going to take time.

With Bamboo, we can take a snapshot of our object or object tree at any moment, so that we don’t need to playback all our object changes each time we’re starting our application. On startup of our application, Bamboo will first load the last snapshot (if any exists) and then append the commandlogs for any changes applied after the last snapshot was taken.

To make a snapshot, the application has a TakeSnapshot() method to the ContactManager class.

public class ContactManager
{
    ...

    [PassThrough]        //Its not necessary to log this...
    public void TakeSnapshot()
    {
        mEngine.TakeSnapshot();
    }
}

This method (along with the addition of a button) is also included in the version 2 folder in the sourcecode download.

When running the application now, you can experiment with adding data, taking snapshots, restarting the application and removing older files, and you’ll see how the snapshot and commandlog files work together.

Links to more information

You can find the full source code and compiled files for Bamboo at the project information page or the Bamboo Prevalence homepage on Sourceforge. This site has a FAQ, feature requests page and more for Bamboo Prevalence. Download the full package, including docs, source, bins, tools and samples from there. Look at the samples to get more ideas on how to use this.

The http://www.prevayler.org/ website where this idea stems from has a lot of good startingpoints. They also maintain the "Prevalence Sceptical FAQ" on object prevalence.

Peter A. Bromberg takes this example a step further in his article on EggHeadCafe.com, creating a ASP.NET application that uses Bamboo Prevalence.

Stephan Meyn has written an article here on CP, where he looks into using XSD schemas to generate code for use with Bamboo Prevalence.

Licence

Bamboo Prevalence is licensed under the GPL licence, with one important exception (as copied from the source code):

"As a special exception, if you link this library with other files to produce an executable, this library does not by itself cause the resulting executable to be covered by the GNU General Public License. This exception does not however invalidate any other reasons why the executable file might be covered by the GNU General Public License."

Future exploration...

There is a lot more to Bamboo Prevalence, and I hope this article has inspired you to look further into this exiting technology. Try it out yourself.

Some topics to look at include:

• Automating snapshot taking and removal of old files.
• Indexing, searching and filtering.
• Performance.
• Transactions.
• Special considerations when coding.
• When to use and when not to use object prevalence.

Terminology

• OO: Object Oriented
• RDBMS: Relational DataBase Management System. SQL Server or Oracle are examples of RDBMSs.

History

• October 10th, 2003: Released to codeproject.com
• October 18th, 2003: Fixed bug in code, and changed attribute usage on TakeSnapshot method.
• November 1st, 2003: Added more links to more information, fixed several typos.