Iterators and Snapshots for LevelDB UWP

Introduction

In my last article, I introduced the basics of LevelDB. I introduced the basics on how to install the library and use basic Get, Put, Delete, and Batch operations. In this article, I will go a little deeper and introduce iterators, and snapshots feature of LevelDB.

Creating Snapshots

Snapshots are extremely powerful constructs that let you freeze a particular view for your read operations. Snapshots provide consistent read-only views for all the read operations. When reading values from database, it's quite possible other instances of DB objects are making changes to database. In certain scenarios, you might want to freeze a snapshot of database at a particular point in time. This is where Snapshot object will help you. Now since Snapshot has to keep some additional data and resources while you are doing reads, it must be released once you are done with your operation. Creating and releasing snapshot is super simple:

using (var snapshot = db.GetSnapshot())
{
    ...
}

If you are not writing using blocks in your code, be sure to call snapshot.Dispose().

Using Snapshots

Each ReadOption object can take a Snapshot property, and each read operation requires you to pass ReadOption object. Once you have snapshot, you can use it as the following code illustrates:

using (var snapshot = db.GetSnapshot())
{
    var val = db.Get(new ReadOptions { Snapshot = snapshot }, Slice.FromString("foo"));
}

Iterators

Iterators of course are required when you want to iterate over the data stored in your key-value store. Now I won't go into details of comparators but by default (LevelDB UWP) orders data by key bytes lexicographically. This behaviour can be customized with comparators, but it's out of scope for this article. So the best way to summarize the default key order is imagining my key value store has the following key value pairs inserted:

db.Put(Slice.FromString("b"), Slice.FromString("Two"))
db.Put(Slice.FromString("d"), Slice.FromString("Three"))
db.Put(Slice.FromString("a"), Slice.FromString("One"))

It would be stored in the following order:  a => One, b => Two, d => Three. This means iterating from beginning to end, you will encounter keys in order of a, b, c since a < b < c. Keeping this in mind, let's create an iterator that iterates over all key value pairs in database:

using (var itr = db.NewIterator(new ReadOptions()))
{
    itr.SeekFirst();
    while (itr.Valid())
    {
        byte[] key = itr.Key().ToByteArray();
        byte[] val = itr.Value().ToByteArray();
        itr.Next();
    }
}

As you can see, just like anything else, an iterator must be disposed when you don't need it anymore. Iterator has methods like Seek, SeekToFirst, SeekToLast, Valid, Next, and Prev to navigate between your records (Checkout documentation). Additionally, it provides method Key, and Value to read the slices themselves.

Using Seek, you can do powerful things like iterating over records with key X..Y, for example:

using (var itr = db.NewIterator(new ReadOptions()))
{
    itr.Seek(Slice.FromString("record001"));
    while (itr.Valid())
    {
        var keySlice = itr.Key();
        if (keySlice.ToString() == "record007") break;
        // use keySlice
        itr.Next();
    }
}

The same can be done in reverse order:

using (var itr = db.NewIterator(new ReadOptions()))
{
    itr.Seek(Slice.FromString("record007"));
    while (itr.Valid())
    {
        var keySlice = itr.Key();
        if (keySlice.ToString() == "record001") break;
        // use keySlice
        itr.Prev();
    }
}

Just remember doing iteration in reverse order is an expensive operation and thus a little slower. Try using forward iterations whenever possible.

Combining Snapshots and Iterators

We can fuse iterators with snapshots to create really powerful features like getting set of key-value pairs at a particular instance in time while mutations are happening (something close to MVCC). It can't be any simpler:

using (var snapshot = db.NewSnapshot())
using (var itr = db.NewIterator(new ReadOptions({ Snapshot = snapshot })))
{
  // use itr
}

Combining WriteBatches and Iterators with proper synchronization can yield powerful results. Imagination is your limit.

Conclusion

This completes the basic usage of LevelDB. There is more advanced stuff that I will visit in future articles. You can look into the detailed documentation by going to LevelDB UWP Wiki.