Click here to Skip to main content
15,867,594 members
Articles / Web Development / ASP.NET
Tip/Trick

Virtual Cache

Rate me:
Please Sign up or sign in to vote.
5.00/5 (5 votes)
10 Oct 2014MIT4 min read 27.1K   188   12   6
Virtualization of RAM & Disk for high-speed, low-footprint & scalable data caching

Introduction

Virtual Cache is a high performance, scalable data caching implementation using the SOP data framework. This implementation uses the disk to extend memory capacity, limited only by the underlying hard disk drive (HDD). SOP provides very efficient RAM & Disk virtualization which was a feature inherited by the VirtualCache API for using SOP as the backing store for the cached data.

SOP's new, unique method that fuses together key algorithms & services for data management within the Data Store brings high degree of performance & scalability, "in-memory" performance when data fits in RAM, and close to "in-memory" performance when managing large data sets, bigger than RAM capacity. Following describes few of the key unique data handling improvements of SOP:

  • Application data container disk based B-Tree was purposely designed to work optimally with an internal caching mechanism utilizing Most Recently Used (MRU) algorithm
    • All operations occur in memory with zero data persistence when data fits in memory.
    • When data doesn't fit in memory, page swapping driven by the MRU unit between RAM & Disk occurs in a very efficient manner using Async, unbuffered I/O to produce minimal latency, very high speed data access & management.
  • Data Block Management logic that encourages bulk I/O.
  • Custom fitted (low-level) lightweight, Copy On Write (COW) based Transaction for data protection.
  • Please check out http://sop.codeplex.com for more details of SOP standouts. :)
  • Please visit this link for downloading the source code: https://sop.codeplex.com/releases/view/119752

Background

For .NET data caching, a little over two years ago, I initially implemented an Entity Framework 2nd level data caching, extending this EF caching solution by Julie Lerman. This solution has been bundled with the SOP framework part of the open source package, in SOP v4.5 release. As time went by and few releases after, the .NET 4.5 framework offered a new, general purpose standalone object caching solution, the ObjectCache derived MemoryCache implementation.

MemoryCache is a great solution for caching data using the host computer's memory (RAM). However, the amount of data that can be cached is limited by the memory available to the application. In order to alleviate this limitation, I decided to create a solution that uses the host computer's HDD for extending the memory using the available storage space on the disk. The performance & low footprint bar is high for this endeavor since I don't want to duplicate existing data caching initiatives by Microsoft and 3rd party solution providers. Microsoft's distributed caching, the AppFabric is already available and recommended for Enterprise class, multi-server caching.

However, for applications that don't benefit from distributed caching due to latencies (e.g. network latency, multi-AppFabric service orchestration, etc.) brought about by this solution, VirtualCache is now available. This is a solution optimized, designed from the ground up for virtualizing RAM and disk with minimal footprint, to offer the optimal data caching solution that fits the use-case scenario outside of what AppFabric is recommended for.

Using the Code

Please check out .NET 4.5 MemoryCache documentation & sample discussion for details on using the ObjectCache API, which is the abstract base class of (MemoryCache &) VirtualCache, in this link: http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache(v=vs.110).aspx.

Sample programs specifically for the VirtualCache implementation are available for download. The "VirtualCache" & "VirtualCacheThreaded" sample programs in the SOP v4.7 zip file (attached or downloadable from http://sop.codeplex.com) show a basic usage of this highly optimized cache virtualizer.

Also, below is a snippet to show the basic usage of the VirtualCache API:

C#
using System;
using System.Runtime.Caching;

namespace Sop.Samples
{
    /// <summary>
    /// Virtual Cache demo module.
    /// </summary>
    public class VirtualCacheDemo
    {
        private Log.Logger _logger;
        public void Run()
        {
            using (Sop.Caching.VirtualCache vc = new Sop.Caching.VirtualCache("MyCacheStore"))
            {
                _logger = vc.Logger;
                _logger.LogLevel = Log.LogLevels.Verbose;
                _logger.Information("Start of VirtualCache demo.");
                Populate(vc);
                RetrieveAll(vc);
                _logger.Information("End of VirtualCache demo.");
            }
        }
        const int MaxCacheEntries = 1000000;
        private void Populate(ObjectCache target)
        {
            _logger.Information("{0}: 
            Start of Populating target cache {1}.", DateTime.Now, target.Name);
            for (int i = 0; i < MaxCacheEntries; i++)
            {
                target.Set(string.Format("Hello{0}", i), string.Format("Value{0}", i), null);
            }
            _logger.Information("{0}: 
            End Populating target cache {1}.", DateTime.Now, target.Name);
        }
        private void RetrieveAll(ObjectCache source)
        {
            _logger.Information("{0}: 
            Start of RetrieveAll cache entries from {1}.", DateTime.Now, source.Name);
            for (int i = 0; i < MaxCacheEntries; i++)
            {
                if ((string)source[string.Format("Hello{0}", i)] == null)
                    Console.WriteLine("Failed, entry {0} was not found in cache {1}!", 
            string.Format("Hello{0}", source.Name));
            }
            _logger.Information("{0}: 
            End of RetrieveAll cache entries from {1}.", DateTime.Now, source.Name);
        }
    }
}

Instructions to create & build the sample program:

  • Download & extract the sources from SOP v4.7 zip file.
  • Open & build in release mode the SOP.sln.
  • On a new VS solution, create a new C# console project.
  • Add system reference to System.Runtime.Caching & DLL references to the newly built SOP.dll & SOP.Server.dll from the step above.
  • Paste the above code snippet to the generated code file.
  • Build and run the application.

Multi-Threaded

Virtual Cache is multi-thread safe. It has built-in locking mechanism that allows your code to use a Virtual Cache instance across different threads without worrying for data corruption.

Sample code that uses Virtual Cache in a multi-thread scenario:

C#
/// <summary>
/// Heavily threaded Virtual Cache(VC) demo module.
/// </summary>
public class VirtualCacheThreaded
{
    private Log.Logger _logger;
    const int ThreadCount = 20;
    public void Run()
    {
        List<Action> actions = new List<Action>();
        List<VirtualCache> vcs = new List<VirtualCache>();
        // create threads that will populate Virtual Cache and retrieve the items.
        for (int i = 0; i < ThreadCount; i++)
        {
            // create a new VirtualCache instance where cached data is stored together
    // with the cached data key (clustered).
    var vc = new VirtualCache(string.Format("MyCacheStore{0}", i), true);
            if (_logger == null)
            {
                _logger = vc.Logger;
                _logger.LogLevel = Log.LogLevels.Verbose;
                _logger.Information("Start of VirtualCache demo.");
            }
            // function to execute by the thread.
            actions.Add(() =>
            {
                Populate(vc, i % 2 == 0);
                RetrieveAll(vc);
                var name = vc.Name;
                vc.Dispose();
                Console.WriteLine("VirtualCache {0} was disposed.", name);
            });
            vcs.Add(vc);
        }
        List<Task> tasks = new List<Task>();
        // launch or start the threads all at once.
        foreach (var a in actions)
        {
            var t = TaskRun(a);
            if (t == null)
                continue;
            tasks.Add(t);
        }
        // wait until all threads are finished.
        if (_threaded)
            Task.WaitAll(tasks.ToArray());
        Console.WriteLine("End of VirtualCache demo.");
    }
    private bool _threaded = true;
    private Task TaskRun(Action action)
    {
        if (!_threaded)
        {
            action();
            return null;
        }
        return Task.Run(action);
    }
    const int MaxCacheEntries = 50000;
    private void Populate(ObjectCache target, bool slidingExpiration = false)
    {
        _logger.Information("{0}: 
        Start of Populating target cache {1}.", DateTime.Now, target.Name);
        for (int i = 0; i < MaxCacheEntries; i++)
        {
            target.Set(string.Format("Hello{0}", i), string.Format("Value{0}", i), 
        new CacheItemPolicy() { SlidingExpiration = new TimeSpan(0, 15, 0) });
        }
        _logger.Information("{0}: 
        End Populating target cache {1}.", DateTime.Now, target.Name);
    }
    private void RetrieveAll(ObjectCache source)
    {
        _logger.Information("{0}: Start of RetrieveAll cache entries from {1}.", 
    DateTime.Now, source.Name);
        for (int i = 0; i < MaxCacheEntries; i++)
        {
            if ((string)source[string.Format("Hello{0}", i)] == null)
                Console.WriteLine("Entry {0} 
                was not found in cache {1}, 'it could have expired and got evicted.", 
        string.Format("Hello{0}", i), source.Name);
        }
        _logger.Information("{0}: End of RetrieveAll cache entries from {1}.", 
    DateTime.Now, source.Name);
    }
}

Data Persistence

Virtual Cache supports two modes of operation:

  1. Memory Extender - In this mode, cached data are not persisted, the data files are used only to swap in/out of memory pageful of data.
  2. Persistent Caching - In this mode, cached data are persisted & Virtual Cache makes them available even when the application and/or the computer is restarted.

Setting the 3rd parameter of VirtualCache to true sets it to do persisted caching. It defaults to false.

Points of Interest

.NET 4.5 ObjectCache & VirtualCache API are good APIs to learn and use in our Apps development especially in authoring data caching solutions.

History

  • 10/05/2015 - Changed to MIT license; attached sop.zip downloadable file.
  • 10/09/2014 - Initial version. More discussions to follow, 'just wanted to get the message out about Virtual Cache availability

License

This article, along with any associated source code and files, is licensed under The MIT License


Written By
United States United States
Ex Google, ex Microsoft
I'm just fresh off from a software eng'g gig & is looking forward to my next.
Pls. do drop me a line @gerardorecinto@yahoo.com if interested or having any question/feedback on these Open Source projects.

I'm excited to help/volunteer my services.
Have a great day!

Comments and Discussions

 
QuestionProvide Code Pin
Christopher R Davis24-Feb-15 7:47
Christopher R Davis24-Feb-15 7:47 
AnswerRe: Provide Code Pin
Gerardo Recinto14-Mar-15 19:03
Gerardo Recinto14-Mar-15 19:03 
QuestionAnother Question... Pin
Dewey10-Oct-14 22:59
Dewey10-Oct-14 22:59 
AnswerRe: Another Question... Pin
Gerardo Recinto11-Oct-14 8:36
Gerardo Recinto11-Oct-14 8:36 
QuestionCorruption Pin
Dewey10-Oct-14 8:13
Dewey10-Oct-14 8:13 
AnswerRe: Corruption Pin
Gerardo Recinto10-Oct-14 10:29
Gerardo Recinto10-Oct-14 10:29 
SOP is configurable to use multi-Files if needed. And to answer your question about data corruption: if in case an external program changes the SOP data file and thus, there is no transaction protection (bcoz it is an external program), then yes indeed, you will lose your data contained in this data file.

But if your worry is about data protection within usage of SOP, it will almost be very very difficult to impossible to corrupt it because SOP sports industrial strength Transaction feature. It uses Copy on Write (COW) technique to do backup of any data changes it does and ensures any uncommitted transaction change will be rolled back during application restart or when the SOP transaction is rolled back.

It is however, recommended for IT to do backups of SOP data files. This way, in any case or situation, you can restore the data file when/if needed. This, on top of the built-in data protection schemes SOP provides.

For Virtual Cache running in "Memory Extender" mode, data file is always cleared upon application restart, so, there is no need of backups in this case.

Thank you very much for your good question. I appreciate it.
-G.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.