Distributed Cache using Redis and ASP.NET Core

• Download sample solution (display Twitter messages) - 42.4 KB

What is Redis?

Redis is a super fast non-relational database that uses keys to map to different data types. It is a key value data store (NoSQL) allowing to efficiently solve many different problem sets. Redis was created by Salvatore Sanfilippo in 2009, and Sanfilippo still remains the lead developer of the project today. It is a mature and hugely popular open source project, being used by many companies and in countless mission-critical production environments.

Here is an interview with the inventor of Redis, Salvatore Sanfilippo.

Why is Redis Popular?

Not only is it extremely effective, but it is also relatively simple. Getting started with Redis is quite fast, and it usually takes only a few minutes to set up and get them working within an application. Thus, a small investment of time and effort can have an immediate, dramatic impact on performance of the application.

Just to name two cases when Redis is helpful:

• Redis is used at Pinterest – see use case
• or at Twitter, where Raffi Kirkorian, VP of Engineering at Twitter, explains how Redis helps to support over 30 billion timeline updates per day based on 5000 tweets per second or 400,000,000 tweets per day – see the presentation here.

To install

Here are all the things needed to be installed locally (except Redis Server which is presented below):

• Visual Studio Community 2015 with Update 3 (or just Update 3 if you have already installed) and .NET Core 1.0.1 - VS 2015 Tooling Preview 2
• .NET Core 1.1 SDK - the solution uses .NET Core 1.1, but it could be easily converted to run with .NET Core 1.0

Installing Redis Server on Windows

Create a new ASP.NET Core Web Application (.NET Core) project. Once the project is created in Visual Studio, install using Nuget the server package: Redis-64. The server is then installed in the default Nuget path. To start the server, just run next in command prompt:

C:\Users\[logged in user]\.nuget\packages\Redis-64\[version installed]\tools>redis-server.exe

# in my case
C:\Users\petru\.nuget\packages\Redis-64\3.0.503\tools>redis-server.exe

In the same folder, there is a document describing how to install Redis as a Windows Service (check the file Windows Service Documentation.docx). For running the simple scenarios from this article, running just from command line should be enough.

Caching in ASP.NET Core using Redis

To use Redis in ASP.NET Core, we need to reference Microsoft.Extensions.Caching.Redis.Core package. Additionally, in our sample, we would also need the Session package. Installing these could be done either using Nuget or extending the project.json file:

"Microsoft.Extensions.Caching.Redis.Core": "1.0.3",
"Microsoft.AspNetCore.Session": "1.1.0

To enable Redis in the application, we need to add AddDistributedCache method in ConfigureServices method.

public void ConfigureServices(IServiceCollection services)
{
    services.AddDistributedRedisCache(options =>
    {
        options.InstanceName = "Sample";
        options.Configuration = "localhost";
    });
    services.AddMvc();

To enable the session, we need to make changes in both ConfigureServices and Configure:

public void ConfigureServices(IServiceCollection services)
{
    services.AddDistributedRedisCache(options =>
    {
        options.InstanceName = "Sample";
        options.Configuration = "localhost";
    });

    services.AddSession();
    services.AddMvc();
}

public void Configure(IApplicationBuilder app, 
    IHostingEnvironment env, ILoggerFactory loggerFactory)
{
    app.UseSession();
    app.UseMvc(routes =>
    {
        routes.MapRoute(
            name: "default",
            template: "{controller=Home}/{action=Index}/{id?}");
    });
}

How Do We Access Redis?

To cache a value in Redis, we use:

var valueToStoreInRedis = Encoding.UTF8.GetBytes("This is a cached value from Redis");
HttpContext.Session.Set("TestProperty", valueToStoreInRedis)

To retrieve the value from cache, we use:

var valueFromRedis = default(byte[]);
if (HttpContext.Session.TryGetValue("TestProperty", out valueFromRedis))
    valueToDisplay = Encoding.UTF8.GetString(valueFromRedis)

Twitter Client Example

External interfaces to connect to social platforms are usually relatively slow to access. If we would do a web application showing messages from Twitter, it would take a couple of seconds to load the page (to make the search and retrieve). If the same user would connect again to our application, Redis could retrieve from memory the same messages, without accessing the server again. Caching the results, and updating them only when new messages appear, bring a huge improvement to the overall performance.

To keep things simple, I have earlier written a small client in Python, that connects to Twitter and saves the details in a JSON file. You can read more details in the article Visual Studio Code – Connect to Twitter with Python. It includes the full source code, and you can access and save data in a JSON file very fast.

In our sample here, we start from an already available JSON file.

[
 {
        "Id": 0,
        "ProfileImage": "https://pbs.twimg.com/profile_images/1772973596/inogiclogo_normal.png",
        "ProfileDescription": "Microsoft Partner with Gold Competency in #Dynamics365 
         #MSDynCRM providing Solns/Services. Innovators of @Maplytics & 
         Inolink #QuickBooks Int #MSDyn365 #MVPBuzz",
        "Username": "Inogic",
        "Text": "Execute the Global Action Using Web API in Dynamics CRM 
         https://t.co/DAuzP6L7FE #MSDyn365 #webapi https://t.co/v0XgyotaFn",
        "ScreenName": "@inogic"
    },
   ....
]

Retrieval of data from Twitter takes a couple of seconds. To simulate this, we add a delay of 20 seconds in our code. Here is the code from the controller.

public IActionResult Index()
{
	var watch = Stopwatch.StartNew();
	string jSONText = RetrieveOrUpdateRedis();
	watch.Stop();

	TempData["DataLoadTime"] = watch.ElapsedMilliseconds;
	var itemsFromjSON = JsonConvert.DeserializeObject<IEnumerable<TwitterItem>>(jSONText);
	return View(itemsFromjSON);
}

private string RetrieveOrUpdateRedis()
{
	var valueFromRedis = default(byte[]);
	string valueToReturn = string.Empty;
	if (HttpContext.Session.TryGetValue("TwitterDataset", out valueFromRedis))
	{
		// Retrieve from Redis
		valueToReturn = Encoding.UTF8.GetString(valueFromRedis);
		TempData["DataLoadType"] = "From Redis";
	}
	else
	{
		// read the file and update the URLs
		var jSONText = System.IO.File.ReadAllText("twitter.json");
		valueToReturn = GetUpdatedFileContent(jSONText);

		// add an artificial delay of 20 seconds 
        // (simulating the search is done directly against a Twitter server)
		Thread.Sleep(20000);

		// store values in Redis
		var valueToStoreInRedis = Encoding.UTF8.GetBytes(valueToReturn);
		HttpContext.Session.Set("TwitterDataset", valueToStoreInRedis);
		TempData["DataLoadType"] = "From file";
	}

	return valueToReturn;
}

// a minimum data processing, updating the URLs
private string GetUpdatedFileContent(string jSONText)
{   
	var itemsFromjSON = JsonConvert.DeserializeObject<IEnumerable<TwitterItem>>(jSONText);
	foreach (var item in itemsFromjSON)
	{
		Regex r = new Regex(@"(https?://[^\s]+)");
		item.Text = r.Replace(item.Text, "<a href=\"$1\">$1</a>");
	}

    return JsonConvert.SerializeObject(itemsFromjSON);
}	

And here is the code to display data in the view. Twitter messages are displayed within a table:

@{
    ViewData["Title"] = "Home Page";
}

@model IEnumerable<WebCache.Models.TwitterItem>

<h2>Twitter messages #webapi</h2>
<h4>Time to load the data: @TempData["DataLoadTime"]ms</h4>
<h4>From where is loaded: @TempData["DataLoadType"]</h4>

<table  class="bordered">
    <thead>
        <tr>
            <th>#</th>
            <th>Message</th>
            <th>Who</th>
        </tr>
    </thead>
    <tbody>
        @foreach (var item in Model)
        {
            <tr>
                <td>
                    @item.Id
                </td>
                <td>
                    @Html.Raw(item.Text)
                </td>
                <td>
                    <img src="@item.ProfileImage" />
                    <br /> @item.Username
                    <br /> @item.ScreenName
                </td>
            </tr> 
        }
    </tbody>
</table>

How Fast Does the Application Run with Redis?

The initial time to load is displayed below (this also includes the artificial timeout of 20 seconds):

Just taking the results from Redis runs much faster. This excludes any “connection” to Twitter or any other internal update. Here is the print screen with the data retrieved from memory:

To replicate this multiple times, and not just at the beginning, use command flushall to clean the entire Redis cache:

C:\Users\[logged in user]\.nuget\packages\Redis-64\[version installed]\tools>redis-cli.exe

# in my case
C:\Users\petru\.nuget\packages\Redis-64\3.0.503\tools>redis-cli.exe

# and then
127.0.0.1:6379> flushall

Easy Access to the Code

The full Twitter sample solution could be download from the top of the article or found on Github – https://github.com/fpetru/redis-aspnetcore-caching.

Is Redis Just A Cache? When to Use It?

Redis is much more than just a cache. Like any cache, Redis stores indeed [key, value] pairs. More than this, Redis allows us to use values in a very efficient ways. Using different data structures (not just strings), we gain a lot of power (such as the ability to fine-tune cache contents and durability) and greater efficiency overall. Once we start using the data structures, the efficiency boost becomes tremendous for specific application scenarios.

Here is a summary of the data structures, along with few concrete use cases associated with each type:

	Uses
Strings	Caches, counters, and clever bit-operations. Session management is key to many online application. Their success depends by responsiveness and accuracy of data displayed, just to name here shopping carts for eCommerce websites. Distributed cache, it is a great alternative to the local cache.
Lists	Queues, stacks and cyclical lists. All these help to decouple processes, while receiving many new requests. A simple example would be an web application that is used to place orders. The processing of the orders may take a while, and these could be decoupled using queues.
Hashes	These are field-value pairs, very efficient when storing many small values. An example would be for implementing of a black lists checking feature. Or accessing very fast lots of images from a gallery.
Sets	Are very good for expressing relations between objects. These could make easy implementation of social tags in social networking applications, or discussion tags in blogs, etc.
Sorted Sets	These are a mix between sets and Hashes. One example would be online games, where scoreboards and player statistics are vital.
HyperLogLog	Counting unique things. A simple use case, counting unique queries performed by users in a search form during the day.