A Simple Key-Value Store Microservice

• Download source code - 4.3 KB

Introduction

I have a very specific use-case where I need a microservice that manages a simple in memory data store, which I call a "bucket." What surprises me is that my brief Googling for an existing simple solution came up empty, but that's par for the course as simple things grow into complex things and the simple things end up being forgotten.

My specific use case is that I need to track who is using a resource and for how long. The user has the ability to indicate that they are using a resource and when they are done using that resource, and the user has the ability to add a note regarding the resource usage. Other users can therefore see who is using a resource, how long they've been using, and any notes about their usage. Given this, my requirements are more about what I don't need than what I do need.

What I Need

1. The ability to set a key to a value.

What I Don't Need

1. I don't even need the ability to manage different buckets but it seems sort of silly not to implement that feature, so it's implemented.
2. I don't care if the memory is lost if the server restarts. This is for informational data only that will be shared across different clients. Technically, one of the front-end clients (because the client would have the full data) could even restore the data for everyone else.
3. I don't need complex data structures, just key-value pairs where the value is some value type, not a structure.
4. I don't even care about one user stepping on top of another - in actual usage, this doesn't happen, and if it did, I don't care -- whoever updates a key last wins.

Redis

"Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster. "

Yikes. Definitely major overkill for what I need. That said, please don't use this to replace something like Redis unless your requirements really fit the bill here.

The API

Browsing to http://localhost:7672/swagger/index.html, we see:

As you can see (sort of) from the Swagger documentation, we have endpoints for:

• Get the contents of a bucket by its name.
• List all the buckets currently in memory, by their name.
• Get the bucket object, which will include the bucket's data and the bucket's metadata which is just the bucket name.
• Delete a bucket.
• Set the bucket's data to the key-value pairs in the POST body.
• Update a bucket's data with the specified key-value.

Implementation

The implementation (C# ASP.NET Core 3.1) is 123 lines of controller code. Very simple. Here it is in its entirety.

using System.Collections.Generic;
using System.Linq;

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.ModelBinding;

using MemoryBucket.Classes;

namespace MemoryBucket.Controllers
{
  [ApiController]
  [Route("[controller]")]
  public class MemoryBucketController : ControllerBase
  {
    private static Buckets buckets = new Buckets();
    private static object locker = new object();

    public MemoryBucketController()
    {
    }

    /// <summary>
    /// Returns the contents of an existing bucket.
    /// </summary>
    [HttpGet]
    public object Get([FromQuery, BindRequired] string bucketName)
    {
      lock (locker)
      {
        Assertion.That(buckets.TryGetValue(bucketName, out Bucket bucket), 
            $"No bucket with name {bucketName} exists.");

        return bucket.Data;
      }
    }

    /// <summary>
    /// Lists all buckets.
    /// </summary>
    [HttpGet("List")]
    public object GetBucket()
    {
      lock (locker)
      {
        return buckets.Select(b => b.Key);
      }
    }

    /// <summary>
    /// Returns the bucket itself.
    /// </summary>
    [HttpGet("Bucket")]
    public object GetBucket([FromQuery, BindRequired] string bucketName)
    {
      lock (locker)
      {
        Assertion.That(buckets.TryGetValue(bucketName, out Bucket bucket), 
           $"No bucket with name {bucketName} exists.");

        return bucket;
      }
    }

    /// <summary>
    /// Deletes an existing bucket.
    /// </summary>
    [HttpDelete("{bucketName}")]
    public void Delete(string bucketName)
    {
      lock (locker)
      {
        Assertion.That(buckets.TryGetValue(bucketName, out Bucket bucket), 
           $"No bucket with name {bucketName} exists.");

        buckets.Remove(bucketName);
      }
    }

    /// <summary>
    /// Creates or gets an existing bucket and replaces its data.
    /// </summary>
    [HttpPost("Set")]
    public object Post(
      [FromQuery, BindRequired] string bucketName, 
      [FromBody] Dictionary<string, object> data)
    {
      lock (locker)
      {
        var bucket = CreateOrGetBucket(bucketName);
        bucket.Data = data;

        return data;
      }
    }

    /// <summary>
    /// Creates or gets an existing bucket and updates the specified key with the 
    /// specified value.
    /// </summary>
    [HttpPost("Update")]
    public object Post(
    [FromQuery, BindRequired] string bucketName,
    [FromQuery, BindRequired] string key,
    [FromQuery, BindRequired] string value)
    {
      lock (locker)
      {
        var bucket = CreateOrGetBucket(bucketName);

        var data = bucket.Data;
        data[key] = value;

        return data;
      }
    }

    private Bucket CreateOrGetBucket(string bucketName)
    {
      if (!buckets.TryGetValue(bucketName, out Bucket bucket))
      {
        bucket = new Bucket();
        bucket.Name = bucketName;
        buckets[bucketName] = bucket;
      }

      return bucket;
    }
  }
}

The Helper Classes

Try not to be awed. I really don't think this needs an explanation.

The Bucket Class

using System.Collections.Generic;

namespace MemoryBucket.Classes
{
  public class Bucket
  {
    public string Name { get; set; }
    public Dictionary<string, object> Data { get; set; } = new Dictionary<string, object>();
  }
}

The Buckets Class

using System.Collections.Generic;

namespace MemoryBucket.Classes
{
  public class Buckets : Dictionary<string, Bucket>
  {
  }
}

Testing

Some Postman tests suffice here.

Create a Bucket

curl --location --request POST 'http://localhost:7672/memoryBucket/Set?bucketName=Soup' \
--header 'Content-Type: application/json' \
--data-raw '{
"Name":"Marc",
"Resource": "Garlic",
"Note": "For the soup"
}'

and we see the response as:

{
  "Name": "Marc",
  "Resource": "Garlic",
  "Note": "For the soup"
}

Update a Bucket

Kate is taking over making the soup:

curl --location 
--request POST 'http://localhost:7672/memoryBucket/Update?bucketName=Soup&key=Name&value=Kate'

and we see the response as:

{
  "Name": "Kate",
  "Resource": "Garlic",
  "Note": "For the soup"
}

Listing Buckets

I'm adding another bucket:

curl --location --request POST 'http://localhost:7672/memoryBucket/Set?bucketName=Salad' \
--header 'Content-Type: application/json' \
--data-raw '{
"Name":"Laurie",
"Resource": "Lettuce"
}'

And when I list the buckets:

curl --location --request GET 'http://localhost:7672/memoryBucket/List'

I see:

[
  "Salad",
  "Soup"
]

Get the Bucket Itself

curl --location --request GET 'http://localhost:7672/memoryBucket/Bucket?bucketName=Soup'

And I see:

{
  "name": "Soup",
  "data": {
    "Name": "Marc",
    "Resource": "Garlic",
    "Note": "For the soup"
  }
}

Deleting Buckets

Dinner's ready:

curl --location --request DELETE '<a href="http://localhost:7672/memoryBucket/Soup">
http://localhost:7672/memoryBucket/Soup</a>'
curl --location --request DELETE '<a href="http://localhost:7672/memoryBucket/Salad">
http://localhost:7672/memoryBucket/Salad</a>'
curl --location --request GET 'http://localhost:7672/memoryBucket/List'

No more buckets:

[]

Conclusion

Simple, right? Makes me wonder why we don't do simple anymore.

History

• 26^th June, 2021: Initial version