Click here to Skip to main content
15,125,299 members
Articles / Programming Languages / C#
Article
Posted 26 Jun 2021

Tagged as

Stats

7.6K views
95 downloads
11 bookmarked

A Simple Key-Value Store Microservice

Rate me:
Please Sign up or sign in to vote.
4.14/5 (7 votes)
26 Jun 2021CPOL3 min read
I have a very specific use-case where I need a microservice that manages a simple in memory data store, which I call a "bucket."
My specific use case is that I need to track who is using a resource and for how long. After listing the requirements about what I need and what I do not need, we will take a look at the API, implementation, helper classes and testing.

Introduction

I have a very specific use-case where I need a microservice that manages a simple in memory data store, which I call a "bucket." What surprises me is that my brief Googling for an existing simple solution came up empty, but that's par for the course as simple things grow into complex things and the simple things end up being forgotten.

My specific use case is that I need to track who is using a resource and for how long. The user has the ability to indicate that they are using a resource and when they are done using that resource, and the user has the ability to add a note regarding the resource usage. Other users can therefore see who is using a resource, how long they've been using, and any notes about their usage. Given this, my requirements are more about what I don't need than what I do need.

What I Need

  1. The ability to set a key to a value.

What I Don't Need

  1. I don't even need the ability to manage different buckets but it seems sort of silly not to implement that feature, so it's implemented.
  2. I don't care if the memory is lost if the server restarts. This is for informational data only that will be shared across different clients. Technically, one of the front-end clients (because the client would have the full data) could even restore the data for everyone else.
  3. I don't need complex data structures, just key-value pairs where the value is some value type, not a structure.
  4. I don't even care about one user stepping on top of another - in actual usage, this doesn't happen, and if it did, I don't care -- whoever updates a key last wins.

Redis

"Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster. "

Yikes. Definitely major overkill for what I need. That said, please don't use this to replace something like Redis unless your requirements really fit the bill here.

The API

Browsing to http://localhost:7672/swagger/index.html, we see:

As you can see (sort of) from the Swagger documentation, we have endpoints for:

  • Get the contents of a bucket by its name.
  • List all the buckets currently in memory, by their name.
  • Get the bucket object, which will include the bucket's data and the bucket's metadata which is just the bucket name.
  • Delete a bucket.
  • Set the bucket's data to the key-value pairs in the POST body.
  • Update a bucket's data with the specified key-value.

Implementation

The implementation (C# ASP.NET Core 3.1) is 123 lines of controller code. Very simple. Here it is in its entirety.

C#
using System.Collections.Generic;
using System.Linq;

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.ModelBinding;

using MemoryBucket.Classes;

namespace MemoryBucket.Controllers
{
  [ApiController]
  [Route("[controller]")]
  public class MemoryBucketController : ControllerBase
  {
    private static Buckets buckets = new Buckets();
    private static object locker = new object();

    public MemoryBucketController()
    {
    }

    /// <summary>
    /// Returns the contents of an existing bucket.
    /// </summary>
    [HttpGet]
    public object Get([FromQuery, BindRequired] string bucketName)
    {
      lock (locker)
      {
        Assertion.That(buckets.TryGetValue(bucketName, out Bucket bucket), 
            $"No bucket with name {bucketName} exists.");

        return bucket.Data;
      }
    }

    /// <summary>
    /// Lists all buckets.
    /// </summary>
    [HttpGet("List")]
    public object GetBucket()
    {
      lock (locker)
      {
        return buckets.Select(b => b.Key);
      }
    }

    /// <summary>
    /// Returns the bucket itself.
    /// </summary>
    [HttpGet("Bucket")]
    public object GetBucket([FromQuery, BindRequired] string bucketName)
    {
      lock (locker)
      {
        Assertion.That(buckets.TryGetValue(bucketName, out Bucket bucket), 
           $"No bucket with name {bucketName} exists.");

        return bucket;
      }
    }

    /// <summary>
    /// Deletes an existing bucket.
    /// </summary>
    [HttpDelete("{bucketName}")]
    public void Delete(string bucketName)
    {
      lock (locker)
      {
        Assertion.That(buckets.TryGetValue(bucketName, out Bucket bucket), 
           $"No bucket with name {bucketName} exists.");

        buckets.Remove(bucketName);
      }
    }

    /// <summary>
    /// Creates or gets an existing bucket and replaces its data.
    /// </summary>
    [HttpPost("Set")]
    public object Post(
      [FromQuery, BindRequired] string bucketName, 
      [FromBody] Dictionary<string, object> data)
    {
      lock (locker)
      {
        var bucket = CreateOrGetBucket(bucketName);
        bucket.Data = data;

        return data;
      }
    }

    /// <summary>
    /// Creates or gets an existing bucket and updates the specified key with the 
    /// specified value.
    /// </summary>
    [HttpPost("Update")]
    public object Post(
    [FromQuery, BindRequired] string bucketName,
    [FromQuery, BindRequired] string key,
    [FromQuery, BindRequired] string value)
    {
      lock (locker)
      {
        var bucket = CreateOrGetBucket(bucketName);

        var data = bucket.Data;
        data[key] = value;

        return data;
      }
    }

    private Bucket CreateOrGetBucket(string bucketName)
    {
      if (!buckets.TryGetValue(bucketName, out Bucket bucket))
      {
        bucket = new Bucket();
        bucket.Name = bucketName;
        buckets[bucketName] = bucket;
      }

      return bucket;
    }
  }
}

The Helper Classes

Try not to be awed. I really don't think this needs an explanation.

The Bucket Class

C#
using System.Collections.Generic;

namespace MemoryBucket.Classes
{
  public class Bucket
  {
    public string Name { get; set; }
    public Dictionary<string, object> Data { get; set; } = new Dictionary<string, object>();
  }
}

The Buckets Class

C#
using System.Collections.Generic;

namespace MemoryBucket.Classes
{
  public class Buckets : Dictionary<string, Bucket>
  {
  }
}

Testing

Some Postman tests suffice here.

Create a Bucket

curl --location --request POST 'http://localhost:7672/memoryBucket/Set?bucketName=Soup' \
--header 'Content-Type: application/json' \
--data-raw '{
"Name":"Marc",
"Resource": "Garlic",
"Note": "For the soup"
}'

and we see the response as:

JavaScript
{
  "Name": "Marc",
  "Resource": "Garlic",
  "Note": "For the soup"
}

Update a Bucket

Kate is taking over making the soup:

curl --location 
--request POST 'http://localhost:7672/memoryBucket/Update?bucketName=Soup&key=Name&value=Kate'

and we see the response as:

JavaScript
{
  "Name": "Kate",
  "Resource": "Garlic",
  "Note": "For the soup"
}

Listing Buckets

I'm adding another bucket:

curl --location --request POST 'http://localhost:7672/memoryBucket/Set?bucketName=Salad' \
--header 'Content-Type: application/json' \
--data-raw '{
"Name":"Laurie",
"Resource": "Lettuce"
}'

And when I list the buckets:

curl --location --request GET 'http://localhost:7672/memoryBucket/List'

I see:

JavaScript
[
  "Salad",
  "Soup"
]

Get the Bucket Itself

curl --location --request GET 'http://localhost:7672/memoryBucket/Bucket?bucketName=Soup'

And I see:

JavaScript
{
  "name": "Soup",
  "data": {
    "Name": "Marc",
    "Resource": "Garlic",
    "Note": "For the soup"
  }
}

Deleting Buckets

Dinner's ready:

curl --location --request DELETE '<a href="http://localhost:7672/memoryBucket/Soup">
http://localhost:7672/memoryBucket/Soup</a>'
curl --location --request DELETE '<a href="http://localhost:7672/memoryBucket/Salad">
http://localhost:7672/memoryBucket/Salad</a>'
curl --location --request GET 'http://localhost:7672/memoryBucket/List'

No more buckets:

JavaScript
[]

Conclusion

Simple, right? Makes me wonder why we don't do simple anymore.

History

  • 26th June, 2021: Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Marc Clifton
Architect Interacx
United States United States
Blog: https://marcclifton.wordpress.com/
Home Page: http://www.marcclifton.com
Research: http://www.higherorderprogramming.com/
GitHub: https://github.com/cliftonm

All my life I have been passionate about architecture / software design, as this is the cornerstone to a maintainable and extensible application. As such, I have enjoyed exploring some crazy ideas and discovering that they are not so crazy after all. I also love writing about my ideas and seeing the community response. As a consultant, I've enjoyed working in a wide range of industries such as aerospace, boatyard management, remote sensing, emergency services / data management, and casino operations. I've done a variety of pro-bono work non-profit organizations related to nature conservancy, drug recovery and women's health.

Comments and Discussions

 
SuggestionWhat about ConcurrentDictionary<TKey,TValue> Pin
Alex Sanséau2-Jul-21 3:29
MemberAlex Sanséau2-Jul-21 3:29 
GeneralRe: What about ConcurrentDictionary<TKey,TValue> Pin
Nelek6-Jul-21 1:21
protectorNelek6-Jul-21 1:21 
GeneralMy vote of 5 Pin
raddevus29-Jun-21 4:25
mvaraddevus29-Jun-21 4:25 
GeneralRe: My vote of 5 Pin
Marc Clifton30-Jun-21 3:02
mvaMarc Clifton30-Jun-21 3:02 
SuggestionI am doing something similar in my IoT projects Pin
wmjordan29-Jun-21 3:32
professionalwmjordan29-Jun-21 3:32 
GeneralRe: I am doing something similar in my IoT projects Pin
Marc Clifton30-Jun-21 3:01
mvaMarc Clifton30-Jun-21 3:01 
I considered ConcurrentDictionary but can't remember why I chose to use locks. If I remember, I'll update the article as to the reason for that decision.

Thank you for your other comments.

QuestionRedis Pin
Amaury Fages27-Jun-21 23:15
MemberAmaury Fages27-Jun-21 23:15 
AnswerRe: Redis Pin
Marc Clifton28-Jun-21 10:00
mvaMarc Clifton28-Jun-21 10:00 
SuggestionLarge Object Heap Fragmentation Pin
Stylianos Polychroniadis27-Jun-21 10:57
MemberStylianos Polychroniadis27-Jun-21 10:57 
GeneralRe: Large Object Heap Fragmentation Pin
Marc Clifton28-Jun-21 9:59
mvaMarc Clifton28-Jun-21 9:59 
QuestionHmmm Pin
Tony 227-Jun-21 7:58
MemberTony 227-Jun-21 7:58 
AnswerRe: Hmmm Pin
Tomaž Štih27-Jun-21 23:50
MemberTomaž Štih27-Jun-21 23:50 
GeneralRe: Hmmm Pin
Marc Clifton28-Jun-21 9:55
mvaMarc Clifton28-Jun-21 9:55 
AnswerRe: Hmmm Pin
Marc Clifton28-Jun-21 9:34
mvaMarc Clifton28-Jun-21 9:34 
GeneralRe: Hmmm Pin
Member 1453048028-Jun-21 12:43
MemberMember 1453048028-Jun-21 12:43 
GeneralRe: Hmmm Pin
Marc Clifton30-Jun-21 3:00
mvaMarc Clifton30-Jun-21 3:00 
GeneralRe: Hmmm Pin
Member 1453048030-Jun-21 12:39
MemberMember 1453048030-Jun-21 12:39 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.