Click here to Skip to main content
15,031,842 members
Articles / Hosted Services / Azure
Project
Posted 1 Aug 2021

Tagged as

Stats

38.9K views
29 downloads
14 bookmarked

Event Sourcing on Azure Functions

Rate me:
Please Sign up or sign in to vote.
5.00/5 (11 votes)
1 Aug 2021CPOL3 min read
A library to demonstrate doing Event Sourcing as a data persistence mechanism for Azure Functions
In this post, you will see a library that demonstrates doing Event Sourcing as a data persistence mechanism for Azure functions.

A library to demonstrate doing Event Sourcing as a data persistence mechanism for Azure Functions.

.NET

Introduction to event sourcing

At its very simplest, event sourcing is a way of storing state (for an entity) which works by storing the sequential history of all the events that have occurred to that entity. Changes to the entity are written as new events appended to the end of the event stream for the entity.

When a query or business process needs to use the current state of the entity it gets this by running a projection over the event stream which is a very simple piece of code which, for each event, decides (a) do I care about this type of event and (b) if so what do I do when I receive it.

There is a 50 minute talk that covers this on YouTube, or if you already have an understanding of event sourcing you can go straight to the Getting started wiki page

Example

There is a rudimentary "retail bank accounts" example (as a Blazor front end) here ( source repository on github ) that demonstrates the different types of operation on an event stream and the source code for that is included in this repository. This is an entirely serverless system with no underlying database which does "scale to zero".

How to use

This library allows you to interact with the event streams for entities without any extra plumbing in the azure function itself - with both access to event streams and to run projections being via bound variables that are instantiated when the azure function is executed.

To add events to an event stream you use an Event stream attribute and class thus:-

C#
[FunctionName(<span class="pl-pds">"OpenAccount"</span>)]
public static async Task<HttpResponseMessage> OpenAccountRun(
              [HttpTrigger(AuthorizationLevel.Function, <span class="pl-pds">"POST"</span>, Route = <span class="pl-pds">"OpenAccount/{accountnumber}"</span>)]HttpRequestMessage req,
              string accountnumber,
              [EventStream(<span class="pl-pds">"Bank"</span>, <span class="pl-pds">"Account"</span>, <span class="pl-pds">"{accountnumber}"</span>)]  EventStream bankAccountEvents)
{
    if (await bankAccountEvents.Exists())
    {
        return req.CreateResponse(System.Net.HttpStatusCode.Forbidden , <span class="pl-pds">$"Account {accountnumber} already exists"</span>);
    }
    else
    {
        <span class="pl-c">// Get request body</span>
        AccountOpeningData data = await req.Content.ReadAsAsync<AccountOpeningData>();

        <span class="pl-c">// Append a "created" event</span>
        DateTime dateCreated = DateTime.UtcNow;
        Account.Events.Opened evtOpened = new Account.Events.Opened() { LoggedOpeningDate = dateCreated };
        if (! string.IsNullOrWhiteSpace( data.Commentary))
        {
            evtOpened.Commentary = data.Commentary;
        }
        await bankAccountEvents.AppendEvent(evtOpened);
                
        return req.CreateResponse(System.Net.HttpStatusCode.Created , <span class="pl-pds">$"Account {accountnumber} created"</span>);
    }
}

To get the values out of an event stream you use a Projection attribute and class thus:-

C#
[FunctionName(<span class="pl-pds">"GetBalance"</span>)]
public static async Task<HttpResponseMessage> GetBalanceRun(
  [HttpTrigger(AuthorizationLevel.Function, <span class="pl-pds">"GET"</span>, Route = <span class="pl-pds">"GetBalance/{accountnumber}"</span>)]HttpRequestMessage req,
  string accountnumber,
  [Projection(<span class="pl-pds">"Bank"</span>, <span class="pl-pds">"Account"</span>, <span class="pl-pds">"{accountnumber}"</span>, nameof(Balance))] Projection prjBankAccountBalance)
{
    string result = <span class="pl-pds">$"No balance found for account {accountnumber}"</span>;

    if (null != prjBankAccountBalance)
    {
        Balance projectedBalance = await prjBankAccountBalance.Process<Balance>();
        if (null != projectedBalance )
        {
            result = <span class="pl-pds">$"Balance for account {accountnumber} is ${projectedBalance.CurrentBalance} (As at  {projectedBalance.CurrentSequenceNumber}) "</span>;
        }
    }
    return req.CreateResponse(System.Net.HttpStatusCode.OK, result);
}

All of the properties of these two attributes are set to AutoResolve so they can be set at run time.

Chosen technologies

For production use and especially for higher volume streams there is an Azure Tables back end that I recommend be used.

Alternatively, because an event stream is an inherently append only system the storage technology underlying can be AppendBlob - a special type of Blob storage which only allows for blocks to be appended to the end of the blob. Each blob can store up to 50,000 events and the container path can be nested in the same way as any other Azure Blob storage.

The choice of storage technology and storage target is switchable by a configuration setting on the application, on a per domain/entity basis.

The azure functions code is based on version 2.0 of the azure functions SDK and is written in C#.

Comparison to other event sourcing implementations

In this library the state of an entity has to be retrieved on demand - this is to allow for the functions application to be spun down to nothing and indeed for multiple independent azure functions applications to use the same underlying event stream without having to have any "always on" consistency service.

Requirements

In order to use this library you will need an Azure account with the ability to create a storage container and to host an azure functions application.

If you want to use the notifications functionality you will also need to set up an event grid topic that the library will push notifications to.

Roadmap

The current version allows the storing and projection of event stream backed entities in either Azure Tables storage (recommended) or AppendBlob. This includes the concurrency protection that is needed to do this safely in a many-writers, many-readers scenario.

It also has outgoing notificiations - effectively a change feed - that are raised whenever a new entity is created or when a new event is appended to the event stream of an existing entity. This is done via event grid to allow massive scale assembly of these event sourcing backed entities into an ecosystem.

The next phase is to have notification derived triggers which can listen to these outgoing notifications and trigger serverless functions when they occur, and to provide the scaffold for creating CQRS systems on azure serverless.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Duncan Edwards Jones
Software Developer
Ireland Ireland
C# / SQL Server developer
Microsoft MVP (Azure) 2017
Microsoft MVP (Visual Basic) 2006, 2007

Comments and Discussions

 
BugBoth code links are dead Pin
joer009-Mar-21 8:46
Memberjoer009-Mar-21 8:46 
GeneralRe: Both code links are dead Pin
Duncan Edwards Jones12-May-21 1:11
professionalDuncan Edwards Jones12-May-21 1:11 
QuestionMessage Closed Pin
27-Feb-21 22:34
MemberMember 1508594027-Feb-21 22:34 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.