Concepts of event sourcing

Posted on Dec 18, 2022

This post belongs to a series about my DDD journey.

Event sourcing is a pattern which interacts very well with DDD and creates something like a log book of your application state. This helps you to recreate your app state every time when needed. This post will introduce you to this topic and show the challenges in this pattern.

The idea behind event sourcing is, that every change of the state in an application is captured in an event object. These events are stored in an append-only database and read by the application when a recreation is needed.

So the stored data is a sequence of events bounded to an id. When the data needs to be recreated, the events get read from the storage and the events replayed. As shown in the following figure:

graph LR; es[(Event\nStorage)] es -->|Rehydrate by events| Aggregate Aggregate --> |Commit events| es

The events itself are created through commands which are applied by a command handler in the domain model:

The events

As an example for an event check the following event for a Client model, which got created:

{
  "client-id": 1,
  "event-id": 1,
  "event-type": "client-created",
  "name": "Alexander Humboldt",
  "phone": "555-2940-122",
  "timestamp": "2022-12-16T16:43:12.89Z"
}

As you see, it includes some information about the client like the id to which this event gets applied and the name and phone number. Furthermore it includes the event id, event type and timestamp. These are event-related information which will help to create the model out of it.

This events get stored in a event storage and appended as soon as another event happens. The storage is the source of truth and an append-only database. If you want to update or delete something, you also need to add an event to it.

Like here as an example where the name gets updated:

{
  "client-id": 1,
  "event-id": 2,
  "event-type": "client-updated",
  "name": "Wilhelm Humboldt",
  "phone": "555-2940-122",
  "timestamp": "2022-12-17T20:04:01.32Z"
}

So overriding the current data is not possible. This leads to a log book of your data.

Recreation of the model

The model itself needs to take the events and apply them to itself. It is event-aware.

I’ll show the implementation and its context in another post as the implementation of event sourcing is not part of this post (check my collection of posts about the DDD journey).

Let’s look at the (dis-)advantages and concepts of event-sourcing.

Advantages

Event-sourcing has mainly these advantages:

The domain model is more expressive, as you exactly can read out of the domain model how the business logic works.
You can track the state of an application at every time (time traveling).
You can rebuild the application state at every point of time.
You get deep insights into the behavior of the system and it’s state, especially when something fails.
You have more information about your data and what happens with it (important for audits).

Disadvantages

And yes, like every decision in software engineering, this pattern also has disadvantages:

The implementation can be complicated.
Restoring a lot of events can slow the application down.
There are some challenges like evolving the model and the complexity of the architecture.

Examples of use cases for event sourcing

Some examples are:

Version control systems
CRMs where you can track customer journeys
Banking applications to track the accounting

Reversing events

Mistakes happen. So for sure you’ll face the requirement to reverse events.

The problem in an event-sourcing application is the append-only database, which doesn’t allow you to delete events. But this is a good thing for us as we will not face the problem that a deletion leads to an inconsistent state.

So every event which modifies your state, needs to have a compensating event, if you want to revert it.

Applying differences

The easiest way to revert an event is when you can apply a difference. For example:

The event ‘Add 10€ to my account’ is easier to reverse than ‘set my account balance to 210€’. Because the compensating event to ‘Add 10€ to my account’ is ‘Remove 10€ from my account’.

Store needed data

But what if you cannot apply differences or your inputs don’t store the differences? In this case the stored events should add all needed information for the reverting of itself.

This can be done in two ways:

Store the previous value of every changed value.
Calculate and store the differences by yourself.

Example: Compensating an update

The client-updated event from above could contain the previous values in a oldValuesBeforeChange field:

{
  "client-id": 1,
  "event-id": 2,
  "event-type": "client-updated",
  "name": "Wilhelm Humboldt",
  "phone": "555-2940-122",
  "oldValuesBeforeChange": {"name": "Alexander Humboldt"},
  "timestamp": "2022-12-17T20:04:01.32Z"
}

The compensating event would take oldValuesBeforeChange map and reset the fields accordingly.

Delete data

When you want to delete data, you’ll face the same problem as at the reverting of data: The storage is append-only!

Well, but sometimes the deleteion of data is needed and I’m not talking about soft-deletes.

For that you can use the forgettable payload pattern which works as follow:

The sensitive information is stored in an encrypted form.

To encrypt it you need a key which is stored in a (external) key-value store. The key is the aggregate id and the value is the encryption key.

When you want to delete an aggregate you need to delete the encryption key. Now you cannot read the values anymore!

Things to consider

There are many things you need to keep an eye on when using event sourcing. Some of them are:

When your service sends notifications into the outside as soon as events happen, you need to make sure that these notifications aren’t send to the outside while replaying.
Parallel requests, versioning of the events and transactions are the hugest challenges.
Failure of events and handling them.
In brownfield projects you’ll need to create initial events out of the existing data.
Changes of the temporal logic. For example: Rules that have been applied for a specific time and changed afterwards.

Event sourcing is a very huge field! Check the links in the next section if you want to know more about it.