Running an event store in anger

#Why this became its own package

The event sourcing layer inside Ultimate Backend attracted a steady stream of questions from people who wanted that part and none of the rest. That is a reliable signal, so I pulled it out. The package gives NestJS command handling a durable store underneath, with the subscription and projection plumbing that makes it usable.

#Two kinds of subscription

The distinction that causes the most confusion is between catch-up subscriptions and persistent subscriptions, and it matters operationally.

A catch-up subscription is a client-side cursor. Your process remembers where it got to, reads forward, and continues from that position after a restart. It is exactly once from the perspective of the stream and at least once from the perspective of your side effects, because your process can die between handling an event and recording the position.

A persistent subscription puts the cursor on the server and lets several consumers share the work, with acknowledgement and retry. That is what you want for work distribution and not what you want for building a projection, because ordering across consumers is no longer guaranteed.

Choosing the wrong one produces a projection that is subtly wrong under load and correct in development, which is the worst available failure mode.

#The problem nobody warns you about

Events are permanent. That is the point of them. It also means an event written in 2019 has to be readable by the code you deploy in 2024.

Three things happen in practice. A field gets renamed. A field that was optional becomes required. A single event turns out to have been two events all along and needs splitting.

None of those can be handled by changing the stored data, because the stored data is the record of what happened. They are handled by upcasting: reading the old shape and converting it to the new shape on the way in. That code accumulates, it is rarely deleted, and it is the actual long-term cost of event sourcing.

1// Version 1 had a single name field. Version 2 splits it in two.
2// The v1 handler cannot be deleted while any v1 event exists, which is forever.
3const upcast = (e: StoredEvent): DomainEvent =>
4  e.version === 1
5    ? { ...e.data, given: e.data.name.split(' ')[0], family: e.data.name.split(' ').slice(1).join(' ') }
6    : e.data
Copy

#Projection rebuilds

The advertised benefit is that you can throw away a read model and rebuild it from history. That works, and there are two conditions nobody states.

The projection must be a pure function of the events. If it calls an external service, or reads the current time, or uses a random value, replaying it produces a different answer. This is easy to violate by accident and hard to notice.

The rebuild must be tested. A rebuild path that has never been run is a rebuild path that does not work, and you will discover that on the day you need it.

#Would I reach for it again

For a domain with real auditing requirements or genuine temporal questions, yes. For general application state, a log of changes alongside a conventional table gets most of the benefit at a fraction of the cost [2] [3]. The full pattern is worth its price when the history is part of the product [1] [4], and it is expensive ceremony when it is not.