Event sourcing means the source of truth for an entity is an append-only sequence of facts that happened to it (AccountOpened, MoneyDeposited, MoneyWithdrawn), and its current state is derived by folding those facts, not stored as a mutable row. Interviewers probe it because it is a genuine change in what "the database" means, it is often proposed for the wrong reasons, and every follow-up (how do you query it, how do you change an old event, how long does replay take) has a real cost behind it. For a Backend or AI engineer it matters in two ways: as a way to get audit, history, and rebuildable read models, and as the model behind the logs (Kafka topics, WAL) that most data pipelines are actually built on.
STATE-ORIENTED (normal CRUD) EVENT-SOURCED
accounts events (append-only, never UPDATE / DELETE)
+----+---------+ +-----+---------+-------------+--------+
| id | balance | | ver | stream | type | amount |
+----+---------+ +-----+---------+-------------+--------+
| 1 | 12000 | <-- only the | 1 | acct-1 | Opened | |
+----+---------+ result | 2 | acct-1 | Deposited | 10000 |
UPDATE accounts SET balance = ... | 3 | acct-1 | Withdrawn | 3000 |
(previous value is gone unless | 4 | acct-1 | Deposited | 5000 |
you bolt on an audit table) +-----+---------+-------------+--------+
state = fold(apply, initial, events)
= 0 + 10000 - 3000 + 5000 = 12000In a state-oriented table you overwrite: the update destroys the previous value and, more importantly, the reason for the change. An event-sourced store records what happened and never rewrites it, so history is not a feature you add, it is the storage format. The current state is a left fold over the events with a pure function apply(state, event) -> state. Two consequences follow. Events are named in the past tense as business facts ("ItemAddedToCart", not "cart row updated"), which is why they can be understood and consumed by other parts of the system. And any state that can be computed from the events (a balance, a projection, an index) is disposable, because you can recompute it. The price is that "give me all accounts with balance over 10,000" is no longer a query on the store, which is why event sourcing is nearly always paired with read models (see CQRS).
import sqlite3, json
db = sqlite3.connect(":memory:")
db.executescript("""
CREATE TABLE events (
global_seq INTEGER PRIMARY KEY AUTOINCREMENT,
stream_id TEXT NOT NULL,
version INTEGER NOT NULL, -- position within the stream
type TEXT NOT NULL,
schema_v INTEGER NOT NULL DEFAULT 1,
data TEXT NOT NULL,
UNIQUE (stream_id, version) -- optimistic concurrency lives here
);
CREATE TABLE snapshots (stream_id TEXT PRIMARY KEY, version INTEGER NOT NULL, state TEXT NOT NULL);
""")
class Conflict(Exception): pass
def append(stream, expected_version, type_, data, schema_v=1):
try:
with db:
db.execute("INSERT INTO events(stream_id,version,type,schema_v,data) VALUES (?,?,?,?,?)",
(stream, expected_version + 1, type_, schema_v, json.dumps(data)))
except sqlite3.IntegrityError:
raise Conflict(f"{stream}: version {expected_version + 1} already taken")
def upcast(type_, schema_v, data):
# old v1 Deposited events had {"amount": 100} in whole rupees; v2 uses cents + currency
if type_ in ("Deposited", "Withdrawn") and schema_v == 1:
return {"amount_cents": data["amount"] * 100, "currency": "INR"}
return data
def apply(state, type_, data): # pure function: (state, event) -> state
s = dict(state)
if type_ == "Opened": s["balance"] = 0
elif type_ == "Deposited": s["balance"] += data["amount_cents"]
elif type_ == "Withdrawn": s["balance"] -= data["amount_cents"]
return s
def load(stream, up_to_version=None, use_snapshot=True):
state, start = {"balance": None}, 0
if use_snapshot and up_to_version is None:
row = db.execute("SELECT version, state FROM snapshots WHERE stream_id=?", (stream,)).fetchone()
if row: start, state = row[0], json.loads(row[1])
q = "SELECT version,type,schema_v,data FROM events WHERE stream_id=? AND version>?"
args = [stream, start]
if up_to_version is not None: q += " AND version<=?"; args.append(up_to_version)
n = 0
for v, t, sv, d in db.execute(q + " ORDER BY version", args):
state = apply(state, t, upcast(t, sv, json.loads(d))); start = v; n += 1
return state, start, n
append("acct-1", 0, "Opened", {})
append("acct-1", 1, "Deposited", {"amount": 100}, schema_v=1) # old shape (rupees)
append("acct-1", 2, "Withdrawn", {"amount": 30}, schema_v=1) # old shape
append("acct-1", 3, "Deposited", {"amount_cents": 5000, "currency": "INR"}, schema_v=2)
append("acct-1", 4, "Withdrawn", {"amount_cents": 1200, "currency": "INR"}, schema_v=2)
print("full replay: ", load("acct-1", use_snapshot=False))
print("as of version 3: ", load("acct-1", up_to_version=3)) # time travel
state, ver, _ = load("acct-1", use_snapshot=False)
db.execute("INSERT INTO snapshots VALUES ('acct-1', ?, ?)", (ver, json.dumps(state))); db.commit()
append("acct-1", 5, "Deposited", {"amount_cents": 700, "currency": "INR"}, schema_v=2)
s, v, n = load("acct-1")
print(f"snapshot + tail: state={s} version={v} events_replayed={n}")
# two writers both loaded version 6 and both try to append version 7
for who, amt in (("writer A", 100), ("writer B", 999)):
try:
append("acct-1", 6, "Withdrawn", {"amount_cents": amt, "currency": "INR"}, schema_v=2)
print(f"{who}: appended version 7")
except Conflict as e:
print(f"{who}: rejected -> {e}")
# read model projection, built by folding the global log
totals = {}
for _, stream, t, sv, d in db.execute("SELECT global_seq,stream_id,type,schema_v,data FROM events ORDER BY global_seq"):
dd = upcast(t, sv, json.loads(d))
if t == "Deposited": totals[stream] = totals.get(stream, 0) + dd["amount_cents"]
print("projection: total deposited (cents) per stream:", totals)
# Real output (python3, run locally):
# full replay: ({'balance': 10800}, 5, 5)
# as of version 3: ({'balance': 7000}, 3, 3)
# snapshot + tail: state={'balance': 11500} version=6 events_replayed=1
# writer A: appended version 7
# writer B: rejected -> acct-1: version 7 already taken
# projection: total deposited (cents) per stream: {'acct-1': 15700}Read the output line by line. A full replay of five events gives 10800 (100 rupees = 10000 cents, minus 3000, plus 5000, minus 1200). Replaying only up to version 3 gives 7000: that is time travel, and it costs nothing extra because history is the store. With a snapshot at version 5, loading needs to replay only the one event after it, not all six. Two writers who both read version 6 race to append version 7; the UNIQUE (stream_id, version) constraint lets exactly one win, and the loser must reload, re-run its business rule against the new state, and retry. That single constraint is optimistic concurrency, and it is how event stores protect an aggregate's invariants without holding a lock. Finally, the version-1 events (rupees, no currency) are still on disk in their original shape; the upcast function translates them to the version-2 shape at read time, so old data is never rewritten.
CREATE TABLE events (global_seq INTEGER PRIMARY KEY AUTOINCREMENT, stream_id TEXT NOT NULL, version INTEGER NOT NULL,
type TEXT NOT NULL, amount_cents INTEGER, UNIQUE (stream_id, version));
INSERT INTO events(stream_id,version,type,amount_cents) VALUES
('acct-1',1,'Opened',NULL),('acct-1',2,'Deposited',10000),('acct-1',3,'Withdrawn',3000),
('acct-2',1,'Opened',NULL),('acct-2',2,'Deposited',500),('acct-1',4,'Deposited',5000);
-- current balance per stream: the fold is a SUM over the events
SELECT stream_id,
SUM(CASE type WHEN 'Deposited' THEN amount_cents WHEN 'Withdrawn' THEN -amount_cents ELSE 0 END) AS balance_cents,
MAX(version) AS version
FROM events GROUP BY stream_id ORDER BY stream_id;
-- state after every event (a running fold) for acct-1
SELECT version, type, amount_cents,
SUM(CASE type WHEN 'Deposited' THEN amount_cents WHEN 'Withdrawn' THEN -amount_cents ELSE 0 END)
OVER (ORDER BY version) AS balance_after
FROM events WHERE stream_id='acct-1' ORDER BY version;
-- a second writer reusing version 4 is rejected by the UNIQUE constraint
INSERT INTO events(stream_id,version,type,amount_cents) VALUES ('acct-1',4,'Withdrawn',100);
-- Real output (sqlite3, run locally):
-- stream_id balance_cents version
-- --------- ------------- -------
-- acct-1 12000 4
-- acct-2 500 2
--
-- version type amount_cents balance_after
-- ------- --------- ------------ -------------
-- 1 Opened 0
-- 2 Deposited 10000 10000
-- 3 Withdrawn 3000 7000
-- 4 Deposited 5000 12000
--
-- Runtime error: UNIQUE constraint failed: events.stream_id, events.version (19)This is worth showing because it demystifies "event store": in its simplest form it is one append-only table with a unique (stream_id, version). The window-function query is the fold expressed in SQL, giving the state after every event. Dedicated event stores and log systems add things this toy lacks (subscriptions, global ordering guarantees under concurrency, retention and compaction policy), but the core contract is the same: append with an expected version, read a stream in order.
Load an aggregate with 1,000,000 events:
no snapshot : read + apply 1,000,000 events (slow, grows forever)
snapshot @ ver 999,900 : read snapshot + apply 100 events (bounded)
Snapshot rules of thumb:
* a snapshot is a CACHE of the fold, never the source of truth: it can be deleted and rebuilt
* store (stream_id, version, serialized state); take one every N events or on a timer
* if the apply() logic or state shape changes, old snapshots may be invalid: rebuild them
Replaying to rebuild a PROJECTION:
for event in log (from the beginning): projection.apply(event)
DANGER: a handler that also sends email / charges a card / calls an API
will do it AGAIN on replay. Keep projections pure; keep side effects
in separate "reactors" that track their own position and are NOT replayed.Replay is the superpower and the trap. It lets you fix a bug in a projection and rebuild it, or build a brand-new read model for a question nobody asked when the events were written. But replay time grows with history, which is why long-lived aggregates get snapshots, and why very long streams are usually split into shorter ones by design (for example one stream per account per month, or per shopping-cart session). The subtler trap is side effects: a handler that sends a confirmation email is fine on first delivery and a disaster when you replay two years of events to rebuild a read model. Keep state-building code pure and idempotent, and put effects behind handlers that are tracked separately and not replayed.
Events are immutable facts stored forever. Your code, however, keeps changing.
Strategies (usually combined):
1. Additive / weak schema add optional fields only; readers tolerate unknown and missing fields
2. Versioned types Deposited (schema_v=1) and Deposited (schema_v=2) coexist on disk
3. UPCASTING at read time a pure function translates old shape -> latest shape before apply()
(the demo above: rupees -> cents + currency)
4. New event type stop emitting MoneyDeposited; emit FundsCredited; keep handling both
5. Copy-and-transform read the old stream, write a corrected new stream, cut over
(a migration you run deliberately; the only way to truly "change" history)
Never: UPDATE events SET data = ... in place (breaks audit meaning, other consumers, and any hashes/signatures)In a CRUD system a schema change is one migration on one table. In an event-sourced system the old shape is permanent, so the reader must understand every shape ever written. Upcasting keeps that complexity in one place, a chain of small pure functions from vN to vN+1, so the domain logic only ever sees the latest shape. The design lesson is to model events as narrow business facts and avoid packing them with fields that will later prove wrong. Corrections are also events: you do not edit a wrong deposit, you append a compensating DepositReversed, which preserves the record of what the system believed and when. For personal data under deletion laws (for example GDPR), immutable logs collide with the right to erasure; common mitigations are keeping personal data outside the event (referenced by id and deletable) or encrypting it per subject and discarding the key (crypto-shredding), and you should say plainly that this is a real design constraint, not an afterthought.
Event sourcing CDC (log-based) Audit table / triggers
source of truth the event log the normal tables the normal tables
what is recorded business facts, intent physical row changes row before/after values
("OrderCancelled: reason") ("row 7: status A -> B") (often + who/when)
who designs it the domain model the database's log format the DBA / trigger author
rebuild state? yes, replay is THE mechanism only via snapshot + log, not no: audit is a side record
guaranteed to retain full history
state queries need projections plain SQL on tables plain SQL on tables
main cost model + tooling + evolution pipeline ops, slot retention write overhead, storage
They combine: CDC can ship events out of an event-store table, and an outbox is "event
sourcing lite" (events are emitted alongside state, but state remains the source of truth).The confusion between these is the single most common interview trap. With event sourcing the events are the database; if you delete the events you have lost the entity. With CDC the tables are the database and the change stream is a by-product that the database already produces for its own recovery, so it tells you what changed but not why, and it is not a durable history unless you retain it. An audit table is closer to CDC than to event sourcing. Choose event sourcing when the business itself thinks in terms of a history of decisions (ledgers, order lifecycles, compliance-heavy domains) and you would otherwise rebuild that history by hand; choose state storage with CDC or an outbox when you just need other systems to hear about changes.
fold(apply, initial, events) with apply a pure function.(stream_id, version); the loser reloads and retries against fresh state.# Save the Python from "Append, Optimistic Concurrency, Replay, Snapshot and Upcasting" as es.py:
# python3 es.py
# Expected: the six lines shown in "Real output" (10800 / 7000 / 11500 / writer A ok / writer B rejected / 15700).
#
# Save the SQL from "The Same Idea in SQL" and run: sqlite3 :memory: < es.sql
# Expected: balances 12000 and 500, the running balance history, then a UNIQUE constraint error.
#
# Exercises:
# a. Add a "Transferred" event type and update apply(); confirm old events still replay untouched.
# b. Add a schema_v=3 that splits "amount_cents" into "amount" + "exponent", and write a v2 -> v3
# upcaster. Confirm the v1 events flow through BOTH upcasters (v1 -> v2 -> v3) and give the same balance.
# c. Delete the snapshots table row and confirm load() still returns the same state (snapshot = cache).
# d. Add an invariant "balance may not go negative" in the command handler (not in apply()) and show that a
# rejected command appends nothing. Why must the check live in the handler, not in apply()?
# (apply() replays facts that already happened; it must never refuse them.)Q: What is event sourcing, and how does it differ from storing current state?
A: Instead of overwriting a row, you append immutable events that record each state change, and derive current state by folding them in order. You get a built-in audit trail, the ability to see state at any past point, and the freedom to build new read models later. The trade-off is that state queries are no longer simple (you need projections), the event schema must be maintained forever, and the model is harder to operate than a table.
Q: How do snapshots work, and are they part of the source of truth?
A: A snapshot stores the aggregate's folded state at a specific version so that loading only replays events after that version, which bounds load time on long streams. It is purely a cache: it can be dropped and rebuilt from the events, and it must be invalidated or rebuilt if the fold logic or state shape changes. Take them every N events or on a schedule, and prefer shorter streams by design before reaching for snapshots as the only fix.
Q: What is replay used for, and what can go wrong with it?
A: Replay rebuilds a read model from the log, whether to fix a projection bug, to add a new view, or to migrate storage. What goes wrong is time and side effects: replaying millions of events can take hours, so you build the new projection alongside the old and cut over, and any handler that sends emails or charges cards would repeat those effects, so state-building must be pure and side effects must live in separate handlers that track their own position and are not replayed.
Q: How do you handle schema evolution when events are immutable?
A: You never rewrite old events in place. You prefer additive changes with tolerant readers, version event types, and add upcasters that translate old shapes to the current one at read time so domain code sees a single shape. For genuine structural changes or corrections you either introduce a new event type or run a deliberate copy-and-transform migration into a new stream, and you fix wrong facts with compensating events so the history stays honest.
Q: Event sourcing versus CDC: when would you use each?
A: Event sourcing makes domain events the source of truth, so history and intent are first-class and state is derived; CDC leaves state in normal tables and streams out the physical row changes the database already logs. Use event sourcing when the domain is naturally a sequence of decisions and you need auditability and rebuildable read models; use CDC (or an outbox) when you want to keep an existing state-based system and feed other systems. CDC tells you a row changed but not why, and it is not a complete history unless you retain it.
Follow-up: How would you handle a user's GDPR erasure request in an immutable event log?
A: Avoid the problem structurally: keep personal data out of events and reference it by id from a mutable, deletable store, or encrypt each subject's personal fields with a per-subject key and delete the key on request (crypto-shredding), so the events remain but the personal data is unreadable. Rewriting the log is a last resort because it breaks consumers, offsets, and audit meaning. This has to be designed in from the start, since retrofitting it onto years of plaintext events is painful.
Follow-up: Two commands hit the same aggregate at the same time. How do you prevent an invalid state?
A: Each command loads the aggregate at some version and appends with that expected version; the store rejects the second append because the version is already taken. The rejected handler reloads the new state, re-checks the business rule (the balance may now be insufficient), and either retries or fails. This is optimistic concurrency and works well when conflicts on a single aggregate are rare; if one aggregate is a hot spot, retries pile up and you reconsider the aggregate boundary.
Follow-up: Can you guarantee a global order of events across all streams?
A: Within one stream, yes, by version. Across streams, a single-node store can assign a global sequence, but under concurrent writers that number can be committed out of order (a lower number becoming visible after a higher one), so subscribers that checkpoint on it can skip events unless the store handles this. Distributed logs generally order per partition only. The design answer is to require ordering only within an aggregate, and to make projections tolerate cross-stream reordering.
Event sourcing fits domains where the history is a first-class business asset: financial ledgers, order and payment lifecycles, inventory movements, and workflow or approval systems that need to answer "how did we get here?". In a design interview, present it as an append-only event store per aggregate with optimistic concurrency, a subscription that feeds projections (search index, dashboards, a vector store for retrieval), and snapshots for long-lived aggregates. Then show you know the operational side: how projections are rebuilt and cut over, how event versions evolve, how you keep side effects out of replays, and how personal data is handled. It also gives you a vocabulary for the systems you already use: a Kafka topic with long retention is an event log, and a database WAL is an event log the database keeps for itself. Most real systems use event sourcing selectively for the one or two aggregates where it pays off and plain tables for everything else.
You gain a complete history, the ability to reconstruct any past state, natural integration with event-driven systems, and read models you can rebuild or invent later. You pay in complexity: every read of current state goes through a fold or a projection, the read side is eventually consistent, event schemas are a permanent public contract that must be versioned and upcast, replay time and storage grow without bound, privacy erasure fights immutability, and ad-hoc reporting on the write side is hard. Teams also underestimate the tooling, debugging, and onboarding cost. It is the wrong choice for ordinary CRUD, for data whose history nobody cares about, for teams without the appetite to operate projections and evolve event schemas, and whenever an audit table, CDC, or a simple outbox would satisfy the actual requirement. If you only need history for a few entities, keep state as the source of truth and add an append-only audit or outbox instead of committing the whole system to event sourcing.