Coordination & Transactions

Work through examples, try the reasoning yourself, then put it into practice.

View Slides

Before you start

  • Trace maps keyed by participant, resource and operation identity.

What you will practice

  • Derive decision/retry and compensation plans from their exact deduplication rules.
  • Explain which durable fact survives a crash and why replay cannot apply an effect again.
In this lesson

Lesson

6 min read

Coordination & Transactions

The shared problem is partial success: inventory is reserved, payment may have succeeded, and shipment is unknown. A timeout does not undo an effect. This sequence separates three responsibilities: choose a transaction decision, plan business compensation, and reject stale ownership. Start with maps, ordered event logs, and the difference between a result and an acknowledgement.

Derive the decision and the delivery plan separately

TwoPhaseCommit consumes participant snapshots, not a network transcript. The last record for each ID is authoritative. Consider:

Input record Vote Acked Survives deduplication?
inventory commit true yes
payment timeout false no
shipment commit false yes
payment commit false yes

The three surviving votes are commit, so Decision=commit. Delivery still owes [payment,shipment], in lexicographic order. Changing shipment's vote to abort changes Decision to abort, but the same unacknowledged participants still need the decision. Acknowledgements control retransmission; they do not count as votes. Empty participant input aborts by the course rule. A malformed vote also aborts.

Implement deduplication before either decision or retry selection. Otherwise, a stale timeout record can poison a later authoritative commit snapshot, or an old Acked=false can generate a spurious retry after a newer acknowledgement.

A pure planner does not make recovery durable

Real 2PC needs durable prepared state and a durable final decision. Once a commit decision exists, a restart must replay that decision; it must not rerun a vote calculation on an incomplete recovered list and turn commit into abort. A prepared participant that cannot learn the decision may have to wait. The transaction-commit analysis by Gray and Lamport explains this blocking boundary.

The exercise has no transaction ID, persisted decision or disk/network API. It returns a plan for one supplied snapshot. To embed it in a coordinator: first establish the complete participant set and transaction identity; durably record the decision before sending it; recover that decision on restart; retry notification idempotently until acknowledged. These are integration obligations, not guarantees proved by passing the planner tests.

Reconstruct compensation from the successful prefix

SagaCompensations stops at the first non-ok status, including skipped or malformed statuses. Entries after that boundary cannot retroactively justify compensation. For the prefix below, eligibility also requires Compensable=true and a nonempty action.

Index Name Status Action Compensable
0 reserve ok release-v1 yes
1 charge ok refund yes
2 reserve ok release-v2 yes
3 audit ok undo-audit no
4 ship failed cancel yes
5 notify ok undo-notify yes

Boundary=4. Scan eligible successes backward: reserve at 2 contributes release-v2, charge at 1 contributes refund, reserve at 0 is already seen. Result is [release-v2,refund]. The latest eligible success wins, not simply the latest record with the same name: a noncompensable retry does not erase an earlier compensable success. With no failure boundary, return nil.

This planner orders actions; it does not execute them, persist retries or make compensation succeed. A refund is a new business action and may fail or require human resolution. Give compensations stable operation IDs, scope them to the original effect, and track their outcomes. Reversing a list does not create an atomic rollback across services.

Put fencing at the resource boundary

FencingDecisions stores the greatest granted token per resource and the first decision per OperationID. New writes require equality with the known grant. A forged larger token is rejected too. Client is descriptive metadata; the exercise authenticates neither clients nor grants.

Event Stored token for cart Result
write op-a with token 4 absent reject no-grant, cache rejection
grant cart token 4 4 no write decision
retry op-a token 4 4 replay original no-grant rejection
write op-b token 4 4 accept, cache acceptance
grant cart token 5 5 ownership advances
retry op-b token 4 5 replay acceptance, no new write
write op-c token 4 5 reject stale-token
write op-d token 6 5 reject stale-token; 6 is not granted

Check the idempotency cache before fencing. A replayed accepted decision reports what already happened; it does not authorize applying an old write again. Caching only successes also violates the contract: op-a must keep its first rejection. A new logical attempt needs a new ID. IDs are global across this event stream, so the same ID cannot mean unrelated writes on different resources.

In a real sink, checking the token and applying the effect must form one atomic operation relative to ownership changes. Checking a lease only at the client leaves the paused-client race open. This model has a grant stream visible to the sink; other fencing designs may enforce an increasing accepted-token rule instead of the exact-equality rule used here.

Complete a changed trace

Take the saga prefix above, but make index 2 noncompensable and index 4 skipped. Predict rollback order. Then insert a grant of token 3 after token 5 in the fencing trace and predict whether a fresh token-3 write succeeds.

Check both predictions

Skipped still creates the boundary. The eligible reverse order is refund, release-v1; the noncompensable reserve record cannot hide the earlier eligible one. Grant 3 does not lower the stored maximum 5, so the fresh write is rejected.

Implement saga, 2PC and fencing, then design a crash timeline for the integrated inventory/payment/shipment workflow. Identify the durable fact required at each restart. Your design should say who retries, what key deduplicates the effect, and what remains unknown after a timeout; naming a protocol is not enough.


Practice sequence

Use the order below for a guided path, or choose an item to revisit. A pass records a successful attempt; try a fresh example later to check what you can reconstruct.

Join Discord