5 min readRishi

Multi-Region Active-Active: Patterns for Serving Writes from Everywhere

Multi-region starts as a checkbox ("we need DR") and becomes a philosophy exam the moment someone asks: if both regions accept writes, what happens when they disagree? Active-passive dodges the question and pays for it — an idle region that costs full price, and a failover path exercised only during real disasters, which is why it fails. Active-active answers the question deliberately, and the answer is never "we handle conflicts everywhere"; it's a portfolio of per-data-class decisions. That portfolio is the staff-level interview.

The taxonomy: where does a write's truth live?

Pattern 1 — Partitioned ownership (region pinning). Every entity gets a home region (user's geography, tenant's residency); writes for it route home, other regions hold read replicas. Both regions are fully active — for different subsets. Conflicts are impossible by construction because each datum has one writer region.

user in EU -> EU region owns their writes; US serves their reads (async replica)
routing: edge/gateway consults entity->home mapping (config/directory)

The costs: a traveling user (or cross-entity operation) pays cross-region latency to their home; the entity→region directory is now critical infrastructure; and rehoming (migrating an entity's ownership — for a move, or a regional evacuation) is a real protocol: freeze-drain-flip with fencing (epoch per entity, storage rejects stale-epoch writers), the distributed-locking playbook applied at region grain. This is the workhorse pattern — most "active-active" systems are region-pinned underneath, and B2B/data-residency requirements push the same shape anyway.

Pattern 2 — True multi-writer with convergence. Any region accepts any write; replication is async and bidirectional; conflicts are expected and resolved by data design: CRDTs and version vectors where merge semantics exist (carts, presence, counters — the Dynamo lineage), LWW where losing a write is genuinely acceptable, and application-level merge queues for the rest. Reserve this for data that has a safe merge — and say explicitly that uniqueness invariants (usernames, payments, inventory) cannot live here.

Pattern 3 — Consensus-spanning (Spanner-class). Synchronous quorums across regions: real global consistency, write latency = inter-region RTT, cost = running the quorum everywhere. Reserved for the small core that truly needs global linearizable writes — the config plane, the ledger of record — not the request path at large.

The senior move is refusing to pick one: classify data, assign each class a pattern. Sessions → multi-writer CRDT-ish; user content → region-pinned; payment ledger → single-home (or consensus); derived caches → rebuild locally per region.

The unglamorous 80%: routing, replication lag, and drills

Routing is where active-active succeeds or dies: geo-DNS/anycast steers users to the nearest healthy region, but stickiness matters more than nearness — a user bouncing between regions mid-session experiences every replication lag as a bug (write in US, refresh hits EU, post "vanished"). Session affinity to the write path + read-your-writes tokens (session guarantees, deployed at planetary scale) paper over async replication for the author; everyone else's staleness is declared acceptable per data class. Cross-region replication is a product surface, not plumbing: lag is monitored per class with SLOs, and the event backbone (Kafka mirroring / CDC streams) carries ordering caveats — per-key order survives mirroring, global order doesn't; design consumers accordingly.

Failover is a practiced operation, not a wiki page. Evacuating a region means: flip routing weights (gradual, not cliff), rehome pinned entities (or accept their unavailability — say which), let multi-writer classes just keep going (their whole point), and fence the sick region so a zombie (partitioned-but-alive, still processing queued writes) can't split-brain — epoch bump distributed via the config plane, storage and consumers reject the old epoch. The teams that survive real regional outages are the ones running game days: scheduled evacuations of production traffic, because a failover path that isn't exercised monthly is fiction. Capacity math follows: N regions must each carry the load of N-1 failing (2 regions = each runs below 50% — active-active's "efficiency" is partly an illusion you budget honestly).

And the quiet prerequisite for all of it: regional isolation — each region a self-sufficient stack (its own caches, queues, DBs, dependencies), cross-region calls forbidden on the request path. Most "multi-region" outages are one region synchronously calling another because someone took a shortcut; blast-radius discipline is what makes the patterns above meaningful.

Interview probeAnswer sketch
Global unique username at signup?The one linearizable moment: single-home the claim (global uniqueness service / consensus), then pin the user; don't make the whole user multi-writer for one constraint
User flies from EU to US?Reads go local (replica), writes go home (latency, hidden by async UX) or trigger rehoming after a stability window
Replication link severed 10 minutes?Pinned classes: remote reads stale, home writes fine. Multi-writer: divergence accrues, converges on heal — inventory-like data must not be in this class
How do you test conflict handling?Chaos: inject partition + concurrent writes in staging with production topology; conflicts you've never manufactured are conflicts you can't survive

The distilled doctrine: classify data by its merge-ability and invariants; pin what can't merge, converge what can, consensus the tiny core that needs the world to agree; make regions islands; and rehearse the evacuation until it's boring. Multi-region isn't a topology — it's a set of promises, kept per data class.

Keep reading

Newsletter

New posts, straight to your inbox

One email per post. No spam, no tracking pixels, unsubscribe anytime.

Comments

  • No comments yet. Be the first.