Multi-Region Active-Active: Patterns for Serving Writes from Everywhere
Multi-region starts as a checkbox ("we need DR") and becomes a philosophy exam the moment someone asks: if both regions accept writes, what happens when they disagree? Active-passive dodges the question and pays for it — an idle region that costs full price, and a failover path exercised only during real disasters, which is why it fails. Active-active answers the question deliberately, and the answer is never "we handle conflicts everywhere"; it's a portfolio of per-data-class decisions. That portfolio is the staff-level interview.
The taxonomy: where does a write's truth live?
Pattern 1 — Partitioned ownership (region pinning). Every entity gets a home region (user's geography, tenant's residency); writes for it route home, other regions hold read replicas. Both regions are fully active — for different subsets. Conflicts are impossible by construction because each datum has one writer region.
user in EU -> EU region owns their writes; US serves their reads (async replica)
routing: edge/gateway consults entity->home mapping (config/directory)
The costs: a traveling user (or cross-entity operation) pays cross-region latency to their home; the entity→region directory is now critical infrastructure; and rehoming (migrating an entity's ownership — for a move, or a regional evacuation) is a real protocol: freeze-drain-flip with fencing (epoch per entity, storage rejects stale-epoch writers), the distributed-locking playbook applied at region grain. This is the workhorse pattern — most "active-active" systems are region-pinned underneath, and B2B/data-residency requirements push the same shape anyway.
Pattern 2 — True multi-writer with convergence. Any region accepts any write; replication is async and bidirectional; conflicts are expected and resolved by data design: CRDTs and version vectors where merge semantics exist (carts, presence, counters — the Dynamo lineage), LWW where losing a write is genuinely acceptable, and application-level merge queues for the rest. Reserve this for data that has a safe merge — and say explicitly that uniqueness invariants (usernames, payments, inventory) cannot live here.
Pattern 3 — Consensus-spanning (Spanner-class). Synchronous quorums across regions: real global consistency, write latency = inter-region RTT, cost = running the quorum everywhere. Reserved for the small core that truly needs global linearizable writes — the config plane, the ledger of record — not the request path at large.
The senior move is refusing to pick one: classify data, assign each class a pattern. Sessions → multi-writer CRDT-ish; user content → region-pinned; payment ledger → single-home (or consensus); derived caches → rebuild locally per region.
The unglamorous 80%: routing, replication lag, and drills
Routing is where active-active succeeds or dies: geo-DNS/anycast steers users to the nearest healthy region, but stickiness matters more than nearness — a user bouncing between regions mid-session experiences every replication lag as a bug (write in US, refresh hits EU, post "vanished"). Session affinity to the write path + read-your-writes tokens (session guarantees, deployed at planetary scale) paper over async replication for the author; everyone else's staleness is declared acceptable per data class. Cross-region replication is a product surface, not plumbing: lag is monitored per class with SLOs, and the event backbone (Kafka mirroring / CDC streams) carries ordering caveats — per-key order survives mirroring, global order doesn't; design consumers accordingly.
Failover is a practiced operation, not a wiki page. Evacuating a region means: flip routing weights (gradual, not cliff), rehome pinned entities (or accept their unavailability — say which), let multi-writer classes just keep going (their whole point), and fence the sick region so a zombie (partitioned-but-alive, still processing queued writes) can't split-brain — epoch bump distributed via the config plane, storage and consumers reject the old epoch. The teams that survive real regional outages are the ones running game days: scheduled evacuations of production traffic, because a failover path that isn't exercised monthly is fiction. Capacity math follows: N regions must each carry the load of N-1 failing (2 regions = each runs below 50% — active-active's "efficiency" is partly an illusion you budget honestly).
And the quiet prerequisite for all of it: regional isolation — each region a self-sufficient stack (its own caches, queues, DBs, dependencies), cross-region calls forbidden on the request path. Most "multi-region" outages are one region synchronously calling another because someone took a shortcut; blast-radius discipline is what makes the patterns above meaningful.
| Interview probe | Answer sketch |
|---|---|
| Global unique username at signup? | The one linearizable moment: single-home the claim (global uniqueness service / consensus), then pin the user; don't make the whole user multi-writer for one constraint |
| User flies from EU to US? | Reads go local (replica), writes go home (latency, hidden by async UX) or trigger rehoming after a stability window |
| Replication link severed 10 minutes? | Pinned classes: remote reads stale, home writes fine. Multi-writer: divergence accrues, converges on heal — inventory-like data must not be in this class |
| How do you test conflict handling? | Chaos: inject partition + concurrent writes in staging with production topology; conflicts you've never manufactured are conflicts you can't survive |
The distilled doctrine: classify data by its merge-ability and invariants; pin what can't merge, converge what can, consensus the tiny core that needs the world to agree; make regions islands; and rehearse the evacuation until it's boring. Multi-region isn't a topology — it's a set of promises, kept per data class.
Keep reading
Consistency Models Beyond CAP: Linearizability, Causal, and Session Guarantees
'Strong vs eventual' is a cartoon. The real spectrum — linearizable, sequential, causal, session guarantees — and how to pick per operation, not per system.
Designing an API Gateway: The Front Door as a System
Authentication, routing, rate limits, transformations, and a plugin chain — in a tier that must add ~1ms and never be the outage. Envoy/Kong architecture from scratch.
Designing a CDC Pipeline: Change Data Capture from Binlog to Downstream
Dual writes are a lie; the database's own log is the truth. Log-based CDC, the snapshot-plus-stream handoff, schema evolution, and ordering guarantees that survive resharding.
Newsletter
New posts, straight to your inbox
One email per post. No spam, no tracking pixels, unsubscribe anytime.
Comments
- No comments yet. Be the first.