Designing a CDN: Cache Hierarchy, Invalidation, and Request Routing

Every large system design answer ends with someone waving a hand and saying "and we put a CDN in front." Fine — but what is inside that box? Designing the CDN itself is a fantastic senior-level interview because it composes half the classics: geo-routing, cache hierarchies, consistent hashing, invalidation fan-out, and hot-object load spikes, all in one product.

Requirements

Serve static and streaming content from locations near users. Cache hit ratio >90%, added latency under 30ms for a hit, origin protected from thundering herds, global purge in seconds, per-customer configuration (TTLs, cache keys, signed URLs).

The hierarchy: edge, shield, origin

A flat "edge talks to origin" design collapses on cache misses: 300 PoPs missing the same object simultaneously means 300 origin fetches. Real CDNs are a tree.

users -> edge PoP (100s, small hot cache)
             -> regional / shield PoP (10s, large cache)
                    -> customer origin (one fetch, ideally)

The origin shield is a designated mid-tier PoP per customer origin; all misses funnel through it, so the origin sees ~1 fetch per object per TTL instead of hundreds. Within a PoP, servers use consistent hashing on the cache key so each object lives on one machine rather than being duplicated across all of them — the same ring pattern as a distributed cache, applied inside the rack.

Two more protections every interviewer will probe:

• Request coalescing: 10,000 concurrent requests for a just-expired object must become one upstream fetch; the other 9,999 wait on that in-flight fetch (or receive stale-while-revalidate). Without this, every TTL expiry of a popular object is a self-inflicted DDoS on your shield.
• Negative caching: cache the 404/error for a short TTL, or a misbehaving client hammers your origin through every miss.

Cache keys and correctness

The cache key defaults to host + path, but query strings, Vary headers, device class, and cookies all threaten it. The two failure modes are symmetric: include too much in the key and your hit ratio dies (?utm_source= fragmenting one object into thousands); include too little and you serve user A's private response to user B — the classic cache-poisoning incident. Per-customer key configuration plus a hard rule of never caching Set-Cookie responses by default is the sane baseline.

Invalidation: purge in seconds, globally

TTL expiry is the passive path. The active path — "purge this object everywhere, now" — is a distributed broadcast problem: one API call must reach hundreds of PoPs and thousands of machines in under a few seconds, surviving PoPs that are offline mid-purge.

The standard design is a pub/sub control plane: purges append to a replicated log (Kafka-style); every PoP tails the log and applies deletions locally; a PoP that was offline replays from its last offset — the log's ordering gives you correctness after partitions. Fastly famously gets this under ~150ms globally.

Two implementation tricks worth naming:

• Tag-based purge ("purge everything tagged product-123"): store tag→object mappings at the edge, or version the cache key (/v42/product-123.jpg) so "purge" is just bumping a pointer — the old objects age out untouched.
• Soft purge: mark stale instead of deleting, serve stale while revalidating. Purge storms then never translate into origin storms.

Request routing: getting the user to the right PoP

Two mainstream mechanisms, and a good answer contrasts them:

	DNS-based	Anycast
How	Resolver's location picks PoP IP	Same IP announced from all PoPs; BGP routes to "nearest"
Granularity	Coarse (resolver ≠ user; ECS helps)	Network-topology accurate
Failover	TTL-bound (60s of stale answers)	Near-instant (BGP reconverges)
Control	Easy weighted steering	BGP "nearest" ≠ lowest latency; long-lived TCP can reroute mid-flow
Used by	Akamai (classically)	Cloudflare, Google

A hybrid is common: anycast to reach an entry PoP cheaply, then internal latency-measured steering (an in-house traffic map recomputed from real client RTTs) to pick where the request is actually served.

The hot object problem

A world-cup goal clip goes viral: one object, millions of QPS, concentrated on one consistent-hash owner per PoP. The mitigation ladder: detect hotness via a lightweight counter, then (1) replicate the hot object across all servers in the PoP instead of one, (2) serve from memory not disk, (3) for live streaming, chunk the stream so load spreads across many small objects with staggered TTLs. This is the same hot-partition medicine as in databases — detect skew, break the single owner.

What the strong answer sounds like

Tree-shaped cache hierarchy with an origin shield; request coalescing and negative caching as origin armor; cache keys as a correctness surface, not a config detail; purge as a replicated log fan-out with soft-purge semantics; anycast plus measured steering for routing; and an explicit hot-object story. If you can also say why each layer exists in terms of the failure it prevents, you are answering at the level the question is testing for.