CDNs and edge caching

Copies of static bytes near the user, and how they stay fresh.

System design conceptCaching and edges1 Nov 2026

01The intuition

The problem it solves

A video server sits in a data center in Iowa. A student in Hyderabad presses play. Her request crosses the Pacific, about 12,000 km of fibre, and the first bytes come back after roughly 250 milliseconds of round trip. Every chunk of the video repeats that trip, and a single server's network link, say 10 gigabits a second, can feed only about 2,000 viewers at 5 megabits each. A million viewers in India would need 500 such links and would still feel the ocean in every rebuffer.

Notice two things. Most of those million viewers are watching the same few thousand videos. And nothing about the bytes changes between viewers. Shipping the same bytes across the ocean a million times is the waste. The fix is to keep copies of popular files on servers near the viewers. That network of nearby servers is a content delivery network, a CDN, and it is a cache with a map.

Build it one piece at a time

Step 1: one edge server. Put a server in Mumbai with a large disk. When the student asks for video123/segment7, the Mumbai server checks its disk. On a miss it fetches the segment from Iowa (the origin), stores it, and serves it. The second Hyderabad viewer gets it from Mumbai in about 30 ms.

Request	Mumbai edge	Origin fetch	Round trip seen by viewer
segment7, viewer A	miss, store	yes	250 ms
segment7, viewer B	hit	no	30 ms
segment7, viewer C	hit	no	30 ms

Step 2: send each viewer to the nearest edge. Add edges in Chennai, Singapore, Frankfurt and so on, and make the name cdn.example.com resolve to a different address depending on where the asker is. Two tools do this: DNS that answers with the nearest edge's address, or anycast, where every edge announces the same address and the internet's routing carries each packet to the closest one.

Step 3: a middle layer so the origin is not hit by every edge. With 200 edges, one new video would be fetched from Iowa 200 times. Put a few regional caches between the edges and the origin: Mumbai asks the Asia regional cache, which asks the origin once and feeds all Asian edges. The origin now sees a handful of requests per file, no matter how many edges exist.

Step 4: decide how long a copy lives. Each file carries a TTL in its HTTP headers. A video segment never changes, so it can live for a year. A news home page changes every minute, so it lives for 30 seconds. For files that must change in place, the CDN offers a purge call that deletes the copy from every edge, which takes seconds to minutes. The cleaner trick is to never change a file: put a version in the name, app.v42.js, cache it forever, and make the HTML point at the new name.

Step 5: which files belong at the edge. Anything that is the same for every viewer: videos, images, scripts, stylesheets, fonts, map tiles, model files. Anything personal, like a feed or a bank balance, still goes to the origin, though the edge can still terminate TLS and absorb bad traffic on the way.

Why it works, with the numbers

Two quantities: the hit rate h at the edge, and the bytes the origin must send. If a million viewers watch 1 GB each and h is 0.95, the origin sends only

(1 − h) × 1,000,000 × 1 GB = 50,000 GB = 50 TB instead of 1,000 TB

and the viewer's time to first byte falls from 250 ms to around 30 ms, because light in fibre travels about 200 km per millisecond and Mumbai is 700 km from Hyderabad, not 12,000. The guarantee is "you receive a copy that was fresh within the TTL". The failures are stale content after an upload if the TTL is too long, a thundering herd on the origin when a popular file is new or just purged, and the unavoidable fact that the first viewer in each region pays the long trip.

The trade-off you must say out loud

A CDN trades freshness and money for latency and origin load. Every byte at the edge costs storage and egress fees, and every purge is a race you can lose for a few seconds. Say which content is immutable (cache forever, version the name), which is short-lived (short TTL), and which is personal (never cached). The follow-up is "what about the first request after a new upload" and the answer is pre-warming: push the file to the regional caches before you announce it.

Where it shows up in the 100 Google problems

YouTube: every segment of every rendition is cached at the edge; the origin serves the long tail.
Video distribution CDN: the CDN itself is the design, with its edges, regional tier and routing.
Google Maps: map tiles are immutable images named by zoom and position, perfect for a year-long TTL.
Model file distribution: a 100 GB model is a static file; a CDN-like tree of caches gets it to 10,000 servers without 10,000 origin reads.
IP denylist: the edge is where blocking is cheapest, before the request reaches anything expensive.