Concepts / Fundamentals / Replication

Replication

Keeping copies so reads scale and a dead machine loses nothing.

System design conceptFundamentals1 Nov 2026

01The intuition

The problem it solves

A school keeps the only copy of its exam results in one notebook in the office. Two bad things can happen. On results day 800 parents want to read it and queue for hours. And one monsoon, the roof leaks over the office, and the notebook is gone. The fix for both is copies: a photocopy in each classroom for parents to read, and one locked in the principal's house.

A database on one machine has the same two problems: it can serve only so many reads, and when its disk dies the data is gone. Replication keeps the same data on several machines so reads spread out and a dead machine loses nothing.

Build it one piece at a time

Step 1: one leader, one follower. All writes go to the leader. The leader appends each change to its log and sends the log to the follower, which applies it in the same order. After a write of "Rao: 91 marks", the leader has it immediately and the follower a few milliseconds later.

TimeLeaderFollowerReader asking the follower sees
0 msRao: 91(nothing yet)no result for Rao
5 msRao: 91Rao: 9191

That 5 ms gap is replication lag, and it is the source of every surprising behaviour in replicated systems.

Step 2: send reads to followers. Three followers triple read capacity. A reader who just wrote and reads back from a lagging follower may not see their own write; the fix is to send a user's reads to the leader for a few seconds after their write, or to carry a version number and wait for the follower to catch up. The consistency models article names these promises.

Step 3: synchronous or asynchronous. If the leader waits for a follower to confirm before telling the client "done", a leader crash loses nothing but every write pays a round trip. If it does not wait, writes are fast and a crash can lose the last few milliseconds. Most systems wait for one follower and not the rest.

Step 4: promote a follower. When the leader dies, a follower becomes leader. Somebody has to decide which one and make sure the old leader does not come back and keep accepting writes; that is leader election, usually done with consensus.

Step 5: more than one leader. For a service in Mumbai and in Virginia, writes to a single leader cost a 250 ms trip. Multi-leader replication accepts writes in both and merges them later, which means two people can change the same record at the same time and the system needs a rule for who wins. Leaderless designs (every replica accepts writes, readers ask several) push the same problem to quorums.

Why it works, with the numbers

Durability multiplies. If one disk has a 2 percent chance of dying in a year and copies fail independently, three copies in three places lose data only if all three die before a replacement is made:

P(lose all three in the same week) ≈ (0.02 × 1⁄52)3 ≈ 6 × 10−11

Read capacity adds: a leader plus three followers serves about four times the reads, at the cost of four times the storage and a little lag. The guarantee of leader-based replication is simple to state: followers apply the leader's changes in the leader's order, so every follower is a slightly old but internally consistent snapshot. The failure modes: a follower too far behind (a reader sees last hour's data), a split brain where two machines both believe they are leader, and a synchronous follower that is down and freezes all writes.

The trade-off you must say out loud

Replication buys durability and read scale with lag and complexity. Say which writes must be acknowledged by how many copies, and what a reader is promised after a write. The next question is "what happens when the leader dies?" and the honest answer includes the seconds of unavailability during election and the writes that may be lost if replication was asynchronous.

Where it shows up in the 100 Google problems

02Used in these problems