Design live chat for a YouTube live stream

A million viewers in one chat: fan-in of messages, fan-out to everyone, with moderation.

Problem 030 of 100Messaging and notificationsL51 Nov 2026

Concepts this design leans on

WebSockets, long polling and SSE — Keeping a connection open so the server can speak first.
Fan-out on write versus fan-out on read — Pushing a post to every follower's inbox, or assembling the feed when asked.
Message queues and pub/sub — Decoupling producers from consumers with a durable log in between.
Rate limiting — Token buckets, sliding windows, and enforcing them across many servers.
Sharding and partitioning — Splitting data by key so each machine holds a slice it can handle.

01The design, in the six-step framework

1. Requirements

Functional

Viewers of a live stream post short messages and see other viewers' messages as they arrive.
Moderation: blocked words, slow mode, timeouts and bans by the creator's moderators.
Highlighted messages (paid or pinned) stay visible longer.
Out of scope: the video itself, replay chat alongside recorded video, emoji packs.

Non-functional

One stream may have 1 million concurrent viewers; the platform has thousands of streams at once.
A message appears for other viewers within 500 ms, p99, in the same region.
During a spike, the room may send 10,000 messages a second; viewers cannot read that, so sampling is allowed, dropping is not an error.
Highlighted messages are delivered to every viewer, never sampled.
Availability over consistency: a missed ordinary message is acceptable, a frozen chat is not.

Back of the envelope

Fan-out is the whole problem: 10,000 messages a second × 1 million viewers = 10 billion deliveries a second if every viewer saw every message.

A human reads at most 5 to 10 messages a second, so cap delivery at 10 per second per viewer: 10 × 10⁶ = 10 million deliveries a second for the biggest room.

A message is about 100 bytes; 10 million a second is 1 GB/s of push traffic for one room, spread over edge servers each holding 50,000 sockets: at least 20 edge servers per million viewers.

Thousands of smaller rooms add less than one big one; design for the largest room and partition by room.

2. Core entities

Room: room_id (one per live stream), settings (slow mode interval, blocked words), moderator list.
Message: message_id, room_id, user_id, text, kind (normal, highlighted, system), sent_at.
Viewer session: user_id, room_id, edge server, joined_at.
Moderation action: room_id, target user_id, type (timeout, ban), until.

3. API

WS /rooms/{id}/connect: the viewer's socket; the server pushes a stream of message batches. Called by the player.
POST /rooms/{id}/messages with text; returns message_id or a reason (slow mode, timed out). Called by the player.
POST /rooms/{id}/moderation with target and action. Called by moderators.
GET /rooms/{id}/recent?limit=50: the last messages for a viewer who just joined.
PUT /rooms/{id}/settings: slow mode, blocked words. Called by the creator.

4. High-level design

One message in a million-viewer room:

A viewer's player sends the text over its WebSocket to the edge server it is connected to; the edge forwards it to the ingest service.
Ingest applies per-user rate limits and the room's slow mode, checks the moderation service for timeouts and blocked words, and rejects fast if any fail.
Accepted messages are published to the room's topic in the pub/sub layer; the topic is partitioned by room_id so a huge room has its own partition.
The room's fan-out process consumes the topic, samples to at most 10 messages per 100 ms window while always keeping highlighted ones, and sends one batch per window to every edge server that has viewers in the room.
Each edge server writes the batch to all of its local sockets for that room: one network message in, 50,000 out, with no per-viewer decision.
A viewer who joins late fetches the last 50 messages from the room's Redis list, then starts receiving batches.

5. Deep dives

How do we deliver to a million viewers without a million lookups?

Treating live chat like a chat app, with a session registry lookup per recipient, costs a million lookups per message. The trick is that every viewer in a room wants the same bytes. So the fan-out tree is two levels: the room's fan-out process knows only which edge servers hold at least one viewer of the room (a few hundred entries), and each edge server knows its own sockets. The registry is per room per edge, not per user. A new edge joins the tree when its first viewer of that room connects and leaves when its last one disconnects.

What happens at 10,000 messages a second, and is dropping acceptable?

Option	Behaviour	Trade-off
Deliver everything	10 billion deliveries a second	Impossible, and unreadable even if possible
Slow mode only	Each user may post every N seconds	Lowers input but 1 M users at one message a minute is still 16,000 a second
Sample at fan-out	Keep a readable rate per window, always keep highlighted and moderator messages	Most viewers never see most messages; senders still see their own echo

Sample at fan-out and echo the sender's own message locally; the interviewer's follow-up is whether a dropped message is a correctness bug, and the answer is no, because the product promise is a readable stream, not a complete one. The archive keeps every accepted message for replay and moderation review.

How do moderation actions take effect within a second across all edges?

A ban must stop future messages (an ingest-side check, cheap) and ideally remove the user's recent messages from screens (an edge-side delete event). The moderation service writes the action to the room's settings record and publishes a system event on the room topic; edges apply deletes locally. Blocked-word lists are compiled into a matcher cached in ingest with a short TTL, so a new rule reaches every ingest instance within a few seconds without a deploy.

Is global replication needed, and what does a 500 ms budget allow?

Viewers of one stream are worldwide, but the room's fan-out process lives in one region, and cross-region latency is 100 to 200 ms. Budget: ingest 20 ms, pub/sub 20 ms, fan-out window 100 ms, cross-region hop 150 ms, edge write and client render 50 ms, about 340 ms. So a single-region room with regional edge servers meets 500 ms; replicating the room's fan-out into every region would only buy latency for the rare room whose viewers are far from its home, at the cost of ordering differences between regions. Place the room in the region nearest the streamer and let edges everywhere subscribe; do not replicate the room state.

What does an edge server failure look like to viewers?

An edge holds 50,000 sockets; when it dies those viewers see a disconnect and reconnect through the load balancer to another edge within a second or two. The new edge adds itself to the room's fan-out tree if it was not already in it, and the client refetches the last 50 messages so the gap is invisible. Nothing about the room state lived on the dead edge, which is the point of keeping edges stateless apart from their sockets.

6. What a strong answer sounds like

State the fan-out number first (10 billion per second if naive) and let it drive the design toward a two-level tree and sampling.
Separate the two message classes: ordinary messages are sampled, highlighted and moderator messages are guaranteed.
Walk the latency budget to show that one region plus edge servers is enough; say why you would not replicate the room.
Expect follow-ups on a viewer who joins late, on what happens when an edge server dies (viewers reconnect to another edge and refetch recent), and on replaying chat with a recording.
A common mistake is to reuse a per-recipient messaging design and discover the session-registry cost only when the interviewer asks about a million viewers.

02Where this question was reported

Exponent · Classic distributed-systems category
gitgood.dev · Candidate reports; mock interviewer attacked a 500 ms budget and asked whether global replication is even needed.