The 100 problems, solved / Messaging and notifications / 026 · Design a chat app (WhatsApp / Google Chat)

Design a chat app (WhatsApp / Google Chat)

One-to-one and group messages, delivery receipts, offline users, and 10 million concurrent connections.

Problem 026 of 100Messaging and notificationsL51 Nov 2026
Concepts this design leans on

01The design, in the six-step framework

1. Requirements

Functional

  • One-to-one and group messages (groups up to 500 members) with text and small attachments.
  • Delivery state per message: sent, delivered, read.
  • Offline users receive everything they missed, in order, when they reconnect.
  • Presence: online, last seen, typing.
  • Out of scope: voice and video calls, end-to-end key exchange details, message search.

Non-functional

  • 500 million daily users; 10 million connections open at any moment in the largest region.
  • Message delivery to an online recipient in under 500 ms end to end, p99.
  • No message lost once the server has acknowledged it; per-conversation ordering is strict.
  • 99.99 percent availability for sending; reads of history may be slightly stale.
  • History retained for one year on the server; clients hold their own copy.
Back of the envelope

500 M users × 40 messages a day = 20 billion messages a day.

20 × 109 ÷ 86,400 s ≈ 230,000 messages per second average, about 700,000 at peak.

A message row is about 200 bytes with metadata, so 20 G × 200 B = 4 TB a day and about 1.5 PB a year before attachments.

10 M open connections, each with a 20 s heartbeat, is 500,000 tiny packets a second just to stay alive. A chat server can hold about 100,000 idle sockets, so the connection tier alone is 100 machines plus headroom.

2. Core entities

  • User: user_id, display name, device list, last_seen.
  • Conversation: conversation_id, type (direct or group), member list, created_at.
  • Message: message_id, conversation_id, sender_id, sequence number within the conversation, body or attachment pointer, sent_at.
  • Receipt: message_id, user_id, state (delivered or read), at.
  • Session: user_id, device_id, chat server id, connected_at; this is the routing table.

3. API

  • POST /conversations with member ids; returns conversation_id. Called by the client.
  • POST /conversations/{id}/messages with client_message_id, body; returns message_id and sequence. Sent over the open WebSocket in practice; the HTTP form exists for retries.
  • GET /conversations/{id}/messages?after_seq=N&limit=100 for catch-up after reconnect.
  • POST /messages/{id}/receipts with state delivered or read.
  • WS /connect: the long-lived socket on which the server pushes message, receipt, presence and typing events.

4. High-level design

ClientsEdgeServicesDataWebSocketsendappendlookuppublishdeliverofflineheartbeatPhone appL4 load balancerChat servers(hold sockets)Message servicePresence servicePub/sub(per-server topics)Message store(wide-column, by conversation)Session registry(Redis)Push gateway(APNs / FCM)

One message from Asha to Ravi:

  1. Asha's app sends the message over its WebSocket to chat server 17 with a client_message_id it generated.
  2. Server 17 forwards to the message service, which assigns the next sequence number for the conversation and appends the row to the message store, sharded by conversation_id so a whole thread lives together.
  3. The message service acknowledges to Asha (state: sent) only after the store confirms the write.
  4. It looks up Ravi in the session registry: Ravi's phone is on chat server 42. It publishes the message on server 42's topic in the pub/sub layer.
  5. Server 42 pushes the message down Ravi's socket; Ravi's app replies with a delivered receipt, which travels the same path back to Asha.
  6. If Ravi had no session, the message service hands a short payload to the push gateway, and Ravi's app fetches history with after_seq when it wakes.

5. Deep dives

How do ten million open connections find each other?

The simple answer is to broadcast every message to every chat server and let the right one deliver it. At 700,000 messages a second across 100 servers that is 70 million deliveries a second of mostly wasted work. The real question is routing: which server holds Ravi's socket right now.

OptionHowCost
BroadcastEvery server sees every messageWork grows with servers × messages; dies at scale
Session registryRedis map user_id → server_id, written on connect, with a TTL refreshed by heartbeatsOne lookup per recipient; registry must survive a Redis failover
Sticky hashingHash user_id to a server with consistent hashingNo lookup, but a server crash disconnects a fixed slice of users and groups still need lookups

Choose the session registry with a TTL; it costs one cache read per recipient and tolerates servers coming and going, which they do constantly behind a rolling deploy.

How do we guarantee order and no loss inside a conversation?

Clients send concurrently and networks reorder. If two members of a group send at the same instant, which is first? The simple answer, client timestamps, fails because phone clocks drift by minutes. The message service assigns a per-conversation sequence number from the store's atomic increment, which makes the store the single source of order. The sequence also makes catch-up trivial: a client stores the highest sequence it has seen and asks for everything after it. Loss is prevented by acknowledging only after the durable append, and by the client resending with the same client_message_id until it hears the ack; the service treats a repeated client_message_id as the same message, which is idempotency doing the work of exactly-once.

What does a 500-member group cost, and where does it break?

A group message is one write and up to 500 deliveries. At peak a popular group sends 10 messages a second, so 5,000 deliveries a second from one conversation. The message service fans out by looking up all 500 sessions in one batched registry call, grouping recipients by chat server, and publishing one event per server rather than per recipient. The chat server then writes to each local socket. Fan-out stays on the server side; the client never learns about other members' devices. Above a few thousand members this pattern becomes a broadcast channel, which is a different system (live chat).

How does presence scale without flooding the network?

If every user told every contact about every state change, 500 M users with 100 contacts each is 50 billion notifications a day for a feature nobody pays for. Three decisions keep it cheap: heartbeats go to the local chat server only, which batches them into the registry TTL; presence is pulled when a conversation is opened rather than pushed continuously; and last_seen is written at most once a minute per user. Typing indicators are sent only to members with the conversation open and are never stored.

Where do attachments go?

A photo is a few megabytes; pushing it through the message path would multiply the store's write volume by ten thousand. The client first uploads the file to object storage through a signed URL, then sends a message whose body is only the object key, size and a thumbnail; recipients fetch the bytes from a CDN when the thread is opened. This keeps the message store at a few hundred bytes per row and lets the two systems scale separately: messages are many and small, attachments are fewer and large.

6. What a strong answer sounds like

  • Say early that the hard part is not storage but routing: the session registry is the heart of the design.
  • Quote the numbers: 700,000 messages a second, 10 M sockets, about 100,000 sockets per server, 4 TB of messages a day.
  • Explain that ordering comes from one sequence per conversation and loss prevention from ack-after-write plus idempotent resend.
  • Expect follow-ups on multi-device (one user, three sockets: the registry holds a set), on multi-region (route by the conversation's home region and replicate asynchronously), and on end-to-end encryption (the server stores ciphertext; receipts and sequence numbers stay in the clear).
  • A common mistake is to put the message in a queue per user and call it done; that gives at-least-once delivery but no ordering and no catch-up story.

02Where this question was reported