The 100 problems, solved / Messaging and notifications / 031 · Design Google Meet

Design Google Meet

Real-time audio and video for a 100-person call with mute, share and recording.

Concepts this design leans on

01The design, in the six-step framework

1. Requirements

Functional

  • Create a meeting, join by link, see and hear up to 100 participants with a gallery of the most active speakers.
  • Mute, camera off, screen share, in-meeting chat.
  • Record the meeting to cloud storage for later playback.
  • Out of scope: calendar scheduling, dial-in by phone, live captions.

Non-functional

  • One million concurrent meetings; the median meeting has 5 participants, the tail has 100.
  • Mouth-to-ear latency under 150 ms within a region, under 300 ms across continents.
  • Survive 5 percent packet loss and bandwidth from 300 kbps to 4 Mbps per participant without the call dropping.
  • A media server failure must not end the meeting; reconnect within 5 s.
  • Recording must capture every participant's audio and the active video even if a participant's connection flaps.
Back of the envelope

One participant sends one video stream (say 1.5 Mbps at 720p) and receives up to 8 tiles. With a selective forwarding server the participant uploads 1.5 Mbps once and downloads 8 × 1.5 = 12 Mbps at most; the server receives 100 × 1.5 Mbps = 150 Mbps for a 100-person meeting and must send each participant up to 12 Mbps.

Server egress for a 100-person meeting ≈ 100 × 12 Mbps = 1.2 Gbps; a 10 Gbps media server hosts about 8 such meetings, or about 500 five-person meetings.

One million concurrent meetings of 5 people: 5 M participants × 1.5 Mbps in = 7.5 Tbps of media into the fleet, so about 2,000 media servers plus headroom, spread across regions.

Recording at 2 Mbps for 1 h is 900 MB; if 10 percent of meetings record, that is tens of petabytes a year.

2. Core entities

  • Meeting: meeting_id, host, settings (lobby, recording), region, assigned media server.
  • Participant: participant_id, meeting_id, user or guest, device capabilities, joined_at.
  • Track: track_id, participant_id, kind (audio, video, screen), simulcast layers available.
  • Subscription: which tracks and layers each participant currently receives.
  • Recording: meeting_id, storage object, segments, state.

3. API

  • POST /meetings: create; returns meeting_id and a join link. Called by the host's client.
  • POST /meetings/{id}/join: returns the media server address, a short-lived token, and ICE/TURN servers. Called by every client.
  • WS /meetings/{id}/signal: the signalling channel for SDP offers and answers, ICE candidates, mute state, and subscription changes.
  • POST /meetings/{id}/recording start or stop. Called by the host.
  • GET /recordings/{id}: playback URL once processing is complete.

4. High-level design

ClientsEdgeServicesDataWebSocketSRTP mediaif NAT blocksmirrored tracksBrowser / app(WebRTC)TURN / STUNrelaysMedia servers(SFU)Signalling serviceMeeting controller(placement, roster)Recording workersMeeting state(Redis)Meetings DBRecordings(object storage)

Joining a meeting:

  1. The client calls join; the meeting controller looks up the meeting's assigned media server (chosen at creation in the region nearest the host) and returns its address with a token.
  2. Over the signalling WebSocket the client and the media server exchange an SDP offer and answer and ICE candidates; if the client's network blocks direct UDP, media flows through a TURN relay.
  3. The client starts sending its audio and video with simulcast: three video layers at different resolutions, so the server can choose per receiver.
  4. The selective forwarding unit forwards each participant's chosen tracks to the others without decoding; it subscribes each receiver only to the tiles on screen, at the layer their bandwidth allows.
  5. If recording is on, the server mirrors the tracks to a recording worker that composes and encodes them into segments written to object storage.
  6. Roster and mute state travel over signalling and are kept in Redis so a reconnecting client, or a replacement media server, can rebuild the picture.

5. Deep dives

Forwarding or mixing: why an SFU and not an MCU or a mesh?

OptionHowTrade-off
Mesh (peer to peer)Every participant sends to every otherUpload grows with participants: 10 people is 13.5 Mbps up; dies at 10
MCU (mixing)Server decodes all streams, composes one picture, re-encodes per receiverOne stream per client, but decoding 100 streams is expensive CPU and adds 100 ms
SFU (forwarding)Server routes encrypted packets, picks layers, never decodesCheap and low latency; the client decodes several streams and the server needs simulcast to adapt

Choose the SFU: it keeps latency near network latency and scales on bandwidth rather than CPU, and simulcast gives per-receiver adaptation without transcoding. An MCU is still used for the recording path and for phone dial-in, where a single composed stream is the product.

How does a 100-person meeting stay under 150 ms and under 12 Mbps per receiver?

Nobody can watch 100 tiles. The server sends each receiver only the tiles its layout shows (the active speakers, chosen by audio energy) at the layer that fits its measured bandwidth: a phone on a weak link gets 180p thumbnails and one 360p speaker, a desktop on fibre gets 720p. Audio is always forwarded from the loudest few participants only; the rest are silence and are dropped at the server. Bandwidth estimation on each receiver drives layer switches within a second or two, and packet loss is covered by forward error correction and retransmission for audio, while video simply drops to a lower layer.

What happens when a media server dies mid-meeting?

The server holds the live forwarding state but the meeting's truth, roster, mute flags and track list, lives in Redis and the meetings database. The controller detects the failure by missed heartbeats within 2 s, assigns a replacement server in the same region, and tells every client over signalling to renegotiate. Clients re-offer, media resumes within about 5 s, and the recording worker, which buffered the last few seconds, resumes from the new server. Large meetings are also split across several media servers in a cascade, so one failure affects a fraction of participants.

How are participants on three continents served?

A single media server in one region gives 300 ms round trips to the far side. The cascade solves this too: each region runs a media server for the meeting, participants connect to the nearest, and the regional servers forward tracks to each other over the provider's backbone. The cross-continent hop is paid once per track rather than once per receiver, and multi-region placement happens at join time based on the client's location. The cost is a second hop of 10 to 20 ms within the backbone, which is well inside the budget.

6. What a strong answer sounds like

  • Say the three architectures (mesh, MCU, SFU) and pick the SFU with simulcast; explain in one sentence why.
  • Quote the per-receiver budget, 1.5 Mbps up and about 12 Mbps down, and show how the server stays at network latency.
  • Separate signalling (small, reliable, over WebSocket) from media (large, lossy, over SRTP and UDP); they have different failure modes.
  • Expect follow-ups on NAT traversal, on end-to-end encryption with an SFU (possible with insertable frames; the server forwards without keys), and on recording.
  • A common mistake is to route media through an HTTP API tier; media must take the shortest path and the API tier only sets it up.

02Where this question was reported