Design Google Home

01The design, in the six-step framework

1. Requirements

Functional.

• A speaker in the home listens for a wake word, records the request, understands it, and either answers (weather, a fact, a timer) or acts (play music, turn off the lights, add to a shopping list) and speaks the result.
• Multiple devices per home and several people per device; the device knows which account is speaking for personal requests (calendar) and which devices it controls.
• Third-party actions (a smart bulb vendor, a music service) are integrated through a fulfilment interface.
• Out of scope: the microphone hardware, the speech models' training, the music catalogue.

Non-functional.

• 100 million devices; 50,000 requests per second at peak (evenings).
• End-to-end response (end of speech to start of spoken answer) under 1 second at p90; lights should change within 500 ms.
• Privacy: audio is processed only after the wake word; recordings are not retained unless the user opts in; the device works offline for timers and local device control.
• Availability 99.95 percent for the cloud path; the local path keeps working through an outage.

2. Core entities

• Device: device_id, home_id, capabilities, firmware version, linked accounts, local network info.
• Home graph: home_id, rooms, devices in rooms (speakers, lights, thermostats), the vendor and control endpoint for each.
• Request: request_id, device_id, speaker identity (if recognised), audio stream, transcript, intent, slots, fulfilment result, timings.
• Session: conversation state across follow-up turns ("and the kitchen?") with a short TTL.
• Action provider: vendor, supported intents, auth link per user, endpoint, latency history.

3. API

• Device → cloud: a bidirectional stream per request: audio chunks up; partial transcripts, the final response (text, synthesised audio chunks, display card) and device commands down.
• POST /homes/{id}/devices and GET /homes/{id}/graph: setup and the assistant's view of what it can control.
• Fulfilment interface for providers: POST {provider}/fulfil {intent, slots, user token} → {speech, display, follow-up expected}; and for device control POST {provider}/execute {device ids, command} → per-device status.
• Local: devices on the same network accept execute over the LAN for supported vendors, used when the cloud is unreachable or for speed.

4. High-level design

"Turn off the kitchen lights":

1. The on-device wake-word model triggers; the speaker opens a stream and sends audio as the person speaks, with the device id and an identity hint from a local voice match.
2. Streaming recognition returns partial transcripts; end of speech is detected on device and the final transcript arrives about 200 ms later.
3. Understanding maps the transcript to intent device.off with slot room=kitchen, resolved against the home graph (which lights are in the kitchen, which vendor controls them).
4. The orchestrator checks the capability contract, sends execute commands to the vendor (or over the LAN if the bulbs support it), and in parallel starts synthesis of the confirmation.
5. Audio chunks stream back to the speaker as soon as the first is ready; the device reports completion and the request's timings are logged for the latency dashboards.

5. Deep dives

Where does the second go, and how do we stay under it?

Everything streams and nothing waits for the whole of the previous stage. Audio is sent while the user is still speaking, so recognition has consumed most of it before they stop; understanding starts on the stable prefix of the transcript for simple intents; synthesis starts as soon as the first sentence of the answer is known; the device plays the first chunk while the rest is generated. The cloud path keeps per-stage deadlines with fallbacks: if a provider does not answer in 400 ms the response says "I've asked the lights to turn off" and the command completes asynchronously, rather than the user waiting in silence. Connection setup is the hidden cost: the device keeps a warm, authenticated connection to the gateway in its region, so a request does not pay for TLS and DNS. The bidirectional stream is what makes partial results possible in both directions.

How does the device keep working offline, and why does that matter for privacy too?

A small on-device model handles the wake word, timers, alarms, volume and local device control for vendors that support LAN commands; when the cloud is unreachable those still work, and when it is reachable they are faster. The same architecture is the privacy story: nothing leaves the device before the wake word, the wake-word model runs locally with no network, and the audio sent is only the request. Recordings are processed in memory and discarded unless the user opted in; transcripts kept for the session TTL enable follow-ups and are then deleted; the user can see and delete history. Multi-user homes rely on voice match computed on device and sent as a hint; the cloud verifies before exposing personal data (a calendar), and ambiguous matches get a neutral answer. Stating that privacy decisions are placed in the architecture, not in a policy document, is the expected level here.

How are third-party providers integrated without their latency and failures becoming ours?

Through a fixed fulfilment contract (intents, slots, response shape) with per-provider deadlines, circuit breakers, and account linking by OAuth so the user's token, not ours, is sent. Device state is cached in the home graph from provider reports (push) and periodic sync, so "are the lights on" is answered from our copy, and "turn off" is sent as a command whose result updates the copy. A provider that is slow degrades to the asynchronous confirmation; one that is down gets an honest spoken error for its devices only. Provider latency and error rates are tracked per vendor and surfaced to them, because the user blames the speaker, not the bulb.

How does understanding handle ambiguity and follow-ups?

Intent and slot models produce candidates with scores; the home graph and context resolve them ("the lights" means the lights in the room this speaker is in, unless a room is named). Follow-ups use the session: "and the bedroom?" inherits the previous intent with a new slot for a short TTL. When confidence is low or two readings are plausible ("play Frozen": the film soundtrack or the audiobook), the assistant asks a one-question clarification, which is far cheaper than a wrong action. Every request logs the transcript, chosen intent and whether the user corrected it, which is the signal for improving the models, with the privacy rules above applied before any human looks at examples.

6. What a strong answer sounds like

• Set the one-second budget and spend it stage by stage, with streaming at every boundary so stages overlap.
• Put the wake word, timers and local control on the device, and explain that this is both the offline story and the privacy story.
• Describe the home graph as the thing that turns "the kitchen lights" into device commands, and keep a cached copy of device state.
• Isolate providers with contracts, deadlines, breakers and asynchronous confirmations.
• Expected follow-ups: how the device identifies who is speaking; how a global fleet gets firmware updates (staged rollouts); multi-device arbitration when two speakers hear the same wake word (nearest by signal strength decides, within 100 ms).
• Common mistake: a request-response design that uploads a complete recording and waits for a complete answer, which cannot meet the latency, or ignoring where audio goes and how long it is kept.

02Where this question was reported

• IGotAnOffer · Glassdoor-reported SWE question
• IGotAnOffer · 'X Google product (e.g. ... Google Home)'