The 100 problems, solved / Storage and files / 004 · Design Google Photos

Design Google Photos

Back up every photo from a phone, find any of them by content, and show a memory from this day years ago.

Problem 004 of 100Storage and filesL51 Nov 2026
Concepts this design leans on

01The design, in the six-step framework

1. Requirements

Functional

  • Automatically back up every photo and video from a phone, even on a slow connection, without duplicating photos already backed up.
  • Browse a lifetime of photos by date instantly, and search them by what is in them ("beach", "dog", a person's face) and where they were taken.
  • Albums, sharing with people, and "memories": photos from this day in earlier years.
  • Out of scope: editing tools, printing, the ML models themselves (treated as services with a latency and a cost).

Non-functional

  • 1 billion users; 100 million active per day uploading 20 photos each on average, 3 MB per photo.
  • Photos are never lost (eleven nines of durability) and never shown to the wrong person.
  • The timeline scrolls at 60 frames a second: thumbnails for the visible screen in under 100 ms.
  • A photo is searchable by content within a few minutes of upload; upload itself must not wait for ML.
Back of the envelope

108 users × 20 photos × 3 MB = 6 PB of new originals per day ≈ 70 GB/s of ingest; 2 × 109 uploads/day ≈ 23,000/s average, about 100,000/s at the evening peak.

Over three years that is about 6.5 EB of originals. Each photo also gets 4 thumbnail sizes totalling about 300 KB, so thumbnails add 10 percent. Metadata: 2 × 1012 photos × 500 bytes = 1 PB, sharded.

Reads are far larger than writes: a user scrolling a year of photos pulls thousands of 20 KB thumbnails, so the thumbnail tier must serve on the order of 10 million thumbnails a second from caches and the CDN.

2. Core entities

  • User: id, quota, device list.
  • Media item: id, owner, content hash, taken-at time, upload time, GPS, camera metadata, original blob id, thumbnail blob ids, width and height.
  • Annotation: media item id, labels with confidence, face cluster ids, OCR text, embedding vector.
  • Album and Share: album id, owner, member items, grantees.
  • Device: device id, user, last backup cursor.

3. API

  • POST /media/check → body: list of content hashes from the device; response: which are already backed up (so the client skips them).
  • POST /media → upload one photo (resumable, in chunks for video); returns the media item id; the original is accepted before any processing.
  • GET /timeline?before=&limit= → media items in taken-at order with thumbnail URLs; cursor pagination.
  • GET /search?q= → items matching labels, people, places or text; ranked.
  • GET /memories → curated sets for today.
  • POST /albums, POST /albums/{id}/shares → organise and share.

4. High-level design

ClientsEdgeServicesDataHTTPSthumbnailsoriginalitem rowjobthumbslabels, vectorsPhone / web appCDN (thumbnails)Load balancerUpload serviceTimeline + search APIProcessing workers(thumbs, labels, faces)Object storage(originals, thumbs)Metadata DB(sharded by user)Processing queueSearch index+ vector index

One photo from the camera to search:

  1. The phone hashes the new photo and calls check; if the hash is known for this user, it is already backed up and nothing is sent.
  2. Otherwise it uploads the original. The upload service stores the bytes in object storage, writes the item row (hash, taken-at, GPS) on the user's metadata shard, and returns. The photo now appears in the timeline, with the phone's local thumbnail until the server one is ready.
  3. The upload service enqueues a processing job. Workers generate thumbnails, run the label and face models, and write annotations and an embedding to the search and vector indexes.
  4. Scrolling the timeline reads item rows from the metadata shard (cursor by taken-at) and fetches thumbnails through the CDN by immutable blob id.

5. Deep dives

How does the timeline scroll through 50,000 photos without stutter?

The timeline is the one query that must be fast: items for a user ordered by taken-at, paginated. Store items on the user's shard with a composite primary key (user id, taken-at descending, item id), so a page is one contiguous range read. Keep a compact index in a cache keyed by user holding only (item id, taken-at, aspect ratio) for the whole library, about 50,000 × 24 bytes = 1.2 MB, so the client can lay out the scrollbar and month headers immediately and fetch thumbnail URLs for the visible rows only. Thumbnails are immutable and named by blob id, so the CDN caches them for a year and most scrolling never reaches the origin. The client prefetches a screen ahead in the scroll direction.

How do we avoid storing the same photo twice, and when is dedup dangerous?

Per-user dedup by content hash is safe and covers the common case: the same photo re-uploaded from a second device or a restored phone. Global dedup across users (a viral meme saved by a million people) saves real storage, but the metadata must keep each user's own copy record so that deleting it for one user never affects another, and the shared blob is reference-counted and reclaimed by a delayed garbage collector. Edited photos are new content (new hash) and keep a pointer to the original. Videos are chunked, so dedup works per chunk and a 2 GB video survives a dropped connection by resuming at the next chunk.

Search by content: how does "dog on the beach" return in 300 ms over a trillion photos?

Two indexes, both per user. Labels, people and places go into an inverted index keyed by (user, term), so a query is an intersection of short posting lists within one user's shard. Free-text and visual similarity use embeddings: each photo's vector lives in a per-user vector index (tens of thousands of vectors, cheap to scan or small enough for an HNSW graph), and the query text is embedded with the same model. Ranking merges both signals with recency. The index is updated by the processing workers minutes after upload, so the requirement "searchable within minutes" is met without putting ML on the upload path. The interviewer's follow-up, "what changes if we want to run new models on all existing photos", has the same answer: a backfill job over the blob store that writes new annotations, throttled so it does not compete with live uploads.

Memories and sharing: where do the hard consistency problems hide?

Memories are a nightly batch job per user: scan taken-at ranges for this day in past years, score by faces and quality, store a small list. Sharing is where privacy risk lives: a shared album is a list of item ids plus grantees, and every thumbnail and original URL is a signed URL with a short expiry bound to the item, so leaking one URL does not expose a library. Removing a photo from a shared album must invalidate caches that hold the album's item list, which argues for short TTLs on album views and immutable, per-item thumbnails that can stay cached.

6. What a strong answer sounds like

  • Lead with the numbers: 6 PB a day in, 100,000 uploads a second at peak, and that reads of thumbnails outnumber uploads by a thousand to one.
  • Keep ML off the upload path; show the queue and workers and the "searchable within minutes" guarantee that follows.
  • Design the timeline query first (composite key by taken-at, cursor pages, CDN thumbnails); it is the feature people judge the app by.
  • Describe dedup by hash and its ownership and deletion subtleties before being asked.
  • Expect follow-ups on face clustering privacy, quota accounting, and a backfill when models improve.
  • Common mistake: one global search index for all users, which makes every query scatter across the planet.

02Where this question was reported