Separate metadata from content
Small hot records in a database, big cold bytes in object storage.
01The intuition
The problem it solves
A photo studio keeps two things about every photo: a small index card (customer name, date, size, where the print is) and the print itself in a big folder. Nobody searches the folders; they search the cards, then fetch one print. If the studio glued every print to its card, the card drawer would weigh a tonne and the daily search through it would take all morning.
Systems that store files make the same split. The bytes of a file are large, change rarely and are read as a whole. The facts about the file (name, owner, size, folder, who may see it, when it changed) are tiny, change often and are queried constantly. Keeping both in the same place makes the small, hot queries pay for the big, cold bytes.
Build it one piece at a time
Step 1: a single table with the bytes inside. A naive Drive stores a row per file with a column holding the whole file. Listing a folder of 200 files reads 200 rows and drags 200 files' worth of bytes through the database. A 4 GB video sits in a row.
Step 2: move the bytes out. Put the file's bytes in object storage under a generated name, and keep only that name in the row.
| Metadata row (database) | Content (object store) |
|---|---|
| file_id 8812, name "trip.mp4", owner 42, parent folder 7, size 4.1 GB, version 3, blob_ref "b/7f3a…", modified 10:42 | key "b/7f3a…": 4.1 GB of bytes, immutable |
Listing the folder now reads 200 small rows, a few kilobytes in total. Downloading one file is one metadata read for the reference, then a direct fetch from the object store, which can be served by a CDN.
Step 3: make content immutable. A new version of the file is a new blob with a new name; the row's blob_ref is updated to point at it. Old versions keep their blobs until a cleanup job deletes them. Immutability is what makes caching and replication of content trivial: a blob never changes, so a cached copy is never wrong.
Step 4: let clients fetch content directly. The metadata service hands the client a short-lived signed URL for the blob. The bytes flow between client and object store without passing through the application servers, which stay small and stateless.
Step 5: cut big content into chunks. For a 4 GB file, the content side becomes a list of chunk references, so uploads can resume and unchanged chunks can be shared. That is the chunking article.
Why it works, with the numbers
The two halves have opposite shapes, so they want opposite storage. For a Drive-like service with 1 billion files averaging 2 MB:
metadata ≈ 1,000,000,000 × 500 bytes = 500 GB; content ≈ 1,000,000,000 × 2 MB = 2 PB, a ratio of 1 : 4,000
500 GB of hot, indexed rows fits a sharded database with plenty of room for indexes on owner and folder. 2 PB belongs in an object store at a fraction of the cost per byte, replicated and tiered. Reads split the same way: a folder listing is a metadata query answered in milliseconds; a download is a content fetch that can stream at line rate from an edge cache. The guarantee is that the small, frequent operations never wait behind the large, rare ones. The failure modes are dangling references (a row pointing at a deleted blob, or a blob nobody points at) and consistency between the two stores during an upload, which is why the row is written only after the blob is safely stored and the orphaned blobs are swept by a later job.
The trade-off you must say out loud
Two stores mean two systems to keep in step. Say the write order (blob first, then the row), say how orphans are cleaned up, and say that a file's permission check happens on the metadata side before a signed URL is issued. The interviewer's follow-up is usually "what if the database says the file exists but the blob fetch fails?", and the answer is that the reference is only written after the blob is confirmed, so that case means a storage failure, not a design hole.
Where it shows up in the 100 Google problems
- Design Google Drive: the central split; the folder tree is metadata, the file bytes are chunks in object storage.
- Design a blob / object storage service: even the object store itself keeps a metadata index separate from the data nodes.
- Design Street View image ingestion: location, heading and timestamp in a geo-indexed table; the panoramas in blob storage.
- Design deduplicated multi-tenant file storage: many metadata rows pointing at one shared content chunk is what deduplication is.