Design a Drive-style usage quota limiter

01The design, in the six-step framework

1. Requirements

Drive gives each user 15 GB and each API project a daily call budget. Unlike a per-second rate limiter, a quota is a long-lived counter that must be accurate enough to bill and fair enough that a user is not blocked at 14.9 GB because two uploads raced.

Functional

• Track usage per subject (user, project, organisation) against a limit for each resource type: storage bytes, API calls per day, uploads per hour.
• Reserve before an expensive operation, commit or release afterwards, so a 2 GB upload that fails does not count.
• Soft limits with a grace period and warnings at 80 and 100 percent; hard limits that reject.
• Administrators change limits and grant temporary increases; usage history is viewable.
• Out of scope: pricing, payments, per-second throttling.

Non-functional

• Scale: 2 billion subjects, 500,000 quota checks a second, 100,000 usage updates a second, mostly small.
• Latency: a check under 10 ms p99 (it is on the upload path); a reservation under 50 ms.
• Accuracy: storage usage exact after a short delay; API-call usage within 1 percent; never block a subject more than 1 percent under its limit.
• Availability: 99.99 percent; a degraded limiter must prefer allowing within a bounded overshoot.

2. Core entities

• Policy: subject type, resource, limit, window (none for storage, day for calls), soft or hard, grace rules.
• Usage: subject, resource, window start, used amount, reserved amount, version.
• Reservation: reservation id, subject, resource, amount, expires at, status (held, committed, released).
• Grant: subject, resource, extra limit, valid until, reason.

3. API

• POST /quota/check with subject, resource, amount; returns allowed, remaining, and warning level. Cheap, read-mostly.
• POST /quota/reserve with subject, resource, amount, ttl; returns a reservation id or a rejection with the shortfall.
• POST /quota/reservations/{id}/commit with the final amount (may be less), and /release.
• POST /quota/consume with subject, resource, amount, idempotency key: the fire-and-forget path for API calls.
• GET /quota/usage?subject= and admin PUT /quota/policies, POST /quota/grants.

4. High-level design

1. Before an upload, Drive asks for a reservation of the file size. The quota service, which owns the subject's shard, checks used + reserved + amount against the limit, records the reservation with a TTL, and answers.
2. After the upload completes, Drive commits with the actual byte count; the reservation becomes usage. If the upload fails or the TTL passes, the reservation is released.
3. API calls do not reserve: gateways emit usage events on a stream; aggregators fold them into hot per-subject counters every second and flush to the usage store every minute.
4. A nightly reconciler recomputes storage usage from the file metadata (the source of truth) and corrects drift.

5. Deep dives

Why two paths, reserve-commit and consume, and how is each kept correct?

Storage bytes must be exact and are consumed in large chunks, so a reservation makes two concurrent 10 GB uploads against a 15 GB limit impossible: the second reservation sees used + reserved and is refused. The reservation is a row in the subject's shard, written with a version check, so two reservations racing are serialised by the store. API calls are tiny and frequent; reserving each would double the latency of every call for no benefit, so they are counted after the fact with approximate hot counters that lag by a second, and a 1 percent overshoot is accepted by the requirement. State this split as the main design decision.

How do we avoid a hot row for a project doing 10,000 calls a second?

The hot counter for a subject is itself sharded into k sub-counters (say 16) across memory nodes; a consume adds to a random sub-counter, and a check sums the 16, which is cheap because they are small integers. The per-second aggregation also batches: 10,000 events a second become 16 adds a second. Checks read a cached sum refreshed every 200 ms; the error is bounded by 200 ms of consumption, which is 2,000 calls against a daily limit of millions, well under 1 percent.

How does the system fail safe without blocking paying users?

If the quota service's shard for a subject is unreachable, callers use a cached last-known remaining value with a bounded allowance: allow up to 1 percent of the limit or 100 MB, whichever is smaller, then deny until the service returns. Reservations are idempotent on a client-supplied reservation id, so retries after a timeout do not double-reserve. Grace periods make soft limits humane: at 100 percent of storage, uploads are refused but reads and deletes work, and a 7-day grace lets a user over the limit after a plan downgrade clean up before hard enforcement.

How are windows, time zones and drift handled?

Daily API windows are UTC days unless the policy says otherwise; the usage row carries the window start, and a new window simply starts a new row, with the old one retained for history and billing. Storage has no window; it is a level, not a rate, so it must be reconciled: the nightly job sums file sizes per owner from the file metadata, compares with the usage store, and writes a correction event when they differ by more than a threshold. Drift sources are failed commits, deleted files whose release event was lost, and replication delays; the reconciler is what keeps the exact resource exact, and the replication of the usage store is synchronous within a region so a crash does not lose a committed reservation.

How do warnings and grants reach users and products?

When a check crosses 80 or 100 percent, the quota service emits a threshold event once per window (deduplicated by subject, resource, threshold and window) onto the usage stream; a notifier turns it into the email and the in-product banner. Grants are additive rows with an expiry: the effective limit is the policy limit plus the sum of live grants, computed at check time and cached with the policy for a minute, so a support agent's temporary increase takes effect within a minute without touching the usage counters at all.

6. What a strong answer sounds like

• Separate rate limiting from quota in the first minute: one is a rate over seconds, the other a level or a count over a day that is billed.
• Explain reserve, commit and release for exact resources, and after-the-fact counting for cheap ones, with the error bound.
• Shard hot counters and batch the stream; say the accuracy you give up and why it is fine.
• Describe fail-safe behaviour with a bounded allowance, and reconciliation against the source of truth.
• Follow-ups to expect: hierarchical quotas (a user inside an organisation; check both, charge both), quota for shared drives (the drive is the subject, not the uploader), and showing usage in the UI (read the usage store, not the hot counters).

02Where this question was reported

• PracHub · 'Drive-Style Usage Quota Limiter', Senior+
• PracHub · 'quota enforcement for high concurrency', Hard