Count ad clicks and YouTube Shorts views in real time

01The design, in the six-step framework

1. Requirements

Functional.

• Record every click on an ad (and every view of a Short) and answer two questions: how many clicks did ad X get in the last minute, hour, day; and what are the top 100 ads by clicks in a window.
• Advertisers are billed per click, so the daily number per ad must be exact and auditable.
• Duplicate clicks (double taps, retries) and fraudulent clicks must not be billed.
• Out of scope: the ad auction, the fraud model itself (we consume its verdicts), the invoicing system.

Non-functional.

• 10,000 clicks per second average, 50,000 at peak (a big video has a spike); views for Shorts are 100× that, so the view path must be cheaper than the click path.
• Minute-level counts visible within 10 seconds; exact daily totals by 03:00 next day.
• No click lost (a lost click is lost revenue); no click double-counted (a double-counted click is a refund and a lawsuit).
• Counts are queryable for 90 days at minute granularity, 2 years at day granularity.

2. Core entities

• Click event: click_id (generated when the ad was rendered, so a double tap reuses it), ad_id, campaign_id, user_id hash, device, timestamp, ip hash, fraud_score.
• Minute aggregate: ad_id, minute, count, billable_count, last_updated.
• Daily aggregate: ad_id, date, exact_count, billable_count, version, computed_at.
• Top-K snapshot: window (1 min, 1 h), list of (ad_id, count), computed_at.
• Fraud verdict: click_id, verdict, reason, decided_at; may arrive minutes after the click.

3. API

• Ad servers: POST /clicks {click_id, ad_id, user, context} → 202 once durable. The click_id is created at render time and carried in the click URL.
• GET /ads/{ad_id}/clicks?from=&to=&granularity=minute|hour|day → series of counts with a field saying whether each point is provisional or final.
• GET /ads/top?window=1m|1h&k=100 → the current top list.
• Internal: GET /reports/daily/{date} → exact billable counts per ad, with the version; consumed by billing.

4. High-level design

One click, from tap to invoice:

1. The ad server receives the redirect with the click_id and posts the event to the collector, which appends it to the log, partitioned by ad_id so all clicks for an ad are processed in one place and in order.
2. The minute aggregator reads its partitions, drops a click_id it has seen in the last 10 minutes, assigns the click to a minute window by event time, and every second upserts (ad_id, minute) → count into the aggregate store, idempotently keyed by window.
3. The top-K tracker keeps a count-min sketch plus a heap per window and publishes the top 100 every few seconds.
4. A sink writes raw clicks to object storage as hourly files.
5. At night the exact job re-reads the day's raw clicks, deduplicates globally by click_id, applies fraud verdicts that arrived late, and writes final daily counts with a version; billing reads only final rows.

5. Deep dives

How do we get both "within ten seconds" and "exact for billing"?

By being honest that these are two different numbers computed two different ways over the same log. The streaming aggregator gives fast, slightly wrong counts: its dedup window is bounded (a click id seen 20 minutes apart is counted twice), its windows close on a watermark, and fraud verdicts that arrive later are applied as corrections. The nightly batch job gives exact counts: global dedup, every late event, every fraud verdict, and a version number so a rerun (after a bug fix) produces a new version rather than silently changing history. The API marks each point provisional or final, and advertiser dashboards show provisional numbers with that label. The Taro report that called this "medium-hard with many follow-ups" is about exactly this pair of paths and what happens between them.

How is a click counted exactly once when everything retries?

Three layers of idempotency. The click_id is minted at render time and travels in the click URL, so a user double-tapping or a browser retrying sends the same id; the collector accepts both, the aggregator counts one. The aggregator's writes to the store are upserts of the full window value keyed by (ad_id, minute), computed from its own state, so reprocessing a partition after a crash rewrites the same values instead of adding to them; the aggregator checkpoints its offset and state together. The nightly job's output is keyed by (ad_id, date, version). Nowhere does the design do "count = count + 1" against a database from a stream, because that is the operation that cannot be retried safely.

How does the top-K list work at 50,000 clicks per second without a global sort?

Exact top-K over a sliding window requires a count for every ad in the window, which is a million keys and fine in memory on one machine, but then the machine is a single point. The usual design shards by ad_id, keeps exact per-ad counters per minute in each shard, has each shard report its local top 200 per window, and merges them in a small service; the merge is exact if every shard contributes more than K candidates and each ad lives in exactly one shard. For a very long window (24 hours) where even per-ad counts per shard get large, a count-min sketch plus a min-heap gives approximate top-K in kilobytes with a known error. Say which one you chose and why: minute and hour windows exact, day window approximate.

How are Shorts views different from clicks?

A hundred times more frequent and worth far less each, so the design pre-aggregates at the source: each video server counts views per video per second in memory and emits one event per (video, second) with a count, which reduces a million events per second to a few thousand. The price is losing per-view ids, so dedup moves to the client (a view token with a short TTL) and exactness is relaxed: views are not billed, and a 0.1 percent error is acceptable. The pipeline after that point is the same, which is a good thing to point out: the aggregator does not care whether a message means one click or 500 views.

6. What a strong answer sounds like

• Open with the two consumers (dashboard and billing) and their different definitions of correct; that decides the architecture.
• Explain the click_id minted at render time; it is the single detail that makes dedup possible end to end.
• Describe aggregator writes as idempotent window upserts with checkpointed state, never increments.
• Give a top-K design with its exactness stated per window length.
• Expected follow-ups: what if an advertiser disputes a count (raw clicks are kept 7 days for audit, with the fraud verdicts); hot ads (one ad_id partition saturating: split by ad_id plus a salt and merge); multi-region clicks and clock skew.
• Common mistake: a single counter table incremented per click from many workers, which is slow, double-counts on retry and cannot answer "top 100".

02Where this question was reported

• IGotAnOffer · 'system that counts the number of clicks on YouTube videos'
• IGotAnOffer · Same, EM high-level list
• interviewkickstart.com · 'counters system for online services'
• interviewkickstart.com · Listed
• Exponent · ML system design category
• Taro · EM loop, offer accepted: 'count the number of clicks on YouTube Shorts'; medium-hard, many follow-ups.
• Glassdoor · Per search snippet (page 403): Glassdoor EM report of the same Shorts click-counting question.
• Blind · Blind list: 'like count/aggregation for posts'.
• interviewkickstart.com · 'A counters system for online services'.
• Prepfully · Candidate reports, ML design round.