Design fraud and DDoS protection for a social service

01The design, in the six-step framework

1. Requirements

A Twitter-like service is seeing fake sign-ups, credential stuffing on login, and bursts of fraudulent requests. Design the protection: tell real users from bots at the edge and in the login path, without turning away the real ones.

Functional

• Score every request for abuse using signals from the network, the client and the account; take an action: allow, challenge, throttle, or block.
• Protect the login and sign-up paths against credential stuffing and automated account creation, including step-up challenges.
• Rules that analysts can change quickly, plus models that update from fresh labels.
• Feedback: confirmed fraud and confirmed false positives flow back into rules and models.
• Out of scope: volumetric network-layer DDoS (the network provider's scrubbing), content moderation.

Non-functional

• Scale: 2 million requests a second normally, 20 million during an attack; 50,000 logins a second at peak.
• Latency: the decision adds under 5 ms p99 on normal requests, under 30 ms on login where a richer check is acceptable.
• Accuracy: false-positive rate under 0.1 percent of real users challenged, under 0.01 percent blocked; catch over 95 percent of known attack traffic.
• Availability: the protection layer must fail open for ordinary traffic and fail closed only for the login path under explicit attack mode.

2. Core entities

• RequestFeatures: IP, ASN, country, TLS fingerprint, user agent, device id, account id, endpoint, timing.
• Counter: a sliding-window count keyed by (dimension, value): requests per IP per minute, logins per account per hour, sign-ups per device per day.
• Rule: condition over features and counters, action, priority, owner, shadow flag.
• Score: request id, model version, probability of abuse, top contributing features.
• Verdict: request id, action taken, rule or model, time; the audit and feedback record.

3. API

• In-line at the edge: evaluate(features) → action, as an embedded library with a local rules snapshot and local counters.
• POST /risk/login with account, IP, device, password-check outcome; returns allow, challenge (which kind), or block. Called by the auth service.
• PUT /rules/{id} with shadow mode; GET /rules/{id}/impact showing what it would have done in the last hour.
• POST /feedback with request ids labelled fraud or legitimate (from takedowns, chargebacks, appeals).

4. High-level design

1. Every request passes the edge library: a few hundred rules over features and local rate counters, evaluated in microseconds. Most traffic is allowed; obvious abuse is blocked or throttled here.
2. Suspicious requests and all login and sign-up attempts go to the risk service, which joins global counters and a model score with the request's features and returns an action.
3. Request events (sampled for normal traffic, complete for flagged traffic) flow onto a stream; aggregators maintain global sliding-window counters per IP, account, device and ASN.
4. Labels from investigations and appeals feed rule authoring and model retraining; new rules and models are pushed to the edge and the risk service in shadow mode first.

5. Deep dives

How is a botnet distinguished from a million real users at the edge?

No single signal works; combine cheap ones. Per-IP rates catch dumb bots; a botnet spreads across 100,000 IPs at low rates each, so add per-ASN and per-country velocity against a baseline (a hosting provider's ASN sending 50× its usual traffic), TLS and HTTP fingerprints (bots share a handful of client stacks while real browsers are diverse), and behavioural cues (no cookies, no JavaScript execution, requests at machine-regular intervals). Each rule contributes points; thresholds map to actions, and the first action is a challenge (a proof-of-work or a CAPTCHA-style check), not a block, because the cost of a wrong challenge is seconds and the cost of a wrong block is a lost user. During an attack, raise the challenge threshold rather than the block threshold. All of this runs on local counters at the edge (stream aggregation provides the global view a few seconds later).

How is credential stuffing stopped without locking real accounts?

Attackers try leaked passwords across many accounts from many IPs. Signals: failed-login rate per IP and per account, the ratio of distinct accounts per IP, known-breached password matches, impossible travel for the account, and a new device. The response ladder: slow down (add latency to failures), challenge, require a second factor, and only then temporarily lock, with a notification to the real owner. Never reveal whether the account exists. Account-level counters live in the global store so a distributed attack on one account is seen as a whole. The risk service has the 30 ms budget to read these counters and run the model, which is why login scoring is not done at the edge. Sign-up abuse adds device and phone reputation and a cost (a verification step) that is cheap for a person and expensive at a million a day (abuse signals).

How do rules and models change safely, and how do we measure false positives?

Every new rule runs in shadow mode first: it records what it would have done, and the impact report shows how many requests and how many previously known-good users it would have affected. Only then is it enforced, and a kill switch disables it fleet-wide in seconds. The model is versioned and scored side by side with the live one; promotion needs better precision at the same recall on held-out labels. False positives are measured through appeals and through the rate at which challenged users complete the challenge: a real user passes 95 percent of the time, a bot almost never, so the pass rate per rule is a live estimate of its precision. Feedback loops are explicit: every verdict is stored with its reason so an analyst can trace a block to a rule and a feature.

What happens under a 20 million request a second attack?

The edge sheds first: known-bad fingerprints and ASNs are dropped before TLS where possible, the CDN absorbs static traffic, and a bloom filter of recently blocked IPs makes repeat offenders a nanosecond check. The risk service is protected by its own concurrency limit; if it is saturated, the edge falls back to rules-only decisions for ordinary endpoints (fail open) while login switches to a stricter rules-only mode that challenges everything new (fail closed by policy). Global counters degrade to local counters if the aggregator lags, with a visible staleness metric. The system's job during an attack is to keep the site up for real users at the cost of a few extra challenges, and saying that priority out loud is part of the answer.

6. What a strong answer sounds like

• Separate the cheap edge layer from the expensive scoring layer and give each a latency budget.
• List concrete signals and insist on combining them; explain why challenge comes before block.
• Give the credential-stuffing ladder and the counters it needs, and why those are global.
• Make shadow mode, kill switches and challenge pass rates the safety and measurement story.
• Follow-ups to expect: privacy limits on signals (retention windows, hashing device ids), attackers adapting (rotate fingerprints; rely on behaviour and velocity), and cost (challenges are cheap, model scoring is not; sample).

02Where this question was reported

• LeetCode Discuss · Interviewer probed different design areas to assess L5 depth (search snippet)