Monitor an ML model's performance in real time

01The design, in the six-step framework

1. Requirements

Functional.

• A fraud model (or any classifier) scores 5,000 transactions per second in production. Monitor its health: are inputs still shaped like the training data, are outputs behaving, and is it still accurate once the true labels arrive (chargebacks come days later)?
• Alert on drift, on a bad rollout of a new version, and on a broken feature pipeline, within minutes; measure accuracy as labels arrive.
• Compare versions side by side and slice every metric by segment (country, merchant category, device).
• Out of scope: training, the decision logic that consumes scores, the labelling process itself.

Non-functional.

• 5,000 predictions per second, each with about 200 features; 1 million labels per day arriving 1 to 30 days late.
• Drift and output alerts within 5 minutes; accuracy metrics refreshed hourly as labels land.
• Monitoring adds under 1 ms to the prediction path and never blocks it.
• Retention: feature and prediction logs 90 days (sampled beyond 30), metrics for 2 years.

2. Core entities

• Prediction record: prediction_id (the transaction id), model version, timestamp, feature vector, score, decision, segment keys.
• Label: prediction_id, outcome (fraud, legitimate), labelled_at, source.
• Reference profile: model version, per-feature distribution from training data (histograms, quantiles, missing rates), output score distribution.
• Window statistic: model version, segment, window, feature or output, distribution summary, drift score against the reference.
• Alert rule: metric, threshold, window, minimum sample size, owner.

3. API

• SDK in the serving path: log_prediction(id, version, features, score, decision, segments), appended to a local buffer and shipped asynchronously.
• POST /labels {prediction_id, outcome, labelled_at} batch → 202; from the chargeback and review systems.
• GET /models/{name}/drift?version=&segment=&from=&to= → per-feature drift scores and output distribution over time.
• GET /models/{name}/performance?version=&segment= → precision, recall, AUC, calibration, with sample sizes and label-lag coverage.
• PUT /rules/{id} → alert rule; POST /references/{version} → upload the training profile when a version is registered.

4. High-level design

Two loops, fast and slow:

1. Every prediction is logged with its full feature vector and the model version, off the request path, into a stream.
2. The fast loop, a stream job, builds per-window summaries (histograms for numeric features, frequency tables for categorical, missing-value rates, score distribution, decision rate) per version and segment, compares them with the version's reference profile and with the previous day's windows, and writes drift scores to the metrics store; alerts fire from there within one window.
3. The same stream is sunk to columnar tables for the slow loop.
4. Labels arrive days later and are joined to predictions by id in an hourly batch; precision, recall, AUC and calibration are computed per version and segment over windows whose labels are mature enough, and written to the metrics store with the label coverage.
5. Dashboards show both loops side by side; a rollout tool reads the same metrics to gate a new version.

5. Deep dives

What do we actually measure when the truth is not available yet?

Three proxies, each cheap and each catching a different failure. Input drift: for every feature, the distance between its current window distribution and the reference (population stability index or a KL-style divergence for numeric, chi-square style for categorical), plus missing-value and out-of-range rates, which is where a broken upstream pipeline shows up first (a feature suddenly 100 percent null). Output drift: the score distribution and the decision rate; a fraud model whose block rate doubles in five minutes is either seeing an attack or has broken. Prediction confidence and agreement: compare the live version's scores against a shadow version on the same traffic; divergence localises a bad rollout. None of these say the model is wrong, only that something changed, which is why alerts carry the segment and feature that moved and a link to examples.

How is accuracy computed honestly with labels that lag by days?

By never computing a metric over a window whose labels are incomplete without saying so. Each prediction window has a label coverage curve (what fraction of its predictions have a label after d days); the join reports metrics only once coverage passes a threshold (say 90 percent at 14 days) and marks earlier estimates as provisional. Labels are biased too: fraud labels come mostly from the transactions that were allowed through (blocked ones are never observed), so recall on the blocked set is unknowable without a holdout; the design keeps a small random allow-everything holdout (0.5 percent) so the metric has an unbiased sample, which is a product decision the interviewer will want stated. Sampling also keeps the 90-day log affordable.

How does a rollout get gated, and how is a bad version caught fast?

A new version is deployed in shadow first (scores logged, not used), then as a canary on a few percent of traffic. The fast loop computes the same window statistics per version, so the dashboard shows the canary's score distribution and decision rate next to the current version's on the same traffic slice, and any divergence beyond the version's expected delta (from offline evaluation) pages. The experiment platform assigns traffic and the monitoring reads the assignment, so performance is attributable once labels land. Rollback is a serving-side switch; monitoring provides the evidence. The key insight to state: a version can be a regression and still have perfect input drift metrics, so output comparison on shared traffic is mandatory for rollouts.

How does alerting avoid crying wolf?

Drift scores are noisy on small windows and small segments, so every rule has a minimum sample size, uses a baseline that includes seasonality (compare with the same window last week as well as the reference), and requires the condition to hold for two consecutive windows. Alerts are routed to the model's owner with the top contributing features and example records, and every alert has a resolution recorded (real drift, pipeline bug, expected change, false alarm), which feeds a weekly review of thresholds. Observability for a model is the same discipline as for a service: few rules, high precision, and a history of what each alert turned out to be.

6. What a strong answer sounds like

• Separate the fast loop (drift, outputs, shadow comparison, minutes) from the slow loop (labelled accuracy, days) and say what each can and cannot detect.
• Log the full feature vector with the version, asynchronously; it is what makes every later question answerable.
• Be explicit about label lag, label bias from the model's own decisions, and the holdout that fixes it.
• Gate rollouts on output comparison over shared traffic, not on drift metrics.
• Expected follow-ups: privacy of logged features; cost of the log at scale (sampling and retention); how to monitor a model whose output is text, not a score (proxy metrics, graders, sampling for human review).
• Common mistake: proposing accuracy dashboards alone, which are blind for days, or alerting on any drift score without sample-size and seasonality controls.

02Where this question was reported

• IGotAnOffer · EM low-level list (also in a Jan 2026 EM report surfaced by search)
• PracHub · Senior+, May 2026
• Taro · Listed system design question on the EM report
• Glassdoor · Per search snippet (page 403): Glassdoor EM interview Jan 2026.