The 100 problems, solved / Commerce, booking and payments / 047 · Design Google Pay

Design Google Pay

Move money between accounts with exactly-once semantics, fraud checks and an audit trail.

Concepts this design leans on

01The design, in the six-step framework

1. Requirements

Functional

  • Send money from one user to another, and pay a merchant, from a linked bank account or a wallet balance.
  • Every transfer either completes fully or not at all; the user sees a clear final state.
  • Fraud and risk checks before money moves; limits per user and per day.
  • Full history and statements; an immutable audit trail for regulators.
  • Out of scope: card networks and bank rails internals, KYC onboarding, lending.

Non-functional

  • 300 million users; 100 million transactions a day; 10,000 a second at peak.
  • A payment completes in under 2 s when the bank rail responds in time.
  • Exactly-once effect on balances; no lost or duplicated transfer, ever, including during failover.
  • Strong consistency for balances and ledger; availability 99.99 percent for initiating payments.
  • Data retention of 7 years; encryption of account identifiers at rest; access logged.
Back of the envelope

108 ÷ 86,400 ≈ 1,160 transactions per second average, 10,000 at peak; each is a few ledger rows, so about 50,000 row writes a second at peak, within reach of a sharded transactional database.

Ledger: 100 M transactions × 4 entries × 200 B = 80 GB a day, 30 TB a year, 200 TB over the retention period, append-only.

Fraud scoring: 10,000 a second against a model with a 50 ms budget, so a few hundred model-serving instances with features from a low-latency store.

Bank rails: the slowest step, 300 ms to several seconds, so payments are a state machine with a pending state rather than a synchronous call.

2. Core entities

  • Account: account_id, owner, type (wallet, bank-linked, merchant), currency, state.
  • Payment: payment_id, idempotency key, payer, payee, amount, instrument, state (created, risk-checked, pending, settled, failed, reversed), created_at.
  • Ledger entry: entry_id, payment_id, account_id, debit or credit, amount, balance_after, posted_at; entries come in balanced pairs.
  • Risk decision: payment_id, score, rule hits, outcome.
  • Rail request: payment_id, rail, external reference, state, response.

3. API

  • POST /payments with payee, amount, instrument, idempotency key; returns payment_id and state. Called by the app.
  • GET /payments/{id}: state; the app polls or subscribes for pending payments.
  • GET /accounts/{id}/balance and GET /accounts/{id}/entries?cursor=…: balance and statement.
  • POST /rails/{name}/callbacks: bank rail confirmations and failures.
  • POST /payments/{id}/reverse: refunds and disputes, creating compensating entries.

4. High-level design

ClientsEdgeServicesDatascorepost / holdtransactionstatecallbackoutboxWallet appAPI gateway(auth, idempotency)Payment orchestrator(state machine)Risk serviceLedger serviceRail adaptersLedger DB(sharded by account)Payments DBBank railsPayment events(outbox)

Sending 500 from a bank-linked account to a friend's wallet:

  1. The gateway authenticates the user and checks the idempotency key; a repeat returns the existing payment.
  2. The orchestrator creates the payment in state created, calls the risk service (limits, velocity, device and model score) and records the decision; a decline ends here with a clear reason.
  3. For a bank-funded payment, the ledger service posts a hold: a pending debit on the payer's bank-linked account and a pending credit on the payee's wallet, in one transaction on the payer's shard, with the payee entry posted through a second local transaction and the pair linked by payment_id.
  4. The rail adapter requests the debit from the bank; the payment is pending and the user sees it.
  5. The bank's callback arrives; the orchestrator moves the payment to settled and the ledger finalises the entries, or on failure moves to failed and reverses the hold with compensating entries.
  6. Every state change is written with the payment and published from the outbox to notifications, statements and analytics.

5. Deep dives

How is exactly-once achieved across an app retry, a crash and a bank timeout?

FailureNaive effectGuard
App retries after a timeoutTwo paymentsIdempotency key stored on the payment row; the gateway returns the first
Orchestrator crashes after posting the holdMoney held, bank never calledThe state machine is persisted; a recovery worker resumes any payment stuck in a state past its deadline
Bank call times out after the bank debitedSecond call debits twiceThe rail request carries our reference; the adapter queries status by reference before retrying
Callback delivered twiceSettled twice, entries duplicatedTransitions are conditional on the current state; a repeat is a no-op

Exactly-once is not one mechanism but idempotency at every boundary plus a durable state machine, and the ledger's balanced entries make any slip detectable by reconciliation.

Why a double-entry ledger, and how does it handle two accounts on different shards?

A balance column that is incremented and decremented cannot prove it is right. A double-entry ledger records every movement as a debit and a credit of equal amount; balances are derived (and cached as balance_after on the latest entry), and the sum of all entries for a payment is zero by construction. The two accounts in a payment usually live on different shards. Rather than a two-phase commit, the ledger posts the payer's entry first (the money leaves) and the payee's entry second, both tagged with the payment_id, and a reconciler verifies every payment has both legs; a payee leg that failed to post is retried idempotently. The invariant is enforced by reconciliation within seconds, not by a global lock.

Where does the risk check sit and what does it cost in latency?

Risk runs before any money moves and has a 50 ms budget: rules (limits, blocked payees, velocity counters from a low-latency store) run first and short-circuit, and a model scores the remainder using precomputed features about the user and device. A payment above a threshold or with a high score is routed to a step-up (confirmation, second factor) rather than declined outright, because false declines cost more than most fraud. Decisions are stored with the features used, which is required for disputes and for training.

How are account identifiers and keys protected?

Bank account numbers and tokens are encrypted with per-tenant data keys wrapped by a key management service; the rail adapter is the only component that decrypts them, and every decryption is logged with the payment_id. Statements and support tools show masked identifiers. The ledger itself holds only internal account ids, so a database dump without the key service reveals balances but no bank details.

What does failover look like for the ledger?

Ledger shards replicate synchronously to a second zone; a write is acknowledged only after both have it, which costs a few milliseconds and guarantees that a zone loss loses nothing. The orchestrator's recovery worker handles the gap: payments that were mid-flight when a node died are found by their state and deadline and driven forward, using the rail status query to learn what the bank did. Cross-region failover is a planned operation with the rails paused, since a payment half-processed in two regions is worse than a few minutes of unavailability.

6. What a strong answer sounds like

  • Say that correctness beats latency here, and show it: synchronous replication, a persisted state machine, idempotency at every boundary.
  • Explain double-entry and reconciliation as the way to prove balances, not just store them.
  • Quote 10,000 payments a second, 50 ms for risk, and the bank rail as the slow, unreliable step that forces the pending state.
  • Expect follow-ups on reversals and disputes, on multi-currency, and on what you would do if the bank's status endpoint is also down (hold the payment pending; never guess).
  • A common mistake is to update two balance columns in one transaction and call it done; the interviewer wants to hear ledger, idempotency and reconciliation.

02Where this question was reported