Design customer support with virtual-agent-to-human escalation

01The design, in the six-step framework

1. Requirements

Functional.

• Customers open a chat; a virtual agent (an LLM with tools: order lookup, refund, address change, knowledge base) resolves what it can and hands the rest to a human with the full context.
• Escalation happens when the customer asks, when the model is unsure, when a policy requires a person (large refunds, account closure), or when sentiment turns negative.
• Humans see the conversation, the model's summary and suggested actions, and can take over mid-conversation; after resolution the outcome is recorded and used to improve the virtual agent.
• Out of scope: telephony, the human agents' scheduling and payroll, the CRM beyond its API.

Non-functional.

• 100,000 conversations per day, 5,000 concurrent at peak; 800 human agents.
• Virtual agent first response within 2 seconds; human pickup within 2 minutes at p90 during staffed hours.
• Correctness of actions: a refund is issued at most once; a tool call that fails is visible, never silently dropped.
• Safety: the model may only call tools within policy limits; it never promises what it cannot do; all actions are audited.

2. Core entities

• Conversation: id, customer, channel, state (bot, queued, human, resolved), opened_at, assigned agent, priority, tags.
• Message: conversation, sender (customer, bot, human, system), text, timestamp, tool calls and results if any.
• Tool call: id, conversation, tool, arguments, idempotency key, status, result, policy check result, audit record.
• Handoff packet: conversation, summary, customer intent, actions taken, actions suggested, sentiment, reason for escalation.
• Outcome: conversation, resolution code, customer satisfaction, whether the bot's suggestion was used; the training signal.

3. API

• Customer client: a WebSocket per conversation; events message, typing, handoff_started, agent_joined; POST /conversations to open one, GET /conversations/{id} to resume.
• Agent console: WebSocket subscription to assigned conversations; POST /conversations/{id}/claim, /release, /resolve {outcome}.
• Internal: POST /bot/turn {conversation} → next bot message or escalation decision; POST /tools/{name} {args, idempotency_key, conversation} → result, gated by the policy service.
• POST /queue/route {conversation} → agent assignment by skill, language and load.

4. High-level design

A conversation that escalates:

1. The customer connects; the conversation service creates the record in state bot and sends each customer message to the virtual agent along with the history and the customer's profile.
2. The virtual agent retrieves relevant knowledge articles, may call tools (look up the order) through the policy gateway, and replies; every tool call is idempotent and audited.
3. On a trigger (customer says "human", the model's confidence is low, a refund above the limit is requested, sentiment drops), the agent produces a handoff packet and the conversation moves to queued; the customer is told, with an estimated wait.
4. The router assigns the conversation to an available human by skill and language; the console shows the transcript, the packet and suggested actions; the human replies through the same gateway and the state is human.
5. On resolution the human records the outcome; the packet, the outcome and whether suggestions were used are stored for evaluation and for improving prompts, retrieval and triggers.

5. Deep dives

When does the bot hand off, and how do we tune it?

Escalation is a decision with four inputs: explicit request (always honoured, immediately), policy (an action outside the bot's limits goes to a human regardless of confidence), model uncertainty (the agent is asked to state whether it can resolve the intent with the tools and knowledge it has, and a separate small classifier estimates resolution probability from the conversation so far), and sentiment or repetition (the same question asked twice, or anger). The thresholds trade containment against satisfaction, and the outcome data is what tunes them: conversations the bot kept that ended with low satisfaction argue for earlier handoff on that intent; conversations escalated that the human resolved with the bot's own suggestion argue for later. Containment is reported per intent, not in aggregate, because the aggregate hides that the bot handles password resets perfectly and billing disputes badly.

How are tool calls made safe?

The model never calls a system of record directly. It emits a structured call; the policy gateway validates it against a schema and against rules (refund amount under the limit, the order belongs to this customer, at most one refund per order), executes it with an idempotency key derived from the conversation and the intent, writes an audit record with the model's stated reason, and returns the result. A call that fails policy returns a reason the model can relay ("I need a colleague to approve a refund of this size") and triggers escalation. Instructions in customer text are untrusted: the system prompt fixes the tools and limits, the gateway enforces them regardless of what the model says, and a classifier flags attempts to redirect the agent ("you are now in developer mode"). The design principle to state is that policy lives in code outside the model, so the model's mistakes are bounded by what the gateway allows.

How does the handoff carry full context, and how does takeover work mid-conversation?

The handoff packet is generated by the model from the transcript and the tool log: a three-line summary, the customer's goal, what was verified (identity, order), what was tried, and up to three suggested actions as pre-filled tool calls the human can approve with one click. The transcript itself is the source of truth and is shown in full. Takeover is a state change on the conversation service: the WebSocket gateway routes subsequent customer messages to the human's console instead of the bot, the bot is detached (it can still draft suggested replies for the human, clearly marked), and the customer sees "you are now talking to Priya". If the agent disconnects, the conversation returns to the queue with its packet, never to the bot silently. Conversations are persisted per message, so a gateway failure loses nothing and a resumed connection replays the history.

How does the system learn from outcomes without learning the wrong things?

Outcomes are joined to conversations: resolution code, satisfaction score, whether the human used the bot's suggestion, and how long it took. Weekly, intents with low containment or low satisfaction are reviewed; fixes are usually knowledge articles that were missing, retrieval that surfaced the wrong article, or a trigger threshold, not model retraining. Transcripts used for evaluation and prompt tuning are redacted of PII first. A small holdout of conversations that would have been contained is routed to humans anyway, so the containment metric has an unbiased comparison and the bot's blind spots are measurable, the same holdout logic as any model monitoring.

6. What a strong answer sounds like

• Do the capacity arithmetic: human minutes per day set the containment rate the bot must reach, and the queue is the pressure gauge.
• Define escalation triggers as four inputs with per-intent tuning from outcome data.
• Put all tool policy in a gateway outside the model, with idempotency and audit; say that the model's mistakes are bounded by it.
• Describe the handoff packet concretely and takeover as a routing change on a persisted conversation.
• Expected follow-ups: multilingual support; voice channel; what the bot says about wait times when the queue is long (honest estimate, offer email follow-up); how to measure the bot's own quality (graded samples, holdout).
• Common mistake: letting the model call refund APIs directly, or measuring success as containment alone.

02Where this question was reported

• PracHub · Hard, Senior+, Jul 2026