Idempotent tool calls: which tools need a key, and how to derive one

01Why retries are the normal case, not the edge case

In an agent, a tool call can be repeated for at least four reasons, and only one of them is a bug.

• The network timed out. The request may or may not have reached the service. The adapter retries, as it should for a transient failure.
• The process crashed mid-step. On restart the runtime replays the step from the last checkpoint, which includes the tool call.
• The model re-issued the call. It lost track of having made it, or the result was unclear, and it asked again with the same arguments.
• The user re-ran the task. "Try that again" after an apparent failure that in fact succeeded.

Each of these is ordinary. A system that cannot survive them is not fragile at the edges; it is fragile in the middle.

02What idempotent means, and the ambiguous timeout

An operation is idempotent if performing it once or many times produces the same final state. The word comes from mathematics, where f(f(x)) = f(x). Setting a status to active is idempotent; incrementing a counter is not.

The reason this matters so much is one specific situation: the request was sent, no response came back, and the caller does not know whether the service acted. Without idempotency the caller has two bad choices. Retry, and risk doing it twice. Do not retry, and risk never doing it. With idempotency the choice disappears: retry, and the service either does it for the first time or tells you it already did. Idempotency is what turns the ambiguous timeout from a dilemma into a retry.

03Three classes of tool

The middle row needs care. A delete by id is idempotent; a delete of "the most recent file" is not, because the second call deletes a different file. A PUT that sets a value is idempotent; a PATCH that adds to one is not. The test is whether the operation names its target and its final state absolutely.

04How an idempotency key works

1. application derives a key for this operation
   charge for order 98765 → "charge:order:98765"

2. adapter sends the key with the request
   Idempotency-Key: charge:order:98765        header, or a field in the body

3. service checks its key store
   never seen   → perform the action, store {key → result}, return result
   seen before  → do nothing, return the stored result
   seen before with different parameters → reject as a conflict

The third branch of step 3 is part of the contract in well-designed services such as payment processors: the same key with different arguments is an error, not a silent replay, because it means two different operations were given the same identity. Key stores are usually time-bounded, often around 24 hours, which is long enough to cover every retry that counts as a retry and short enough that the store does not grow without limit.

From the agent's point of view the result is simple: the adapter can retry a keyed call as many times as it likes, and the action happens at most once. The second and later calls return the first call's response, so the model sees a consistent observation.

05Deriving the key: from the operation, not the attempt

This is where most implementations go wrong, and the mistake is easy to make.

• The key identifies the operation. Two calls that should have the same effect must have the same key. "Charge order 98765" is one operation however many times it is attempted, so the key is built from the order id and the action, and nothing else.
• Never include the attempt. A key such as charge_98765_attempt_2 makes every retry a new operation, which is exactly the double charge the key was meant to prevent. The same applies to timestamps, step numbers and random components: anything that changes between attempts changes the key and defeats it.
• Never let the model generate it. A model asked to supply a key will supply a plausible string, and will not reliably supply the same one on a retry. The adapter derives the key from the arguments and the session; the model does not know the key exists.
• Include what makes the operation distinct. Two legitimate charges on the same order, say a deposit and a balance, need different keys. So the key is built from the identifiers that define the operation: session, action, target, and whichever arguments distinguish one intended action from another, canonicalised so that argument order and formatting do not matter.

def idempotency_key(session_id, tool_name, args, distinguishing=("order_id", "amount")):
    parts = [session_id, tool_name] + [str(args[k]) for k in distinguishing if k in args]
    return sha256(":".join(parts)).hexdigest()

The session id scopes the key so that the same order charged in a different session, a legitimate repeat on another day, is a different operation. A date window serves the same purpose for things like notifications: "welcome email to user 42 this week" is one operation; next week it may be another.

06Two layers of retry, two layers of protection

Repeats arrive from two directions, and each needs its own guard.

• Transport retries happen inside the adapter: the same tool call, with the same call id from the model, is sent again after a timeout. The key for these is derived once, when the call first arrives, and reused on every resend. The call id from the model's response is a fine input to it, because it is stable across the adapter's own retries.
• Model re-issues are a new call, with a new call id, that happens to have the same arguments. The call id does not help here. Two things do: the deduplication check from the termination note, which catches the repeat before execution and tells the model; and a key derived from the arguments rather than the call id, so that even if the repeat reaches the service it is recognised.

In practice the key is derived from session, tool and distinguishing arguments, which covers both directions, and the deduplication check sits in front as a cheaper first line that also keeps the model informed.

07When the service has no key support

Payment processors and most modern APIs accept a key. An internal service, a legacy endpoint or a plain SMTP relay may not. Then the adapter keeps the ledger itself.

ledger  table with a unique constraint on key
        key | status (pending, done, failed) | result | created_at

on a call
  1. derive the key
  2. INSERT key with status pending
       unique violation → someone already started this operation:
         read its row; if done, return its result; if pending, wait or report in progress
  3. perform the action
  4. UPDATE the row to done with the result   (or failed, so a retry may proceed)

The order matters: the intent is recorded before the action, so a crash between steps 3 and 4 leaves a pending row that a later retry can see and resolve, rather than a missing row that a retry would treat as never attempted. The unique constraint is what makes concurrent attempts safe; two inserts of the same key cannot both succeed.

Where the service offers a natural unique key of its own, use that as well: a ticket system that rejects a duplicate external reference, a database with a unique index on the order number. The ledger and the constraint together give at-most-once execution without the service having to cooperate.

08Why read-before-write is not enough

A common pattern is to check first: look up whether the event already exists, and create it only if not.

existing = find_event(key)
if existing:
    return existing
return calendar.create(title, start, key=key)

This is a useful optimisation and a weak guarantee. Two retries running at the same time, or two executors handling the same step, both run the lookup, both find nothing, and both create. The check and the write are not atomic. It only becomes safe when something underneath enforces uniqueness: the service's own key handling, or a unique constraint in a ledger as above, so that the second create fails or is absorbed. Keep the pre-check for the common case; rely on the constraint for correctness.

09Which tools need keys

Two notes on top of the table. First, idempotency is about repeats; it does not make an action reversible. Charging once instead of twice is still a charge, and tools in the money and message rows also belong behind the approval gate from the function-calling note. Second, a tool that is safe to retry should say so in its schema description, and a tool that is not should say that too, so the model and the adapter both know which class they are dealing with.

10Takeaways

1. Repeats are normal. Timeouts, crashes, model re-issues and user re-runs all repeat calls, and the ambiguous timeout is the case that forces the issue.
2. Reads and absolute writes are safe; creates, sends, charges and increments are not.
3. A key names the operation. The service remembers it and returns the stored result for a repeat; the same key with different arguments is a conflict.
4. Derive the key from the operation, never the attempt. No attempt numbers, no timestamps, nothing from the model.
5. Guard both directions. Transport retries reuse the key; model re-issues are caught by deduplication and by an argument-derived key.
6. No key support means your own ledger, with a unique constraint, and the intent recorded before the action.
7. Read-before-write is an optimisation, not a guarantee. Something must enforce uniqueness underneath it.
8. Idempotent is not reversible. Money and messages still need an approval gate.