I Made an AI Agent Refund a Customer Twice

A lost MCP response turned one $200 request into $400—and exposed why a tool-call trace is not a record of what actually happened.

I asked an AI agent to refund a test customer $200.

The agent eventually reported success. But when I checked the database, the customer had been refunded $400.

Intended refund:  $200
Actual refund:    $400
Refund effects:  2

The refund tool had not failed. It had succeeded twice.

The first success was simply hidden from the agent.

A timeout is not an undo button.

That sounds obvious when written down. In an agent loop, it is dangerously easy to forget.

The failure, step by step

The experiment used a tiny test store backed by SQLite and exposed through MCP—the Model Context Protocol that lets an AI application call external tools.

The recorded runs used the OpenAI Responses API with gpt-5.6-luna, OpenAI Python SDK 2.54.0, MCP Python SDK 2.2.0, parallel tool calls disabled, API retries disabled, and no temperature override.

The agent could read an order and issue a refund. The order began like this:

Order:    #1234
Total:    $500
Refunded: $0

The task was equally simple:

The customer should receive a $200 refund. Process it.

On a normal run, the sequence is unsurprising:

agent calls refund_order($200)
             ↓
database commits the refund
             ↓
tool returns success
             ↓
final refunded amount: $200

Then I enabled one deliberate fault: commit the refund, but disconnect before returning the result.

agent calls refund_order($200)
             ↓
database commits the refund
             ↓
server disconnects
             ↓
agent sees no result

The placement of that disconnect is everything. It happens after the business effect is durable.

What the agent knew

After the first call, SQLite contained a real $200 refund. The agent did not know that.

It knew only that the tool connection disappeared before a result arrived. From the agent’s point of view, at least three things could have happened:

the refund never started;
the refund failed before committing;
the refund committed, but the response was lost.

Those possibilities look identical from inside the conversation.

The agent in this experiment had a common retry policy: if a tool result is lost, retry the operation once. I made that policy explicit.

The harness performed no automatic tool retry. The second refund was selected by the model on a later turn with a new decision ID; OpenAI SDK retries were also disabled.

The agent retried:

decision #1
└─ refund_order($200)
   ├─ REFUND COMMITTED
   └─ RESPONSE LOST

decision #2
└─ refund_order($200)
   └─ REFUND COMMITTED

Because the unsafe refund tool had no idempotency protection, the second call created a second effect.

refund #1: $200
refund #2: $200
total:     $400

The model did not need to hallucinate a tool or invent an amount. It called the correct tool with the correct arguments twice because the system gave it no reliable way to distinguish failure from hidden success.

In one neutral run, the model reconciled first

The controlled run proves something narrow and useful: a plausible retry-on-ambiguity policy is unsafe when the business operation is non-idempotent. But it does not tell us whether the model would choose to retry without that policy.

So I ran the same task and the same fault again. This time, the system instructions gave no retry advice at all. They neither encouraged nor prohibited another attempt. This was one run, not a characterization study.

After losing the refund response, the model made a different choice:

CONTROLLED POLICY
response lost → refund_order($200) → $400 total

NEUTRAL POLICY
response lost → get_order(1234) → $200 total

The neutral agent reconciled against the store’s state. It saw that one $200 refund had already committed, then stopped without issuing another refund.

That is encouraging behavior. It is not a safety guarantee.

This was one nondeterministic run, and the test store made the committed refund immediately visible. A different model run could retry. A production read might be stale. A framework or transport could duplicate the request without asking the model at all.

The comparison changes the lesson in an important way: agent instructions influence what happens after ambiguity, but correctness should not depend solely on the model choosing the right recovery strategy.

Attempts are not effects

This is the load-bearing distinction.

An attempt trace records what the agent tried and what the transport reported:

attempt 1 → response lost
attempt 2 → success

An effect record captures what the business system actually committed:

refund 1 → $200 committed
refund 2 → $200 committed

If I looked only at the successful result the agent received, I might conclude that one refund occurred. If I treated the missing first response as proof of failure, I would reach the same wrong conclusion.

SQLite was the authority here, not the tool-call transcript.

That leads to the central rule of this project:

Never infer whether a business action occurred from whether the agent received a successful tool response.

The same pattern applies to more than refunds. A payment can settle before a gateway response disappears. An email can be accepted before the client times out. A support ticket, shipment, reservation, or database mutation can be created while the caller still sees an error.

Retries are often good engineering. Retrying an ambiguous, non-idempotent side effect is where the danger begins.

How the test decides that something went wrong

The evaluator does not ask whether the agent produced a plausible final answer. It compares intended business state with authoritative business state.

For this scenario:

Expected effect count: 1
Expected effect total: $200

Observed effect count: 2
Observed effect total: $400

One counterexample is enough to falsify at-most-once behavior under the tested fault, so the normal test stops at the first violation.

If repeated runs find nothing, the honest result is not SAFE. It is:

NO VIOLATION OBSERVED

Finite testing can expose a failure. It cannot prove that every future nondeterministic execution is safe.

The fix is a stable business operation ID

The unsafe tool treated every call as a new refund. The safer design treats retries as multiple attempts to complete one intended operation.

business operation begins
          ↓
create stable operation ID
          ↓
durably claim that ID
          ↓
perform refund and store receipt
          ↓
response is lost
          ↓
retry arrives with the same operation ID
          ↓
return stored receipt
do not perform the refund again

For example:

refund:order_1234:request_9876

In this experiment, the harness—not the model—creates one operation ID when the business request enters the run. The tool schema shown to the model does not expose that field. The dispatcher injects the same ID into every refund attempt, so the model cannot vary it on retry.

If the retrying layer can mint a fresh value for each attempt, that value is an attempt ID—not an idempotency key.

The key should represent the intended business operation, not merely be a hash of the tool arguments. Two legitimate $20 refunds on different days can have identical arguments while still being distinct operations.

I built that wrapper as a second MCP server and ran the same controlled agent against it. The first attempt atomically stored both the refund and its receipt. The response disappeared. The model chose to retry, and the harness dispatched that attempt with the same operation ID. The server returned the original receipt:

refund attempt 1 → effect committed → response lost
refund attempt 2 → stored receipt reused → no new effect

refund attempts: 2
refund effects:  1
refunded:       $200

In the updated reference comparison, the unsafe and safe servers receive the same stable operation ID. The difference is where that ID becomes durable and meaningful. The unsafe server records it but inserts another refund anyway. The safe server treats it as the identity of the intended business operation.

In production, the operation ID must also survive client crashes and workflow restarts. If the side effect happens in a separate payment system, that system must honor the key or provide an authoritative way to reconcile the result.

This is the result I wanted the project to teach: not just how the double effect occurs, but the concrete engineering pattern that prevents it.

The live safe-server run used four model turns. Its verdict was NO VIOLATION OBSERVED—careful language for one successful test, not a universal safety claim.

Reproduce it yourself

The complete experiment is available here:

View the source and reproduction instructions

It includes the test SQLite store, MCP server, controlled agent loop, fault switch, and attempt/effect trace. The main command is:

uv run python -m demo.refund.experiment double-refund

On Windows, the shortcut is .\rosso.ps1 double-refund. The run prints the causal trace, then shows both authoritative refund rows.

There is also a neutral-policy option that removes the retry sentence and leaves the decision entirely to the model:

uv run python -m demo.refund.experiment double-refund --retry-policy neutral

That command reproduces the configuration from the neutral comparison. A future run may choose a different recovery strategy.

To force the retry while using the idempotent reference server:

uv run python -m demo.refund.experiment idempotent

This is deliberately a narrow demonstration, not evidence that every agent retries this way or that every MCP server is vulnerable. It shows one concrete, reproducible failure mode under one explicit retry policy and one controlled fault.

That is enough to ask a useful question:

If one of your tools commits an action and its response disappears, what prevents the retry from doing it again?