Notes
Every note: experiments that break AI agent systems on purpose, and what the measurements showed.
The Same AI Tool Error Required Two Opposite Responses
I gave a refund agent the same ambiguous failure before and after the refund happened. Always retry and never retry each failed 30 of 30 times, on opposite sides of the boundary.
Did My Prompt Make the AI Agent Refund Twice?
I reran the same lost-response experiment 90 times with three retry policies. Under this configuration, the answer was yes.
I Made an AI Agent Refund a Customer Twice
A lost MCP response turned one $200 request into $400 and exposed why a tool-call trace is not a record of what actually happened.