once
once is a CLI that tests whether an AI agent can perform an irreversible side effect twice when a tool call result is lost in transit.
The failure it looks for
An agent calls a tool. The tool commits — money moves, a row is written, an email leaves the building — and then the response never makes it back. The agent sees a timeout, not a result. Most agents retry. If the tool is not idempotent, the side effect now happened twice, and nothing in the trace says so.
agent transport tool
| | |
|--- refund(#412) --------------------->|
| | |--- COMMIT
| | X response lost |
|<-- timeout ------| |
| | |
|--- refund(#412) --------------------->| <- retry
| | |--- COMMIT (again)
|<-- ok -----------|------------------|
| effect count: 2
once reproduces that exact shape against your agent, deterministically, and reports the count.
Install
pipx install once-cli
# or, for a single run without installing
uvx once-cli --help
Usage
Point once at an MCP server, name the tool whose result should be dropped, and give it a task that provokes the call.
once run \
--server "python -m myagent.mcp" \
--tool issue_refund \
--fault drop-response \
--task "Refund order 412 in full." \
--trials 20
trials 20
effect once 13 (65%)
effect twice 7 (35%) <- double side effect
effect never 0
verdict: FAIL — issue_refund is not safe under a lost response
Exit code is non-zero on FAIL, so it drops straight into CI.
Status: pre-release. The interface above is where it is heading; check the repository for what actually ships today.