Part 2 of **Shipping AI Agents to Production . Part 1 was about the second check: the eval gate judges the version, the smoke eval judges the environment.
I promised bundles, canaries and rollback for this part. Then something happened that no canary could have caught, so those move to Part 3.
A refund call timed out on a Friday evening. No deploy, no prompt change. The customer asked, "Did my refund go through?" So our support agent called the refund tool again. 9 customers were refunded twice in 14 minutes.
We already had an idempotency key on every refund. We built it from the tool call ID. The second call had a new ID, so it got a new key, and the payment provider saw a brand-new request.
The fix: one key per business intent, never per attempt. And for a tool with side effects, an error means "unknown", not "failed".…