Menu

I crash-tested AI agents that issue refunds. After a crash and resume, 22 of 28 refunded twice.
📰

I crash-tested AI agents that issue refunds. After a crash and resume, 22 of 28 refunded twice.

DEV Community: typescript·Alex·7 days ago
#G2Lq7mhu
#devtotypescript#devto#dev#refund#email
Reading 0:00
15s threshold

I've been building agents that refund customers and then email them about it. Testing the happy path is easy. I wanted to know what happens when the network gets in the way at the worst moment: right after the refund has gone through, but before anyone hears back about it.

So I built a harness to find out and ran it across five frameworks. Most of it went roughly how I'd guessed. The OpenAI Agents result didn't.

The setup

The harness runs the agent's real loop (the framework's own tool calling, retries and resume path) against stateful fakes for Stripe and email. It injects a failure at the point where the side effect happens.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.