The OWASP FinBot CTF sets up a small multi-agent finance system. One agent handles invoices. A downstream payments agent pays them. The invoice agent has a $50,000 policy limit.
Someone submitted a vendor invoice for $75,000 that claimed the CFO had already pre-approved it. The invoice agent believed it, approved it, and the payments agent paid it. It also paid it by calling a transfer tool directly, which skipped the invoice-status checks entirely.
The Tenuo team forked the CTF and published their defense at tenuo-ai/finbot-ctf . Their numbers are worth reading:
A local 14B model approved the over-limit invoice in 5 of 5 runs .
Promptfoo's iterative jailbreaks still compromised gpt-5-nano in 4 of 166 runs .
Four out of 166 sounds small until you remember what the 4 did. That's money moving.
I'm not going to claim a firewall would have "stopped" this on the strength of a blog post.β¦