We test guardrails by sending them bad input: an injection, a card number, a refund that's too big. That's necessary, but it only tests the guard when the guard is working. The more uncomfortable question is: what does your agent do when the guard itself breaks? It throws, its dependency is down, it gets wired into the pipeline wrong, or it runs in a code path you didn't expect.
In my experience the answer is usually "it allows the request", and nobody notices until much later. So I went through the guard layers of a .NET helpdesk agent and broke each one on purpose. Below is every place one of them wanted to fail open, and what I changed so it fails closed instead.
The stack is .NET 10 with Microsoft.Agents.AI 1.25.0 and Microsoft.Extensions.AI 10.11.0. The model in the tests is an offline fake that behaves badly on purpose: it calls tools it was never offered, loops, and ignores refusals.…