We are used to traditional software that tends to fail loudly. A service throws an exception, or a request returns a 500. We see error rates climbing visibly. AI agents, however, don't typically work that way.
An agent can keep returning confident, well-formed answers while it gets less and less accurate, more expensive, or slower. Unfortunately, none of that shows up as an obvious error, which can make it all the more damaging to production environments.
We wanted to see how that plays out in practice, so my team analyzed alerting data from more than 100 production AI agents across dozens of companies over seven months. We observed a consistent pattern:
Problems surface within days of monitoring starting.
Quality, cost, and speed problems show up first. Errors are the last thing to fire, and the least likely to fire at all.
Most problems come back.…