You ship a change to your agent's system prompt. You run the tests. They pass. You deploy.
Three days later, your support agent starts filing refund requests as generic support tickets. The prompts look fine. The code looks fine. Nothing broke — except the part that matters.
This is the testing gap in AI systems. Traditional unit tests are still essential, but they don't tell you whether your agent behaves correctly across the range of inputs your users will send.
You need something else: evaluation tests, or evals.
Why Unit Tests Aren't Enough
A unit test checks deterministic behavior:
expect ( parse_date ( "march 20" , today : Date . new ( 2026 , 1 , 15 ) ) ) . to eq ( "2026-03-20" )
Enter fullscreen mode
Exit fullscreen mode
Call it a thousand times, get the same result a thousand times. Any deviation is a bug. (Note the pinned today: — without it, even this test changes answer on January 1st.)
An LLM call is different. It's a probabilistic system.…