A user reports that your agent gave them wrong information. You check the logs — the tool calls look correct, the model returned a response, no exceptions were thrown. Everything ran, but the answer was bad.

This is the hardest problem in AI backend development. The agent doesn't crash when it's wrong. It confidently returns a plausible-sounding answer that happens to be incorrect, and your standard monitoring missed it entirely.

Evaluating AI output means building systems that can catch wrong answers before they reach users — or at least before they become a pattern. This article covers how to approach it practically.

Why traditional testing doesn't work

Unit tests verify deterministic behaviour. Call a function with input X, expect output Y. AI models are not deterministic — the same prompt can produce slightly different answers on different runs, and many correct answers exist for the same question.

You can't write:

assertEquals ( "24°C" , agent .…