Have you ever had an AI agent confidently say, "Task completed successfully!" only to find out it didn't actually change the file, or swallowed an error and returned exit code 0?

We call this False Completion . It's the silent killer of AI agent reliability.

Recently, we ran a simple experiment: we gave an untrained base model ten completed trajectories from a code-execution sandbox. Four of them had demonstrably failed (the code didn't run), and six had succeeded. We asked the model to act as a judge.

The result? It answered PASS on all ten trajectories.

Not only was the evaluator optimistic, but it was a constant . And this isn't just an anecdote—it's a structural mathematical failure.

Why LLMs are structurally blind to their own mistakes

In the AI industry, the standard solution is to use a stronger LLM as a judge (LLM-as-a-Judge).…