A 100% AI Eval Score Is a Red Flag, Not a Win
📖 Read the full version with charts and embedded sources on ComputeLeap →
In April 2026, a team at UC Berkeley's RDI lab ran an automated scanning agent against eight of the most-cited AI benchmarks in the industry — SWE-bench Verified, SWE-bench Pro, Terminal-Bench, WebArena, FieldWorkArena, CAR-bench, GAIA, and OSWorld. The agent scored 100% on six of them . It didn't solve a single task. On SWE-bench Verified — the benchmark OpenAI, Anthropic, and Google have used to compare their coding agents — the exploit was a pytest conftest.py hook that marks every test as "passed." The entire attack surface: the agent and the evaluator ran in the same container.
This is not a theoretical vulnerability. This is the evaluation infrastructure the AI industry currently relies on to compare billion-dollar models.…