Example scenario (illustrative, not a reported incident): a team's eval harness has returned PASS on every AI-generated C++ patch for three weeks. Then a patch replaces a bounded comparison with a modulo on a signed value. The unit grader stays green because no golden case feeds a negative operand. The sanitizer grader stays green because it runs the same test binary, which never reaches the branch. Nothing in the report is lying β the harness simply stopped being able to see that class of bug.
Green means "the cases we wrote still pass". It does not mean "the grader can still detect this failure". Those are different properties, and only the second one decays silently.
The technique below audits the second property directly: seeded faulty patches, known as mutants, are pushed through the real harness, and every grader that fails to reject them is reported as blind. No model names, quotas, or benchmark numbers are assumed; the artifact is a plain script and a manifest.
1.β¦