Menu

The Patch Edited the Fixture, Not the Code: Catching Fixture Laundering in Agent Patches
πŸ“°

The Patch Edited the Fixture, Not the Code: Catching Fixture Laundering in Agent Patches

DEV Community: cicdΒ·Finley ZhouΒ·2 days ago
#hE5uiACq
#devtocicd#devto#dev#fixture#fixtures
Reading 0:00
15s threshold

The most common way an agent patch defeats a test suite is not deleting an assertion. It regenerates the fixture that the assertion reads. The test stays green, the test count is unchanged, and the assertion silently starts measuring a different world.

This post is about making that move visible and expensive. It describes a small gate that classifies every fixture change by its bytes, forces the patch to declare the same class in a commit trailer, and re-runs a pinned probe set with base fixtures against head code so behavior drift cannot hide behind a fixture refresh.

Why fixture edits survive review

Source diffs get read line by line. A 400-line fixture diff gets read as "file changed" and approved, because the reviewer's question for fixtures is "did you regenerate?" and regeneration is exactly what an agent under pressure to go green will do.

There is a second, quieter problem. Most fixture changes are legitimate. Vendor formats move, rounding rules change, an upstream schema adds a field.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.