Menu

A Red Model Check Is Not a Rerun Signal: Triage, Repair Once, Then Merge
πŸ“°

A Red Model Check Is Not a Rerun Signal: Triage, Repair Once, Then Merge

DEV Community: cicdΒ·Emery LinΒ·3 days ago
#cKL0YfUp
#devtocicd#devto#dev#fixture#return
Reading 0:00
15s threshold

Rerunning a red model check until it turns green is how a merge gate becomes decoration. The fix is not a better prompt. It is a policy that decides what kind of red you have before anything is allowed to run a second time.

Below is the gate I would wire into an existing pipeline: a JSONL ledger, a five-class triage table, one bounded repair attempt, and a quarantine rule that expires. Total new code is about 120 lines. No agent framework required.

The three reds you keep mixing together

A model-backed check can fail for three unrelated reasons, and a rerun answers none of them. Your diff genuinely broke something, your fixture or prompt moved underneath the check, or the endpoint answered differently for identical input. Only one of those is a code problem.

When you hit Rerun failed jobs without evidence, you convert the second and third cases into a green check.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.