A single green run of a model-backed test tells you almost nothing. It is one draw from a distribution you have never measured. Until you know that distribution, your merge button is a coin flip with extra steps.

Here is the fix in one sentence: sample each model-backed fixture N times, write a ledger, and let the merge rule read the ledger instead of the last exit code.

This post gives you the fixture contract, a reference sampler, a decision table, and two hooks — a cheap one locally, a strict one in CI.

Why reruns make the problem worse

Most teams handle a red model test by rerunning it. It goes green. Everyone moves on. Repeat that three times and you have trained the team to ignore the check.

There are two failure modes and they look identical in the UI. A real regression that failed once and passed on retry. Or a genuinely unstable case that passed this time and will fail after lunch.…