Your CI failed. Was it a flake or a real break?

Last week I watched a team re-run a red CI job three times, merge on the third green run, and ship the bug anyway. The test was not flaky. The retry just hid it for one run.

The opposite happens just as often: a genuinely flaky test blocks a good PR for an afternoon while someone scrolls a 40,000-line log looking for the first real error.

I built a small step that answers three questions before anyone re-runs anything:

Where did it actually break first?

Is each failing test flaky, or genuinely broken?

Has this exact failure happened before?

It is called CI Failure Detective , and it runs entirely inside your runner. No log ever leaves your repo.…