We Analyzed 10,000 GitHub Actions Runs — Here’s What Flaky Tests Actually Cost

Five findings from real CI data. The numbers are worse than you think.

March 28, 2026 · 9 min read

We looked at 10,000 workflow runs across GitHub Actions repos. Not a survey. Not opinions. Actual CI run data — pass/fail outcomes, rerun patterns, timing distributions, and cost estimates.

Here’s what the data says about flaky tests.

Finding 1: 30% of CI reruns are caused by flaky tests, not real bugs

Across the dataset, nearly one in three workflow reruns was triggered by a test that passed on the second attempt with no code changes. The failure wasn’t a real bug — it was noise.

Metric Value

Total workflow runs analyzed 10,000

Runs that were reruns 2,140 (21.4%)

Reruns caused by flaky tests (passed on retry, no code change) 642 (30% of reruns)

Share of total CI compute wasted on flaky reruns 15–25% depending on repo

Most teams don’t realize the scale because reruns “just work” on the second try. The failure disappears.…