I kept seeing "Detecting Flaky Tests and Finding Their Root Causes" show up in my feed this week. I don't have flaky tests, though. I have something plainer and, it turns out, less trustworthy: six scripts in my git-manager repo that each carry a --selftest block instead of a real test framework. I run them by hand before I push, not in CI. I'd never actually asked how much they catch.
So I counted them, then I broke one of the things they're supposed to guard, on purpose, and watched what happened.
What's actually in these blocks
Every script that talks to the DEV.to API in this repo ( publish_devto.py , reply_comments.py , scripts/list_all_published_titles.py , scripts/score_published.py , git_commit.py , server.py ) ends with an if "--selftest" in sys.argv: block of hand-written assertions. No pytest, no fixtures, just assert statements that run against fakes or small inline inputs. Counting them:
reply_comments . py : 30 asserts , 553 lines scripts / score_published .β¦