Menu

Run It Twenty Times Before You Pin It: Characterization Tests for Messy Repos
πŸ“°

Run It Twenty Times Before You Pin It: Characterization Tests for Messy Repos

DEV Community: pythonΒ·Dakota HuangΒ·7 days ago
#JfY1C1Hs
#devtopython#devto#dev#test#sample
Reading 0:00
15s threshold

A characterization test is only worth writing on output you can reproduce. Sample twenty runs before you pin one line.

Most "refactor safely" advice starts with "write a characterization test." That advice hides an assumption. It assumes the function returns the same value twice. Messy repos break that assumption daily.

Clocks, UUIDs, dict iteration, locale formatting, retry jitter, and float printing all leak into output. Pin that noise and your test goes red for reasons unrelated to your change. A red test you do not trust is worse than no test at all.

Below is the loop I run: sample, classify, freeze, then change one thing. The gate is a rule I call 20/3/2. Twenty runs, three hash seeds, two timezones.

Step 1 β€” Sample the entry point before touching the code

Do not import the module under test into your own process. Set PYTHONHASHSEED in a child process instead.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.