Here’s an awkward question for any codebase that’s been around a few years. Do your tests describe what the system does? Or what somebody, at some point, thought it should do?
Not the same thing. Not even close, usually. Most regression tests get written right alongside a feature, by people writing down their intentions. This function returns that. This endpoint rejects bad input. Perfectly good tests! But give a system a few years in production and it picks up a whole extra layer of behavior that nobody planned and nobody wrote down. Quirks. Weird edge cases. A default value someone picked in 2019 for reasons that are now completely lost. And I’d bet money some of your users, or some other team’s service, quietly depends on every one of them.
That’s where a lot of the really nasty regressions live. And funnily enough, there’s a kind of test built for exactly that gap. Almost nobody writes it.
You’ve probably heard of Hyrum’s Law, named after Hyrum Wright at Google.…