Versyka

Posted on Nov 11

AI-assisted

"I'd never merge a pull request a bot opened." It's the first objection we hear about automated upgrades, and it's the right instinct. The answer isn't trust, it's evidence: you need to know whether a red test was caused by the change at all. Here's the routine, by hand first.

A test goes red on a pull request. The usual responses are "rerun the pipeline" or "start debugging". Both skip a cheaper question: is this failure caused by the change at all?

Step 1: rerun only the failed test

pytest tests/test_checkout.py::test_total -x npx jest -t "total is correct"

Enter fullscreen mode

Exit fullscreen mode

Fails, then passes: flaky. A timeout, a race, a test that depends on order, real network or the current time.

Fails twice: probably real. Keep going.

Step 2: run it on main

git stash && git checkout main pytest tests/test_checkout.py::test_total

Enter fullscreen mode

Exit fullscreen mode

If it fails on main too, your change didn't break it.…