AI writes more and more of our code, and code review can't keep up. So I built BuildProof : a tool that looks at a code change and tries to prove a bug, not just guess one.
How it works
The AI reads the change and suspects a bug.
It writes a test, and that test runs on the old code AND the new code.
If the test passes on the old code and fails on the new one, that's a real behaviour change.
A separate AI "judge" decides whether that change was intended or a bug.
You only get bugs that come with a failing test as proof.
The test
BugsJS is a public dataset of real bugs from real projects (Express, ESLint, Hexo, Karma, Shields, Bower, hessian.js). I re-introduced each bug and let the hunter look at the change, with no hints.
Results
Caught 32 of 44 real bugs.
In the reverse direction (the actual fix, which is correct code), it raised a false alarm 4 times out of 44.
Cost: about $0.15 per check.
It also knows when to stay quiet: I ran it on 4 open pull requests in lodash, koa and axios.…