Most AI spikes test the model. They should test the task. One run tells you a patch exists. Two runs tell you the task is stable enough to hand off. This post is a 90-minute protocol for the second question.
The core claim: if two independent agent runs on the same task disagree wildly, the problem is your task description, not the model. Ship-or-kill evidence should measure that disagreement, not how good one diff looks.
The hypothesis, written before you touch anything
One sentence. One falsifiable claim. Written in a file, committed before the first prompt.
spike/hypothesis.md
Hypothesis: "Add retry with backoff to the S3 uploader" is agent-shaped. Prediction: two independent runs from the same base commit change overlapping code regions and pass the same tests. Kill if: overlap < 0.15, or either run fails the existing suite.
Enter fullscreen mode
Exit fullscreen mode
If you cannot phrase the kill condition as a number, you are not spiking.β¦