Most teams treat "reproducibility" as a machine learning problem. Fix the seed, pin the library versions, snapshot the model weights, and you are done.
That definition breaks the moment the prediction is a business artifact β a number someone acts on, a number you publish, a number you will later claim was right. Under that definition, reproducibility has nothing to do with seeds. It has everything to do with whether the row you are looking at today is the row that existed when the call was made.
I run a football analytics platform with a few thousand model variants writing predictions into one Postgres table. We measured our own table last week. The result was uncomfortable enough that I want to publish it.
The measurement
The table is called ai_predictions . At the time of writing it holds 46,192 rows, produced by 3,712 distinct model variants across 16,731 fixtures, spanning 2026-06-21 to 2026-11-09.β¦