Every model you have ever trained runs through the same two functions: softmax turns raw scores into probabilities, and cross-entropy scores how wrong those probabilities were. On October 22, 2026, three Google DeepMind researchers — Jonathan T. Barron, Dan Gnanapragasam, and Michael Tschannen — posted a paper arguing that every standard implementation of both functions, in JAX, PyTorch, TensorFlow, and the textbooks, contains two numerical bugs. The fix is a short rewrite using softplus , expm1 , and log1p . On the paper's grokking benchmark it takes bf16 test accuracy from 93.3% to 100%.

This is not a story about a new architecture. It is a story about arithmetic — about what happens when you add a very small number to 1.0 and the computer quietly says "that's just 1.0."

The ELI5 version

Imagine a scoreboard with 10,000 slots, and you can only write numbers with about 7 decimal digits of precision. Your model is winning by a lot: the right answer scores far above everything else.…