Classical learning theory tells a tidy story. As a model gets more complex, training error falls, test error first falls and then rises, and the best model sits at the bottom of a U-shaped curve. That story is not wrong. It is incomplete.

In modern machine learning, test error often falls, rises sharply to a peak, and then falls again as capacity keeps growing. This second descent is called double descent. It helps explain why very large models, which can fit their training data exactly, sometimes generalise surprisingly well, while a model just large enough to interpolate the data generalises terribly.

This article explains what double descent is, where the peak comes from, what mathematics describes it, and when you should - and should not - expect to see it.

The classical picture: bias, variance, and the U curve

Start with the familiar decomposition.…