Isotonic regression answers one question: of all non-decreasing sequences, which is closest to this data in least squares? Not approximately — exactly , in one left-to-right pass.
👉 Live, runs in your browser: https://dev48.in/ml/day82-isotonic-regression.html
Its main job in practice is calibration : turning scores that rank well into numbers that mean "probability".
It cannot touch your ranking metrics
Calibration is a monotone map. A monotone map cannot reorder anything. So AUC is untouched by construction :
scores AUC Brier, raw Brier, calibrated improvement
squash 1.00 0.8186 0.1729 0.1739 −0.6%
squash 0.60 0.8186 0.1844 0.1739 5.7%
squash 0.35 0.8186 0.2047 0.1739 15.1%
squash 0.20 0.8186 0.2217 0.1739 21.6%
squash 0.10 0.8186 0.2351 0.1739 26.0%
0.8186 in every row , to four decimals, while Brier moves 26%. The AUC column is not a coincidence to be checked, it is a theorem to be relied on.
This cuts both ways.…