Yesterday's experiment ended with a clean verdict: LR warmup is insurance — free at the tuned learning rate, a full payout at an overheated one. Gradient clipping has the exact same folklore job description. It's the other knob people ship "for stability," the one every training-loop template includes with max_norm=1.0 and no comment. If warmup and clipping are both insurance against early-training violence, they should produce the same 2x2 table. So: same harness, same seeds, Adam at lr 1e-3 and 3e-3, with and without clip_grad_norm_(1.0) , plus an aggressive 0.3 arm, five paired seeds each.
The table that refused to rhyme
arm accuracy (5 seeds) paired Δ t wins
lr 1e-3 89.64 ±0.12 — — —
lr 1e-3 + clip 1.0 89.01 ±0.21 −0.62 −6.4 0/5
lr 3e-3 86.98 ±0.34 — — —
lr 3e-3 + clip 1.0 83.55 ±1.05 −3.43 −10.6 0/5
lr 3e-3 + clip 0.3 73.13 ±1.04 −13.85 −24.5 0/5
Warmup's table, inverted on every row.…