Every deep learning journey has the same plot twist: the loss curve goes flat, or worse, turns into NaN at epoch 3 like it just discovered nihilism. The framework error says nothing useful. Stack Overflow says "lower your learning rate." You lower it by 10x. Nothing changes—now it just reaches NaN with more dignity.
The first time this happened to me, I had written a custom classification loop where I manually applied torch.softmax() at the end of forward() and then fed the probabilities into torch.log() inside my own cross-entropy function. On clean toy inputs, it looked fine. On a real batch with one wildly confident misprediction (a logit gap over 104 in float32 ), the target class probability underflowed past IEEE 754 subnormals straight to 0.0 , torch.log(0.0) returned -inf , and the backward pass cheerfully multiplied -inf * 0.0 into NaN across the network.…