NT-Xent — InfoNCE, the loss behind SimCLR and most embedding models — asks each view of an item to pick its partner out of a line-up. The line-up is the rest of the batch .
👉 Live, every number measured in your browser: https://dev48.in/dl/day82-contrastive-loss.html
That clause is not a footnote. It means the batch size changes the objective , not just the gradient noise.
The loss number is not comparable across batch sizes
batch negatives per row loss collapse floor log(2N−1) partner ranked first
N = 2 4 0.0084 1.0986 100%
N = 8 16 0.0087 2.7081 100%
N = 32 64 0.0379 4.1431 100%
N = 64 128 0.0817 4.8442 100%
Same encoder, same temperature, same data distribution. The loss climbs anyway, because the task got harder .…