Two short pieces of code can both be called “gradient clipping” while doing different things. One caps each coordinate. The other measures the length of the entire gradient vector and, when necessary, rescales every coordinate by the same factor.

That distinction is easy to miss when reviewing generated code. A loop full of min and max calls looks reasonable, and its outputs may even be smaller than its inputs. Neither observation establishes that it implements global norm clipping.

Here is a complete numerical example and a small Python implementation you can run without a machine-learning framework.

Start with a vector you can inspect

Take a gradient [3, 4] and a maximum L2 norm of 2 . Its original length is 5 . For global L2 norm clipping, multiply both coordinates by 2 / 5 . The result is [1.2, 1.6] , whose length is 2 . The vector retains its direction because both coordinates receive the same positive multiplier.

Clamping each coordinate to the interval [-2, 2] instead produces [2, 2] .…