Menu

I animated how a language model picks the next token, with real probabilities from Qwen2.5-0.5B: softmax, greedy vs sampling, temperature, top-k, top-p (7 min video) [D]

Reading 0:00
15s threshold

I made this explainer for my small channel and would like feedback from people learning this, especially if anything is wrong.

It follows one real example the whole way. I gave Qwen2.5-0.5B-Instruct "Write one sentence about the moon." twice with the same settings (temperature 1, no top-k or top-p, different seeds). Both answers start with "The", then split: "The moon, a satellite of the Earth…" vs "The lunar South Equatorial region…".

After "The", the model scores all 151,936 vocabulary tokens. Softmax gives moon 78.04%, full 4.46%, vast 2.24%, Moon 1.51%, lunar 1.50%.

Greedy always takes moon. Sampling is a weighted random pick, so lunar wins about 1 time in 67, and run 2 got it.

The loop: append the token, run the model again.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.