Menu

1 / 4

Stop Typing One Token at a Time: The Coarse-to-Fine Model That's 31 Faster

DEV Community: machinelearning·Daniel Sam Pete Thiyagu·15 days ago
#QtEYg0j4
Reading 0:00
15s threshold

Every language model you have ever used writes the same way: one token, then the next, then the next, in a strict left-to-right line. A 500-word answer takes hundreds of sequential steps, and no amount of GPU parallelism changes that — step 200 cannot start until step 199 finishes. Diffusion language models broke the line by refining all positions at once, but they pay for it: every denoising pass processes the full sequence, and cutting passes degrades quality fast.

A paper from Stanford's Hazy Research lab and Together AI, posted to arXiv on October 9, 2026, proposes a third way. Next-Concept Modeling (NCM) does not predict tokens at all. It predicts a hierarchy of discrete codes, coarse to fine: first a handful of codes that pin down the gist, then progressively more codes that fill in the detail — each level predicted fully in parallel.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.