#Autoregressive

Fully discrete methods dominated for a few years, but language models based on continuous diffusion are making a comeback.

An introduction to diffusion language models and the research advances that underlie today's diffusion LLMs. We describe the building blocks of recent open-source models, starting from simple masking diffusion, and including techniques for iterative…