A flurry of recent activity in the space of continuous diffusion models for language , after a few years of relative dormancy, suggests that this approach is making something of a comeback. Fully discrete diffusion methods had largely supplanted earlier attempts to make continuous diffusion work for language, but the tide is starting to turn. In this post, I want to take a closer look at what’s going on, and why it is happening now.

The recent influx of new research in this space inspired me to write up some of my thoughts. I have written about diffusion language models before, so this mainly serves as an update to cover everything that’s happened since then. This will be a fairly subjective account – other perspectives and dissenting opinions are very welcome in the comments and elsewhere!…