Menu

I Built a Latent Diffusion Model That Fits in 5.5MB and Runs Without a GPU
πŸ“°

I Built a Latent Diffusion Model That Fits in 5.5MB and Runs Without a GPU

DEV Community: cppΒ·BhargavΒ·5 days ago
#rjiqNCsu
#devtocpp#devto#dev#diffusion#model
Reading 0:00
15s threshold

Stable Diffusion needs an A100 to feel fast. It ships with 860 million parameters, a UNet backbone, and a multi-gigabyte footprint that assumes you have real GPU infrastructure behind it.

I wanted to know how small a diffusion model could get before it stopped working at all. The answer, after a lot of ablation studies I'll show you below, is TinyDiffusion 128 : a 1.45 million parameter Diffusion Transformer that fits in 5.55MB of FP32 weights, runs in under 200ms on a plain CPU in the project's fastest benchmark configuration (more on how that number relates to the others below), and needs less than 25MB of RAM to do it.

This isn't a toy that only draws noise. It generates coherent 128x128 images from text prompts, and every TinyDiffusion number in this post is pulled directly from the project's own evaluation suite and ablation reports, not estimated or rounded up for effect.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.