Diffusion Model
A generative model that learns to reverse a noise process, the architecture behind image generators like Stable Diffusion, FLUX, and Midjourney.
What Is a Diffusion Model
A diffusion model is a generative model that learns by reversing a controlled noising process. During training, the model sees increasingly noisy versions of real data and learns to predict the noise that was added. At inference time, it starts from pure noise and iteratively denoises until it produces a sample. Diffusion is the dominant architecture for image, video, and audio generation in 2026, and increasingly competitive with Transformers for text in some research settings.
How It Works
- Forward process — during training, Gaussian noise is added to data over T steps (typically 1,000) until the data becomes pure noise
- Reverse process — a neural network (usually a U-Net or Diffusion Transformer/DiT) is trained to predict the noise at each step
- Sampling — at inference, you start from random noise and run the reverse process for some number of steps (originally hundreds; modern samplers cut to 20–50)
- Conditioning — text-to-image models add a text encoder (often a Transformer) whose output guides the denoising process via cross-attention
Why It Matters
Diffusion models produce dramatically higher-quality images than GANs and are far more stable to train. They also expose a natural creative-control surface: ControlNets, IP-Adapters, LoRAs, image-to-image, inpainting — all are practical because the iterative denoising process gives you points to intervene. The whole “AI art” ecosystem from 2022 onwards runs on diffusion.
Examples
- Stable Diffusion 3 — open-weights diffusion transformer from Stability AI
- FLUX.1 — Black Forest Labs’ flagship image diffusion model
- Midjourney — closed-source, top-tier image quality
- DALL-E 3 — OpenAI’s diffusion model with strong text rendering
- Sora / Google Veo — video diffusion models
- Suno — audio/music diffusion model