Özer Cem KELAHMETKELAHMET
AI Simulations · Neural network basics

Diffusion models: add noise step by step, then learn to remove it

Image generators such as Stable Diffusion learn one skill: take a slightly noisy picture and guess the clean one. Chain that guess a few dozen times, starting from pure noise, and a new sample appears. Here the same process runs on points in the plane, so you can see every step.

Level AdvancedAbout 14 minRead first: Network playground, Loss functions2 challenges in the Lab

Interactive simulation. It needs JavaScript; the text below explains what it shows.

What you are looking at

The grey points are the training data. Slide Forward view to watch the forward process: each point is shrunk towards the centre by √ᾱ and blurred with noise of size √(1−ᾱ) until nothing of the shape is left. Press Generate and 260 new points start as pure noise; at every step the denoiser predicts where the clean point is and the sampler moves a little towards it. The chart on the right shows how much signal and how much noise remain at each step for the chosen schedule.

Things to try

  • Set Model smoothness to 0 and generate: nearly every sample lands exactly on a training point. A perfect denoiser memorises.
  • Raise Model smoothness to 0.25: the samples now fill in the moons between the training points, which is what a real network does.
  • Compare 10 steps with DDPM and with DDIM: the deterministic sampler holds up far better with few steps.
  • Switch to the cosine schedule and watch the chart: it destroys information more slowly at the start.
  • Try the spiral: thin, curved data needs more steps before the samples look right.

How it works in real systems

Real image models use the same equations with a neural network as the denoiser, usually a U-Net or a transformer working on a compressed latent image rather than pixels. The network is trained on millions of images to predict the noise added at a random step. At generation time samplers such as DDIM or DPM-Solver take 20 to 50 steps instead of the 1,000 used in training, and text conditioning steers each step towards the prompt. Here the denoiser is the exact best guess for 260 known points, so you can see the difference between fitting the data perfectly and generalising.

Key formulas

$$q(x_t\mid x_0)=\mathcal N\!\big(\sqrt{\bar\alpha_t}\,x_0,\ (1-\bar\alpha_t)I\big)$$

The forward process: noising to any step t in one jump.

$$\mathcal L=\mathbb E_{t,x_0,\epsilon}\big\lVert \epsilon-\epsilon_\theta(\sqrt{\bar\alpha_t}x_0+\sqrt{1-\bar\alpha_t}\,\epsilon,\ t)\big\rVert^2$$

The DDPM training loss: predict the noise that was added.

$$x_{t-1}=\sqrt{\bar\alpha_{t-1}}\,\hat x_0+\sqrt{1-\bar\alpha_{t-1}}\,\hat\epsilon$$

One deterministic DDIM step from the predicted clean point and noise.

Common mistakes

  • Judging a diffusion model only by how good samples look: check whether they copy training images.
  • Cutting the number of sampling steps without switching to a sampler designed for few steps, such as DDIM or DPM-Solver.
  • Using the linear schedule unchanged at higher resolutions: images keep information longer, so schedules are usually shifted.

Real-world numbers

  • DDPM (2020) used 1,000 noising steps with a linear β schedule from 10⁻⁴ to 0.02.
  • DDIM (2020) produced comparable samples with 10 to 50 times fewer steps.
  • Carlini et al. (2023) showed that diffusion models such as Stable Diffusion can reproduce individual training images.

Further reading

Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

Step 1 · Set up and make the data

You need Python 3.10+, PyTorch and scikit-learn.

pip install torch scikit-learn numpy
import numpy as np, torch, torch.nn as nn
from sklearn.datasets import make_moons

x, _ = make_moons(n_samples=4000, noise=0.05, random_state=0)
x0 = torch.tensor((x - x.mean(0)) / x.std(0), dtype=torch.float32)

T = 200
betas = torch.linspace(1e-4, 0.1, T)        # linear schedule
alphas = 1 - betas
abar = torch.cumprod(alphas, 0)               # signal kept after t steps
print(f'signal left at the last step: {abar[-1].sqrt():.4f}')

Step 2 · Train a network to predict the noise

A small MLP gets the noisy point and the time step, and is trained to output the noise that was added.

class Denoiser(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(nn.Linear(3, 128), nn.SiLU(), nn.Linear(128, 128), nn.SiLU(), nn.Linear(128, 2))
    def forward(self, x, t):
        return self.net(torch.cat([x, t[:, None].float() / T], 1))

model = Denoiser()
opt = torch.optim.Adam(model.parameters(), lr=1e-3)
for step in range(6000):
    i = torch.randint(0, len(x0), (256,))
    t = torch.randint(0, T, (256,))
    eps = torch.randn(256, 2)
    xt = abar[t].sqrt()[:, None] * x0[i] + (1 - abar[t]).sqrt()[:, None] * eps
    loss = ((model(xt, t) - eps) ** 2).mean()
    opt.zero_grad(); loss.backward(); opt.step()
    if step % 1000 == 0:
        print(step, round(loss.item(), 4))

Step 3 · Sample with DDPM

Start from pure noise and apply the reverse step T times.

@torch.no_grad()
def sample(n=1000):
    x = torch.randn(n, 2)
    for t in reversed(range(T)):
        tt = torch.full((n,), t)
        eps_hat = model(x, tt)
        mean = (x - betas[t] / (1 - abar[t]).sqrt() * eps_hat) / alphas[t].sqrt()
        x = mean + (betas[t].sqrt() * torch.randn_like(x) if t > 0 else 0)
    return x

s = sample()
d = torch.cdist(s, x0).min(1).values
print(f'mean distance to the nearest training point: {d.mean():.3f}')

A trained network does not copy the 4,000 points exactly: the distances stay above zero, unlike the simulation's perfect denoiser with smoothness 0. Plot s with matplotlib to see two moons.

More simulations

← Evaluation · All AI Simulations · Speculative decoding →

Last updated Nov 19, 2026
Type to search.
    ↑↓ navigate↵ open