DDIM

Overview

DDIM, short for denoising diffusion implicit model, is a way to sample from a trained diffusion model much faster. It reuses the same network that a DDPM learns, but it walks back from noise to data along a different path. That path can be deterministic and can skip most of the original steps. Where a basic DDPM sampler may need a thousand network calls, a DDIM sampler can often produce a good sample with 20 to 50.

The key point is that no retraining is needed. The training objective of a DDPM only fixes the marginal distributions at each noise level, and many different reverse processes share those marginals. DDIM picks one of them that is easy to compute and fast to run. This article explains the idea in plain terms, shows a worked example, compares DDIM with related samplers and with other generative models such as the VAE and normalizing flows, and describes how fast, repeatable sampling helps when diffusion is used around models that learn from raw tables.

What it is

DDIM is a family of non-Markovian sampling processes that share the training objective of DDPM. It was introduced as a generalisation of the DDPM reverse step.

Key properties:

• Same trained model. Any network trained with the standard noise-prediction loss can be sampled with DDIM.
• Fewer steps. You choose a short list of time points, such as every twentieth step, and jump between them.
• Controllable randomness. A parameter, often called eta, sets how much fresh noise is added at each step. With eta equal to zero, sampling is fully deterministic.
• Consistent encoding. Because the zero-noise path is deterministic, a sample can be mapped back to its starting noise and forward again. This makes the noise act like a latent code.

An everyday analogy

Think of driving from a town to a city. The DDPM way is to follow every small village road and turn at each junction, with a bit of random wandering. The DDIM way uses the same map but picks a straight highway and only checks the signposts at a few exits. You end at the same kind of place, much sooner. If you always take the same highway from the same start, you always arrive at the same spot. That is the deterministic part.

What it is not

• It is not a new model architecture.
• It is not a distillation method. No student network is trained.
• It is not guaranteed to give the same quality at extreme step counts. With very few steps, detail can degrade.

Where it came from

The diffusion approach became popular after the 2020 DDPM paper. Its main drawback was slow sampling. Researchers soon noticed that the training loss does not depend on the exact form of the reverse chain, only on the marginals at each noise level.

In 2020 the paper "Denoising Diffusion Implicit Models" by Song, Meng and Ermon used this fact to define a broader family of reverse processes. Within that family sits a deterministic sampler that skips steps. The authors showed that quality stays high with large speedups.

Later work built on the same observation:

• Continuous-time views treated sampling as solving a differential equation. DDIM turned out to be a first-order solver for the probability flow equation.
• Higher-order solvers improved quality at very low step counts.
• Latent diffusion systems such as Stable Diffusion used DDIM-style samplers as common defaults.
• Distillation and consistency approaches pushed toward one-step or few-step generation.

For a view of how this fits into wider research on generative and table models, see /model/research.

How it works

Step 1: recall what a DDPM network predicts

A trained diffusion network eps_theta takes a noisy sample x_t and a step index t, and predicts the noise eps that was mixed into the clean data. From the closed-form noising formula, this prediction also gives an estimate of the clean sample:

• x0_hat = (x_t - sqrt(1 - alpha_bar_t) * eps_hat) / sqrt(alpha_bar_t)

Here alpha_bar_t tells how much of the original signal remains at step t.

Step 2: choose a shorter list of steps

Instead of using all T steps, pick a subsequence. For example, from T = 1000 choose 50 steps: 1000, 980, 960 and so on down to 0. This list is sometimes called the sampling trajectory. The choice of spacing is a design decision: uniform spacing works well, and quadratic spacing puts more steps at low noise.

Step 3: take a DDIM update

For a move from step t to an earlier step s, the update is:

• x_s = sqrt(alpha_bar_s) * x0_hat + sqrt(1 - alpha_bar_s - sigma^2) * eps_hat + sigma * z

where z is fresh Gaussian noise and sigma controls how random the step is.

Read this in three parts:

1. The first term rebuilds a clean estimate and scales it to the signal level of step s.
2. The second term adds back the predicted noise direction at the right amount.
3. The third term adds new randomness.

When sigma is zero, the third term disappears. The path is then a smooth deterministic curve from noise to data.

Step 4: pick eta

Eta scales sigma between two extremes:

Low eta gives faster convergence in few steps. High eta gives more diversity but needs more steps for good quality.

Step 5: use the deterministic path as an encoder

With eta equal to zero, the update is an invertible map in the limit of small steps. You can run it forward to encode a data point into noise. You can then decode the noise back to nearly the same data point. This is called inversion. It enables:

• Smooth interpolation: mix two noise codes and decode the result.
• Editing: encode a sample, change the condition, decode again.
• Reproducibility: the same noise and same settings give the same sample.

An analogy: a deterministic sampler is like a recipe that always gives the same dish from the same ingredients. A stochastic one is like cooking by feel, where each plate varies a little.

Why skipping steps works

Imagine plotting how a sample moves from noise to data. In many regions the path is nearly straight. When a path is nearly straight, one long step approximates many short ones. Errors only grow where the path bends sharply. DDIM takes long steps in straight regions and relies on the network to correct the direction at each stop.

Where the limits show up

• With very few steps, the first-order update makes visible errors, such as blur or washed-out detail.
• Strong guidance can push the path into regions where the straight-line assumption fails.
• Tables with sharp categorical structure may need more steps than smooth images.

Key concepts and terms

A worked example

A numeric walk-through with one value

Take a toy schedule with alpha_bar at three time points:

• alpha_bar_3 = 0.504
• alpha_bar_1 = 0.9
• alpha_bar_0 = 1.0

Suppose at step 3 we have x_3 = 1.20 and the network predicts eps_hat = 0.40.

1. Compute the clean estimate: x0_hat = (1.20 - sqrt(1 - 0.504) * 0.40) / sqrt(0.504). - sqrt(0.496) is about 0.704, so the product is about 0.282. - sqrt(0.504) is about 0.710. - x0_hat is about (1.20 - 0.282) / 0.710, which is about 1.293.
2. Jump straight to step 1 with sigma = 0: - x_1 = sqrt(0.9) * 1.293 + sqrt(0.1) * 0.40. - sqrt(0.9) is about 0.949 and sqrt(0.1) is about 0.316. - x_1 is about 1.227 + 0.126, which is about 1.353.
3. At step 1, the network predicts a new noise value, and the process repeats until step 0.

Two network calls took us from step 3 to step 0. A DDPM would have used three.

A compact sampler in PyTorch

This sketch assumes a trained noise-prediction model with the signature model(x, t) and a tensor alpha_bar of length T. See PyTorch for the framework.

import torch

@torch.no_grad()
def ddim_sample(model, alpha_bar, shape, steps=50, eta=0.0, device="cpu"):
    T = alpha_bar.shape[0]
    # evenly spaced time points from T-1 down to 0
    times = torch.linspace(T - 1, 0, steps).long().tolist()
    x = torch.randn(shape, device=device)
    for i, t in enumerate(times):
        t_prev = times[i + 1] if i + 1 < len(times) else -1
        ab_t = alpha_bar[t]
        ab_prev = alpha_bar[t_prev] if t_prev >= 0 else torch.tensor(1.0)
        tt = torch.full((shape[0],), t, device=device)
        eps = model(x, tt)
        x0 = (x - (1 - ab_t).sqrt() * eps) / ab_t.sqrt()
        sigma = eta * ((1 - ab_prev) / (1 - ab_t) * (1 - ab_t / ab_prev)).sqrt()
        dir_xt = (1 - ab_prev - sigma ** 2).clamp(min=0).sqrt() * eps
        noise = sigma * torch.randn_like(x) if t_prev >= 0 else 0.0
        x = ab_prev.sqrt() * x0 + dir_xt + noise
    return x

Using it on a table

1. Train a diffusion model on standardised numeric columns as described in the DDPM article.
2. Call ddim_sample with 25 or 50 steps and eta equal to zero for repeatable output.
3. Undo the column transforms.
4. Check marginals, correlations and a downstream task.
5. If quality is poor, double the steps or raise eta slightly.

Interpolation example

def interpolate(model, alpha_bar, z_a, z_b, n=5, steps=50):
    outs = []
    for w in torch.linspace(0, 1, n):
        z = (1 - w) * z_a + w * z_b   # a simple mix of noise codes
        outs.append(ddim_decode(model, alpha_bar, z, steps=steps))
    return outs

Here ddim_decode is the same sampler as above, but it starts from a given noise tensor instead of drawing a new one. In practice, spherical interpolation works better than a straight mix, because Gaussian noise vectors lie near a sphere in high dimensions.

Strengths and limitations

Strengths

• Big speedups. Ten to fifty times fewer network calls is common.
• No retraining. It works on any model trained with the standard loss.
• Determinism. The same noise gives the same output, which helps testing and audits.
• Inversion and editing. Samples can be encoded and edited.
• Simple to implement. The update is a few lines of code.
• Flexible trade-off. You choose steps and eta to match your budget.

Limitations

• Quality at very low step counts. Under about ten steps, results degrade without better solvers.
• Less diversity at eta zero. Deterministic sampling can reduce variety compared with stochastic sampling at the same quality.
• Inversion is approximate. Errors accumulate, especially with strong guidance.
• Depends on the base model. A poorly trained DDPM stays poor.
• No exact likelihood. Like other diffusion samplers, it does not give an exact density. Normalizing flows do.
• Mixed-type tables need care. Discrete diffusion over categories has its own samplers, and the DDIM formulas apply directly only to the continuous part.

How it compares with alternatives

How to choose:

• If you already have a diffusion model and want speed, try DDIM first.
• If you need very few steps, try a higher-order solver or distillation.
• If you need one-pass generation from scratch with exact densities, consider flows.
• If you need a compact latent code with a learned encoder, consider a VAE.

Role in tabular foundation models

Tabular foundation models learn from raw tables. Rows are examples, columns are features of mixed type, cells may be missing, and column names may be absent or unhelpful. These models are pre-trained on large numbers of tables, often synthetic ones, and then predict on a new table through in-context learning. DDIM is not such a model. It is a sampling tool for diffusion models, and it matters wherever a diffusion model is a component of the table workflow. See /model/architecture for how table-native models treat structure, and /model/benchmarks for how they are compared.

Fast synthetic data for pre-training

Pre-training on synthetic tables needs a very large number of rows. If a diffusion model supplies some of them, sampling cost becomes a bottleneck. A sampler that needs 25 steps instead of 1000 makes it practical to generate millions of rows. Deterministic sampling also makes data generation reproducible: with a fixed seed list, the same synthetic corpus can be rebuilt. See synthetic data generation for the wider picture.

Row and column structure

A diffusion denoiser for tables treats a row as a vector, or each column as a token. DDIM does not change this. It only changes how many times the denoiser is called, so any speed gain applies directly to heavy denoisers such as attention-based ones. Models in the style of FT-Transformer benefit most, because each call is costly.

Mixed types and missing values

For numeric columns, DDIM applies as is. For categorical columns, discrete diffusion variants have their own few-step samplers. A practical design is to diffuse numeric columns with DDIM and handle categories through embeddings or a separate head. For missing cells, a mask is passed to the denoiser, and the sampler only updates unknown entries.

Imputation

Imputation by diffusion keeps observed cells fixed and fills the others. With DDIM, each completion costs only a few dozen denoiser calls, so multiple imputation becomes affordable. Two modes are useful:

• Deterministic fill: eta zero and a fixed seed give one repeatable completion.
• Stochastic fill: eta above zero gives varied completions, and their spread estimates uncertainty.

This connects to imputation networks and masked cell modeling, which also predict hidden cells.

Anomaly detection

Inversion gives a neat anomaly signal. Encode a row to noise with the deterministic path, decode it again, and measure the difference. Normal rows round-trip well. Unusual rows come back changed, or their noise codes sit in unlikely regions. This reconstruction idea parallels scoring with a VAE, and can sit beside isolation forest in an anomaly detection toolkit. Treat it as a research-grade signal that needs validation on your data.

In-context learning and continual learning

In-context learners such as TabPFN do not rely on diffusion sampling. Where diffusion is used for conditional generation from context rows, DDIM keeps inference quick. For continual learning, generated rows can be replayed to reduce forgetting. Fast samplers make replay cheaper, and determinism makes the replay set auditable.

Sector and semantic classification

When tables from different sectors are generated or augmented with a condition, such as a sector or semantic type label, DDIM with guidance can create class-specific rows quickly. The same machinery supports balancing rare classes. Always confirm that augmented rows improve accuracy on real held-out data.

Limits of the connection

• DDIM does not make a diffusion model into a predictor.
• It does not fix poor preprocessing of skewed or mixed columns.
• A table foundation model that predicts in one forward pass will still be faster at inference than any diffusion sampler.

Practical guidance and best practices

Choosing steps and eta

1. Begin with 50 steps and eta zero.
2. Compare against a 250-step run on a small batch to see how much quality you lose.
3. If quality drops, raise steps to 100 before changing anything else.
4. If outputs look too similar, raise eta to 0.2 or 0.5.
5. Record the settings with each generated data set.

Trajectory design

• Uniform spacing is a good default.
• Quadratic spacing gives more steps at low noise, which helps fine detail such as exact small counts.
• Always include the final step to zero noise.
• Keep the first step at the largest noise level used in training.

Reproducibility

• Seed the noise tensor and store the seed.
• Use the same step list and eta.
• Save the schedule, since a different alpha_bar changes every update.
• Run the model in evaluation mode and with the same numeric precision.

Validating a fast sampler

Hardware notes

• Sampling is dominated by denoiser calls, so a CUDA device helps for large batches.
• Larger batches amortise overhead. Rows are independent, so batching is easy.
• Mixed precision usually works for sampling, but check numeric stability on tables with large value ranges.

A typical workflow from training to delivery

1. Prepare the data. Clean, split and transform the table. Keep the transforms in a saved object.
2. Train the diffusion model. Use the standard noise-prediction loss and keep EMA weights.
3. Pick a sampler budget. Decide how many seconds or network calls you can spend per thousand rows.
4. Tune steps and eta on a validation slice. Use quality checks, not only visual impressions.
5. Generate in batches. Write each batch with its seed and settings to a manifest.
6. Post-process. Undo transforms, round integer columns, clip to valid ranges, and drop rows that break rules.
7. Evaluate on real data. Train a simple model on the synthetic rows and score it on real held-out rows.
8. Archive. Store the model, schedule, sampler settings and seed list together.

Debugging a DDIM sampler

Choosing a trajectory in more detail

The set of time points matters as much as their count.

• Uniform spacing: equal gaps in step index. Simple and robust.
• Quadratic spacing: gaps are small near zero noise and large near high noise. Often better for fine detail.
• Log-signal spacing: gaps are even in signal-to-noise ratio. Often a good choice when the schedule is steep at the ends.
• Hand-tuned spacing: useful when you know that certain noise levels carry the structure you care about.

For tables, a quick experiment is to compare uniform and quadratic spacing with 25 steps on a validation slice and keep the one with better correlation preservation.

Relation to other few-step ideas

• Consistency models train a network to map any point on a path to the same clean sample, which allows one or two steps. They need extra training.
• Progressive distillation halves the number of steps repeatedly by training a student to match two teacher steps in one.
• Flow matching trains a velocity field directly and samples by solving an ODE. It shares the idea of deterministic paths.
• DDIM needs none of these and is the easiest first speedup.

Working with guidance and conditions

When a condition is used, such as a class label or known columns:

• Use the same condition at every step.
• Keep the guidance scale moderate. Large scales shrink diversity and can push values out of range.
• With few steps, lower the guidance a little, because large steps amplify its effect.
• For known columns in imputation, re-insert the known values after every step so they stay correct.

Example: imputation loop with fixed observed cells

@torch.no_grad()
def ddim_impute(model, alpha_bar, x_obs, mask, steps=30, eta=0.0):
    # mask is 1 where the cell is observed, 0 where it is missing
    T = alpha_bar.shape[0]
    times = torch.linspace(T - 1, 0, steps).long().tolist()
    x = torch.randn_like(x_obs)
    for i, t in enumerate(times):
        t_prev = times[i + 1] if i + 1 < len(times) else -1
        ab_t = alpha_bar[t]
        ab_prev = alpha_bar[t_prev] if t_prev >= 0 else torch.tensor(1.0)
        # noise the known part to the current level and paste it in
        known = ab_t.sqrt() * x_obs + (1 - ab_t).sqrt() * torch.randn_like(x_obs)
        x = mask * known + (1 - mask) * x
        tt = torch.full((x.shape[0],), t)
        eps = model(x, tt)
        x0 = (x - (1 - ab_t).sqrt() * eps) / ab_t.sqrt()
        x = ab_prev.sqrt() * x0 + (1 - ab_prev).sqrt() * eps
    return mask * x_obs + (1 - mask) * x

Notes on this sketch:

• It uses eta zero for brevity, so each seed gives one completion.
• Run it with several seeds to get several completions and measure their spread.
• The last line restores the observed cells exactly.
• Columns that are categorical need a decode step after filling.

A small configuration example

sampler:
  name: ddim
  steps: 30
  eta: 0.0
  spacing: quadratic
  seed_list: seeds/example.internal.txt
output:
  path: example.internal/synthetic/batch_001.parquet
  rows_per_batch: 20000
checks:
  compare_long_run_steps: 250
  max_corr_gap: 0.05

Quick reference for settings

Common mistakes

• Using a different schedule at sampling time than at training time.
• Dropping the final step, which leaves residual noise in the output.
• Setting eta too high with few steps, which adds noise the sampler has no time to remove.
• Setting eta to zero and expecting diversity. Different seeds still differ, but each seed maps to one output.
• Treating inversion as exact. Round trips accumulate error.
• Applying continuous DDIM formulas to categorical columns without adapting them.
• Skipping the check against a long run. You will not know how much quality you lost.
• Mixing up speed and cost. A faster sampler lowers inference cost only; training cost is unchanged.
• Ignoring guidance interaction. Strong guidance with few steps can produce saturated or distorted samples.
• Forgetting to undo preprocessing such as quantile transforms before using synthetic rows.

Frequently asked questions

What does DDIM stand for?

DDIM stands for denoising diffusion implicit model. The word implicit refers to how the sampling process is defined: it relies on a deterministic mapping between noise and data rather than an explicit random step at each stage.

Is DDIM a different model from DDPM?

No. It uses the same trained network as a DDPM. Only the sampling procedure changes, which is why you can speed up an existing model without retraining it.

How many steps does DDIM need?

Common choices are between 20 and 100 steps, compared with hundreds or a thousand for the basic sampler. The right number depends on the data, the schedule and the quality you need, so test it against a long run.

What is eta in DDIM?

Eta sets how much random noise is added at each step. With eta equal to zero, the sampler is deterministic. With eta equal to one, it matches the random variance of the original DDPM reverse process.

Why is a deterministic sampler useful?

It makes results repeatable, supports interpolation between samples, and allows a sample to be encoded back into a noise code. These features help with testing, editing and audits of generated data.

Does DDIM improve sample quality?

Not by itself. It mainly saves time while keeping quality close to the slow sampler. At very low step counts quality can fall, and better solvers or distillation may help.

Can DDIM be used on tabular data?

Yes, for the continuous part of a table it applies directly. Categorical columns need discrete diffusion variants or an embedding trick. It is useful for fast synthetic rows and for imputation by repeated completion.

How does DDIM relate to the probability flow ODE?

The deterministic DDIM update can be seen as a first-order numerical step of the probability flow differential equation, which shares its noise-level distributions with the diffusion. Higher-order solvers improve on it at low step counts.

Can DDIM invert a real sample into noise?

Yes, approximately. Running the deterministic update forward gives a noise code that decodes back to something close to the original. Errors grow with strong guidance and with very coarse step sizes.

Is DDIM a tabular foundation model?

No. It is a sampler for diffusion models. It can make generative components around table models faster, such as synthetic data or imputation, but a tabular foundation model is a pre-trained network that predicts from raw tables directly.

Glossary

• Ancestral sampling: the original stochastic reverse sampling that adds noise at every step.
• Deterministic sampling: sampling where the same noise always yields the same output.
• Diffusion model: a generative model that learns to reverse gradual noising.
• Distillation: training a faster student to mimic a slower teacher.
• Eta: the knob between deterministic and stochastic DDIM.
• Guidance: steering generation toward a condition.
• Inversion: mapping data to its noise code.
• Latent code: the noise tensor that determines a deterministic sample.
• Marginal: the distribution of the data at one noise level.
• ODE solver: a numerical method for stepping a differential equation.
• Probability flow ODE: deterministic equation with the same marginals as the diffusion.
• Schedule: the list of noise levels used in training and sampling.
• Trajectory: the chosen sequence of time steps at inference.

Summary

DDIM is a sampler that lets a trained diffusion model generate data in a small number of deterministic steps. It reuses the DDPM training objective, so no retraining is needed, and it offers a simple knob, eta, to trade determinism for diversity. It speeds up generation by an order of magnitude, supports interpolation and inversion, and works best when checked against a long run. For tables, it makes synthetic data generation, multiple imputation and reconstruction-based anomaly scoring far cheaper, though it applies directly only to continuous columns. It supports, but does not replace, models that learn directly from raw tables. See also /model/research, /model/architecture and /model/benchmarks.

Related

It belongs to Architectures in my stack, under AI and machine learning. Others in the same group: