What is Diffusion language models? Definition and meaning
A diffusion language model generates text by iteratively denoising a whole sequence, usually by filling in masked tokens over several steps, instead of predicting one token at a time from left to right.
Definition
Diffusion language models: A diffusion language model generates text by iteratively denoising a whole sequence, usually by filling in masked tokens over several steps, instead of predicting one token at a time from left to right.
In short: Text that sharpens all at once.
When it is used
Fast code and text generation, infilling and editing, and research into alternatives to autoregressive large language models.
Where it fits
Diffusion language models belongs to Frontier topics in the Generative AI and LLMs topic. These are the other terms in the same group:
- Test-time compute: Test-time compute is the extra computation a model spends while answering a single request, used to raise the quality of that answer.
- Prompt caching: Prompt caching reuses the already-computed internal state for an identical prompt prefix across requests, so only the new part of each prompt has to be processed.
- LLM routing: LLM routing is the practice of choosing, for each incoming request, which language model or configuration should handle it.
- Model merging: Model merging creates a new model by combining the weights of two or more existing models, usually fine-tuned from the same base, without further gradient training.
- AI watermarking and provenance: Techniques that embed a hidden, detectable signal in AI-generated content, or attach signed records of its origin, so it can later be identified.
Learn more
The full guide to Diffusion language models explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: DETR · Next term: Diffusion models