What is Test-time compute? Definition and meaning
Test-time compute is the extra computation a model spends while answering a single request, used to raise the quality of that answer.
Definition
Test-time compute: Test-time compute is the extra computation a model spends while answering a single request, used to raise the quality of that answer.
In short: Let the model think longer.
When it is used
Reasoning models, maths and coding assistants, agents, and any system that trades cost and latency for accuracy.
Where it fits
Test-time compute belongs to Frontier topics in the Generative AI and LLMs topic. These are the other terms in the same group:
- Diffusion language models: A diffusion language model generates text by iteratively denoising a whole sequence, usually by filling in masked tokens over several steps, instead of predicting one token at a time from left to right.
- Prompt caching: Prompt caching reuses the already-computed internal state for an identical prompt prefix across requests, so only the new part of each prompt has to be processed.
- LLM routing: LLM routing is the practice of choosing, for each incoming request, which language model or configuration should handle it.
- Model merging: Model merging creates a new model by combining the weights of two or more existing models, usually fine-tuned from the same base, without further gradient training.
- AI watermarking and provenance: Techniques that embed a hidden, detectable signal in AI-generated content, or attach signed records of its origin, so it can later be identified.
Learn more
The full guide to Test-time compute explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: Terraform · Next term: Text-to-SQL