# What is LLM routing? Definition and meaning

LLM routing is the practice of choosing, for each incoming request, which language model or configuration should handle it.

## Definition

**LLM routing**: LLM routing is the practice of choosing, for each incoming request, which language model or configuration should handle it.

In short: The right model for each job.

## When it is used

Chat products, agents, AI gateways and any system that uses more than one model and cares about cost, latency or reliability.

## Where it fits

LLM routing belongs to **Frontier topics** in the [Generative AI and LLMs](/topics/generative-ai-llms) topic. These are the other terms in the same group:

- **[Test-time compute](/glossary/test-time-compute)**: Test-time compute is the extra computation a model spends while answering a single request, used to raise the quality of that answer.
- **[Diffusion language models](/glossary/diffusion-language-models)**: A diffusion language model generates text by iteratively denoising a whole sequence, usually by filling in masked tokens over several steps, instead of predicting one token at a time from left to right.
- **[Prompt caching](/glossary/prompt-caching)**: Prompt caching reuses the already-computed internal state for an identical prompt prefix across requests, so only the new part of each prompt has to be processed.
- **[Model merging](/glossary/model-merging)**: Model merging creates a new model by combining the weights of two or more existing models, usually fine-tuned from the same base, without further gradient training.
- **[AI watermarking and provenance](/glossary/ai-watermarking-and-provenance)**: Techniques that embed a hidden, detectable signal in AI-generated content, or attach signed records of its origin, so it can later be identified.

## Learn more

The full guide to **[LLM routing](/tools/llm-routing)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [LLaVA](/glossary/llava) · Next term: [LLM-as-judge](/glossary/llm-as-judge)
