Özer Cem KELAHMETKELAHMET
Glossary · Generative AI and LLMs

What is LLM routing? Definition and meaning

LLM routing is the practice of choosing, for each incoming request, which language model or configuration should handle it.

Definition

LLM routing: LLM routing is the practice of choosing, for each incoming request, which language model or configuration should handle it.

In short: The right model for each job.

When it is used

Chat products, agents, AI gateways and any system that uses more than one model and cares about cost, latency or reliability.

Where it fits

LLM routing belongs to Frontier topics in the Generative AI and LLMs topic. These are the other terms in the same group:

  • Test-time compute: Test-time compute is the extra computation a model spends while answering a single request, used to raise the quality of that answer.
  • Diffusion language models: A diffusion language model generates text by iteratively denoising a whole sequence, usually by filling in masked tokens over several steps, instead of predicting one token at a time from left to right.
  • Prompt caching: Prompt caching reuses the already-computed internal state for an identical prompt prefix across requests, so only the new part of each prompt has to be processed.
  • Model merging: Model merging creates a new model by combining the weights of two or more existing models, usually fine-tuned from the same base, without further gradient training.
  • AI watermarking and provenance: Techniques that embed a hidden, detectable signal in AI-generated content, or attach signed records of its origin, so it can later be identified.

Learn more

The full guide to LLM routing explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: LLaVA · Next term: LLM-as-judge

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open