Glossary
What is Mixture-of-Experts? Definition and meaning
A layer with many expert networks where a router activates only a few per token, growing capacity without growing compute.
Definition
Mixture-of-Experts: A layer with many expert networks where a router activates only a few per token, growing capacity without growing compute.
In short: Route each input to a few specialist sub-networks.
When it is used
Scaling model capacity efficiently in large language and foundation models.
Learn more
The full guide to Mixture-of-Experts explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: Mixed precision · Next term: MLflow