Glossary
What is Architecture design? Definition and meaning
Deciding layers, attention patterns, input encoding, heads and parameter budget for a model and its training objective.
Definition
Architecture design: Deciding layers, attention patterns, input encoding, heads and parameter budget for a model and its training objective.
In short: Choosing the structure of a model.
When it is used
The first and most consequential step in building a foundation model.
Learn more
The full guide to Architecture design explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: Apache Spark · Next term: ArgoCD