# Architecture design

Deciding layers, attention patterns, input encoding, heads and parameter budget for a model and its training objective.

## What it is used for

The first and most consequential step in building a foundation model.