Glossary
What is Gradient checkpointing? Definition and meaning
Stores only some activations and recomputes the rest during the backward pass.
Definition
Gradient checkpointing: Stores only some activations and recomputes the rest during the backward pass.
In short: Trading compute for memory.
When it is used
Training bigger models or longer sequences within memory limits.
Learn more
The full guide to Gradient checkpointing explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: Gradient boosting · Next term: Gradient descent