Özer Cem KELAHMETKELAHMET
Glossary

What is GaLore? Definition and meaning

Projects gradients into a low-rank space so optimizer states take far less memory, while training all weights.

Definition

GaLore: Projects gradients into a low-rank space so optimizer states take far less memory, while training all weights.

In short: Memory-efficient training by projecting gradients.

When it is used

Pre-training or full fine-tuning when optimizer memory is the bottleneck.

Learn more

The full guide to GaLore explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: Function calling · Next term: GAN

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open