# GaLore

Projects gradients into a low-rank space so optimizer states take far less memory, while training all weights.

## What it is used for

Pre-training or full fine-tuning when optimizer memory is the bottleneck.