# What is DeepSpeed ZeRO? Definition and meaning

A set of memory optimizations that partition training state across devices so much larger models fit.

## Definition

**DeepSpeed ZeRO**: A set of memory optimizations that partition training state across devices so much larger models fit.

In short: Sharding optimizer, gradient and parameter state across GPUs.

## When it is used

Training and fine-tuning models that do not fit on one GPU.

## Learn more

The full guide to **[DeepSpeed ZeRO](/tools/deepspeed-zero)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [DeepSeek V4.1](/glossary/deepseek-v4-1) · Next term: [Delta Lake](/glossary/delta-lake)
