Özer Cem KELAHMETKELAHMET
Glossary

What is DeepSpeed ZeRO? Definition and meaning

A set of memory optimizations that partition training state across devices so much larger models fit.

Definition

DeepSpeed ZeRO: A set of memory optimizations that partition training state across devices so much larger models fit.

In short: Sharding optimizer, gradient and parameter state across GPUs.

When it is used

Training and fine-tuning models that do not fit on one GPU.

Learn more

The full guide to DeepSpeed ZeRO explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: DeepSeek V4.1 · Next term: Delta Lake

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open