# What is FSDP? Definition and meaning

PyTorch's native method for sharding parameters, gradients and optimizer states across workers.

## Definition

**FSDP**: PyTorch's native method for sharding parameters, gradients and optimizer states across workers.

In short: Fully sharded data parallel in PyTorch.

## When it is used

Multi-GPU training of large models with native PyTorch tooling.

## Learn more

The full guide to **[FSDP](/tools/fsdp)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [Format-preserving encryption](/glossary/format-preserving-encryption) · Next term: [FT-Transformer](/glossary/ft-transformer)
