# FSDP

PyTorch's native method for sharding parameters, gradients and optimizer states across workers.

## What it is used for

Multi-GPU training of large models with native PyTorch tooling.