# Data parallelism

Each device holds a full copy of the model, processes a slice of the batch, and gradients are averaged.

## What it is used for

The simplest way to scale training across GPUs.