# What is Data parallelism? Definition and meaning

Each device holds a full copy of the model, processes a slice of the batch, and gradients are averaged.

## Definition

**Data parallelism**: Each device holds a full copy of the model, processes a slice of the batch, and gradients are averaged.

In short: Copying the model and splitting the data.

## When it is used

The simplest way to scale training across GPUs.

## Learn more

The full guide to **[Data parallelism](/tools/data-parallelism)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [Data mesh](/glossary/data-mesh) · Next term: [Data preprocessing](/glossary/data-preprocessing)
