Glossary
What is Pipeline parallelism? Definition and meaning
Assigns consecutive groups of layers to different devices and streams micro-batches through them.
Definition
Pipeline parallelism: Assigns consecutive groups of layers to different devices and streams micro-batches through them.
In short: Splitting the layers across GPUs.
When it is used
Fitting very deep models across many devices.
Learn more
The full guide to Pipeline parallelism explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: PINN · Next term: Plan-and-execute