# What is NCCL? Definition and meaning

A library that implements fast all-reduce and other communication between GPUs.

## Definition

**NCCL**: A library that implements fast all-reduce and other communication between GPUs.

In short: NVIDIA collective communication library.

## When it is used

Multi-GPU and multi-node training.

## Where it fits

NCCL belongs to **Hardware** in the [MLOps and AI infrastructure](/topics/mlops-ai-infrastructure) topic. These are the other terms in the same group:

- **[CUDA](/glossary/cuda)**: A programming model and toolkit for running general-purpose code on NVIDIA GPUs.
- **[Cloud TPU](/glossary/cloud-tpu)**: Google's custom accelerators for machine-learning workloads.
- **[NVIDIA H100](/glossary/nvidia-h100)**: A GPU with Transformer Engine and fast interconnect, widely used for training and inference.
- **[NVIDIA H200](/glossary/nvidia-h200)**: A Hopper GPU with 141 GB of HBM3e memory.
- **[NVIDIA B200](/glossary/nvidia-b200)**: A GPU with a new architecture, larger memory and support for lower-precision formats.

## Learn more

The full guide to **[NCCL](/tools/nccl)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [NATS](/glossary/nats) · Next term: [NeMo Guardrails](/glossary/nemo-guardrails)
