# What is Megatron-LM? Definition and meaning

A library with tensor, pipeline and sequence parallelism that is tuned for NVIDIA GPU clusters.

## Definition

**Megatron-LM**: A library with tensor, pipeline and sequence parallelism that is tuned for NVIDIA GPU clusters.

In short: NVIDIA's framework for training very large models.

## When it is used

Large-scale pre-training across many GPUs and nodes.

## Learn more

The full guide to **[Megatron-LM](/tools/megatron-lm)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [Medallion architecture](/glossary/medallion-architecture) · Next term: [Meta-learning](/glossary/meta-learning)
