Özer Cem KELAHMETKELAHMET
Glossary

What is Megatron-LM? Definition and meaning

A library with tensor, pipeline and sequence parallelism that is tuned for NVIDIA GPU clusters.

Definition

Megatron-LM: A library with tensor, pipeline and sequence parallelism that is tuned for NVIDIA GPU clusters.

In short: NVIDIA's framework for training very large models.

When it is used

Large-scale pre-training across many GPUs and nodes.

Learn more

The full guide to Megatron-LM explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: Medallion architecture · Next term: Meta-learning

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open