# What is BPE? Definition and meaning

A tokenization algorithm that repeatedly merges the most frequent adjacent symbol pairs into new tokens.

## Definition

**BPE**: A tokenization algorithm that repeatedly merges the most frequent adjacent symbol pairs into new tokens.

In short: Byte-pair encoding.

## When it is used

The most common subword tokenizer for language models.

## Learn more

The full guide to **[BPE](/tools/bpe)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [BM25](/glossary/bm25) · Next term: [BrainScaleS](/glossary/brainscales)
