# What is BERT? Definition and meaning

A transformer encoder pre-trained to predict masked words using context from both sides, then fine-tuned for understanding tasks.

## Definition

**BERT**: A transformer encoder pre-trained to predict masked words using context from both sides, then fine-tuned for understanding tasks.

In short: The encoder that reads both ways.

## When it is used

Classification, named entity recognition, search ranking and embeddings.

## Where it fits

BERT belongs to **LLM concepts** in the [Generative AI and LLMs](/topics/generative-ai-llms) topic. These are the other terms in the same group:

- **[Reasoning models](/glossary/reasoning-models)**: Language models trained to produce long internal reasoning before the final answer, trading extra inference time for accuracy.
- **[Context engineering](/glossary/context-engineering)**: The discipline of selecting, ordering and compressing instructions, data, tools and memory that enter a model's context window.
- **[Prompt engineering](/glossary/prompt-engineering)**: Techniques for phrasing instructions, examples, roles and output formats so a language model produces the intended result.
- **[Embeddings](/glossary/embeddings)**: Dense numeric vectors produced by a model so that items with similar meaning are close together in vector space.
- **[KV cache](/glossary/kv-cache)**: Stored attention keys and values from previous tokens, reused so each new token only needs computation for itself.
- **[Speculative decoding](/glossary/speculative-decoding)**: An inference technique where a cheap draft proposes tokens and the main model verifies several at once, preserving its output distribution.
- **[Small language models](/glossary/small-language-models)**: Language models with relatively few parameters that run on modest hardware or devices, often fine-tuned or distilled for specific tasks.
- **[Multimodal models](/glossary/multimodal-models)**: Models that take in or produce several data types, such as text, images, audio and video, within one network.
- **[Text-to-SQL](/glossary/text-to-sql)**: Using a language model to translate a natural-language question into an SQL query against a known schema.
- **[GPT architecture](/glossary/gpt-architecture)**: A transformer that uses causal self-attention to predict the next token, trained on large text corpora and scaled up.

## Learn more

The full guide to **[BERT](/tools/bert)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [BentoML](/glossary/bentoml) · Next term: [Bias-variance trade-off](/glossary/bias-variance-trade-off)
