# What is Word embeddings? Definition and meaning

Learned vector representations of words where similar words have similar vectors, as in Word2Vec and GloVe.

## Definition

**Word embeddings**: Learned vector representations of words where similar words have similar vectors, as in Word2Vec and GloVe.

In short: Words as points in meaning space.

## When it is used

NLP features and the conceptual basis of modern embeddings.

## Where it fits

Word embeddings belongs to **Building blocks** in the [Deep learning](/topics/deep-learning) topic. These are the other terms in the same group:

- **[Attention mechanism](/glossary/attention-mechanism)**: A mechanism that computes weighted combinations of inputs, with weights based on relevance between queries and keys.
- **[Backpropagation](/glossary/backpropagation)**: An algorithm that uses the chain rule to compute gradients of the loss with respect to every weight.
- **[Normalization layers](/glossary/normalization-layers)**: Layers that rescale activations to a standard range, stabilising and accelerating training.
- **[Dropout](/glossary/dropout)**: A regularisation technique that zeroes random activations during training to prevent co-adaptation.
- **[Optimizers](/glossary/optimizers)**: Algorithms that update model parameters from gradients, such as SGD, momentum, Adam and AdamW.
- **[U-Net](/glossary/u-net)**: A convolutional encoder-decoder with skip connections that preserves fine detail for pixel-level predictions.
- **[Knowledge distillation](/glossary/knowledge-distillation)**: Training a compact model to match the outputs of a larger model, transferring much of its capability.
- **[Pruning](/glossary/pruning)**: Removing weights, channels or layers that contribute little, to shrink and speed up a network.

## Learn more

The full guide to **[Word embeddings](/tools/word-embeddings)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [Whisper](/glossary/whisper) · Next term: [World models](/glossary/world-models)
