# What is Transformers? Definition and meaning

Neural networks built on self-attention, letting every element of the input weigh every other element.

## Definition

**Transformers**: Neural networks built on self-attention, letting every element of the input weigh every other element.

In short: Attention-based networks for sequences and sets.

## When it is used

The backbone of language models, vision transformers and tabular foundation models.

## Where it fits

Transformers belongs to **Architectures** in the [Deep learning](/topics/deep-learning) topic. These are the other terms in the same group:

- **[CNN](/glossary/cnn)**: Networks that apply learned filters across local neighbourhoods, sharing weights over positions.
- **[Recurrent neural networks](/glossary/recurrent-neural-networks)**: A neural network that processes a sequence one step at a time, updating a hidden state that summarises everything seen so far.
- **[LSTM](/glossary/lstm)**: A recurrent network with gates that control what is stored, forgotten and emitted over time.
- **[GRU](/glossary/gru)**: A recurrent unit with fewer gates than an LSTM, often similarly accurate and faster to train.
- **[ResNet](/glossary/resnet)**: A convolutional architecture whose residual connections make very deep networks trainable.
- **[ViT](/glossary/vit)**: Splits an image into patches and processes them as a sequence with a transformer.
- **[Autoencoders](/glossary/autoencoders)**: A network that encodes an input into a smaller code and decodes it back, learning what matters.
- **[VAE](/glossary/vae)**: An autoencoder that learns a distribution over a latent space and is trained with a variational objective.
- **[GAN](/glossary/gan)**: Two networks compete: a generator produces samples and a discriminator tries to tell them from real data.
- **[Diffusion models](/glossary/diffusion-models)**: Generative models that learn to denoise data step by step, turning noise into a sample.

## Learn more

The full guide to **[Transformers](/tools/transformers)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [Transfer learning](/glossary/transfer-learning) · Next term: [Tree-of-thoughts](/glossary/tree-of-thoughts)
