# What is Learning rate schedules? Definition and meaning

A learning rate schedule is a rule that changes the optimiser's learning rate as training progresses.

## Definition

**Learning rate schedules**: A learning rate schedule is a rule that changes the optimiser's learning rate as training progresses.

In short: Fast first, careful later.

## When it is used

It is used in nearly every deep learning training run, from image classifiers to large language models.

## Where it fits

Learning rate schedules belongs to **Neural network basics** in the [Deep learning](/topics/deep-learning) topic. These are the other terms in the same group:

- **[Neural networks](/glossary/neural-networks)**: A neural network is a model made of layers of simple units that each compute a weighted sum and a non-linear function, with the weights learned from data.
- **[Activation functions](/glossary/activation-functions)**: An activation function is the non-linear function applied to a neuron's weighted sum before its output is passed on.
- **[Weight initialization](/glossary/weight-initialization)**: Weight initialization is the choice of starting values for a neural network's weights and biases before training begins.

## Learn more

The full guide to **[Learning rate schedules](/tools/learning-rate-schedules)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [Lava](/glossary/lava) · Next term: [Learning to rank](/glossary/learning-to-rank)
