# What is Gradient descent? Definition and meaning

Gradient descent is an iterative method that lowers a loss by repeatedly moving parameters in the direction of steepest decrease.

## Definition

**Gradient descent**: Gradient descent is an iterative method that lowers a loss by repeatedly moving parameters in the direction of steepest decrease.

In short: Walk downhill, one step.

## When it is used

It trains almost every neural network and many classical machine learning models.

## Where it fits

Gradient descent belongs to **Training fundamentals** in the [Machine learning](/topics/machine-learning) topic. These are the other terms in the same group:

- **[Loss functions](/glossary/loss-functions)**: A loss function is a formula that scores how far a model's predictions are from the targets, giving training something to minimise.
- **[Regularization](/glossary/regularization)**: Any technique that limits or guides what a model learns so that it performs better on unseen data, not just on its training set.
- **[Bias-variance trade-off](/glossary/bias-variance-trade-off)**: The idea that a model's error on new data comes from being too simple (bias), too sensitive to its training sample (variance), and unavoidable noise.
- **[Evaluation metrics](/glossary/evaluation-metrics)**: Numbers that summarise how well a model's predictions match the truth, chosen to reflect what matters for the task.
- **[Hyperparameter tuning](/glossary/hyperparameter-tuning)**: The search for the training settings, such as learning rate or tree depth, that give a model its best performance on unseen data.
- **[Feature engineering](/glossary/feature-engineering)**: The craft of creating, transforming and selecting input variables so that a model can learn the pattern more easily and more reliably.
- **[Data preprocessing](/glossary/data-preprocessing)**: The steps that turn raw, messy data into a clean, consistent numeric form that a model can be trained and evaluated on without leakage.
- **[Synthetic data](/glossary/synthetic-data)**: Data produced by a model, program or simulator rather than recorded from real events, used to train, test or share machine learning systems.

## Learn more

The full guide to **[Gradient descent](/tools/gradient-descent)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [Gradient checkpointing](/glossary/gradient-checkpointing) · Next term: [Grafana](/glossary/grafana)
