# What is Loss functions? Definition and meaning

A loss function is a formula that scores how far a model's predictions are from the targets, giving training something to minimise.

## Definition

**Loss functions**: A loss function is a formula that scores how far a model's predictions are from the targets, giving training something to minimise.

In short: How wrong, in one number.

## When it is used

Every supervised and self-supervised model is trained by minimising a loss.

## Where it fits

Loss functions belongs to **Training fundamentals** in the [Machine learning](/topics/machine-learning) topic. These are the other terms in the same group:

- **[Gradient descent](/glossary/gradient-descent)**: Gradient descent is an iterative method that lowers a loss by repeatedly moving parameters in the direction of steepest decrease.
- **[Regularization](/glossary/regularization)**: Any technique that limits or guides what a model learns so that it performs better on unseen data, not just on its training set.
- **[Bias-variance trade-off](/glossary/bias-variance-trade-off)**: The idea that a model's error on new data comes from being too simple (bias), too sensitive to its training sample (variance), and unavoidable noise.
- **[Evaluation metrics](/glossary/evaluation-metrics)**: Numbers that summarise how well a model's predictions match the truth, chosen to reflect what matters for the task.
- **[Hyperparameter tuning](/glossary/hyperparameter-tuning)**: The search for the training settings, such as learning rate or tree depth, that give a model its best performance on unseen data.
- **[Feature engineering](/glossary/feature-engineering)**: The craft of creating, transforming and selecting input variables so that a model can learn the pattern more easily and more reliably.
- **[Data preprocessing](/glossary/data-preprocessing)**: The steps that turn raw, messy data into a clean, consistent numeric form that a model can be trained and evaluated on without leakage.
- **[Synthetic data](/glossary/synthetic-data)**: Data produced by a model, program or simulator rather than recorded from real events, used to train, test or share machine learning systems.

## Learn more

The full guide to **[Loss functions](/tools/loss-functions)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [LoRA](/glossary/lora) · Next term: [LSTM](/glossary/lstm)
