Glossary · Machine learning
What is Loss functions? Definition and meaning
A loss function is a formula that scores how far a model's predictions are from the targets, giving training something to minimise.
Definition
Loss functions: A loss function is a formula that scores how far a model's predictions are from the targets, giving training something to minimise.
In short: How wrong, in one number.
When it is used
Every supervised and self-supervised model is trained by minimising a loss.
Where it fits
Loss functions belongs to Training fundamentals in the Machine learning topic. These are the other terms in the same group:
- Gradient descent: Gradient descent is an iterative method that lowers a loss by repeatedly moving parameters in the direction of steepest decrease.
- Regularization: Any technique that limits or guides what a model learns so that it performs better on unseen data, not just on its training set.
- Bias-variance trade-off: The idea that a model's error on new data comes from being too simple (bias), too sensitive to its training sample (variance), and unavoidable noise.
- Evaluation metrics: Numbers that summarise how well a model's predictions match the truth, chosen to reflect what matters for the task.
- Hyperparameter tuning: The search for the training settings, such as learning rate or tree depth, that give a model its best performance on unseen data.
- Feature engineering: The craft of creating, transforming and selecting input variables so that a model can learn the pattern more easily and more reliably.
- Data preprocessing: The steps that turn raw, messy data into a clean, consistent numeric form that a model can be trained and evaluated on without leakage.
- Synthetic data: Data produced by a model, program or simulator rather than recorded from real events, used to train, test or share machine learning systems.
Learn more
The full guide to Loss functions explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.