Özer Cem KELAHMETKELAHMET
Glossary · Machine learning

What is Regularization? Definition and meaning

Any technique that limits or guides what a model learns so that it performs better on unseen data, not just on its training set.

Definition

Regularization: Any technique that limits or guides what a model learns so that it performs better on unseen data, not just on its training set.

In short: Keep models from memorising noise.

When it is used

Every kind of model, from linear regression and gradient-boosted trees to large neural networks.

Where it fits

Regularization belongs to Training fundamentals in the Machine learning topic. These are the other terms in the same group:

  • Gradient descent: Gradient descent is an iterative method that lowers a loss by repeatedly moving parameters in the direction of steepest decrease.
  • Loss functions: A loss function is a formula that scores how far a model's predictions are from the targets, giving training something to minimise.
  • Bias-variance trade-off: The idea that a model's error on new data comes from being too simple (bias), too sensitive to its training sample (variance), and unavoidable noise.
  • Evaluation metrics: Numbers that summarise how well a model's predictions match the truth, chosen to reflect what matters for the task.
  • Hyperparameter tuning: The search for the training settings, such as learning rate or tree depth, that give a model its best performance on unseen data.
  • Feature engineering: The craft of creating, transforming and selecting input variables so that a model can learn the pattern more easily and more reliably.
  • Data preprocessing: The steps that turn raw, messy data into a clean, consistent numeric form that a model can be trained and evaluated on without leakage.
  • Synthetic data: Data produced by a model, program or simulator rather than recorded from real events, used to train, test or share machine learning systems.

Learn more

The full guide to Regularization explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: Regression boards · Next term: REINFORCE

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open