Regularization

Overview

Regularisation (often spelt regularization) is the family of techniques that stop a model from fitting its training data too closely. A flexible model can memorise every quirk of the examples it sees, including noise and coincidences. It then scores brilliantly on training data and poorly on anything new. Regularisation adds a gentle pressure towards simpler, smoother or more cautious solutions, so that the model keeps the real pattern and drops the accidents.

The idea appears in many forms. Some methods add a penalty to the loss, such as L1 and L2 penalties on the weights. Some change the training procedure, such as early stopping or dropout. Some change the data, such as augmentation and label smoothing. Some change the model structure, such as limiting tree depth. They all share one aim: to trade a little accuracy on the training set for a lot more accuracy on data the model has never seen. That trade is the practical face of the bias-variance trade-off.

An everyday analogy: a student preparing for an exam can memorise last year's answers word for word, or learn the method behind them. Memorising works perfectly on last year's paper and fails on a new one. A good teacher regularises the student: they change the numbers in practice questions, set time limits and ask for explanations, so that only real understanding survives. Regularisation does the same for a model.

What it is

There are two broad views of what regularisation does. The first is explicit: you write a penalty term into the objective, and the optimiser balances fit against simplicity. The second is implicit: the procedure itself, such as stopping early or adding noise, steers the model towards simpler solutions without a written penalty. In deep learning, much of the regularisation is implicit, including effects that come from the optimiser and the batch size.

Every regularisation method has a strength setting, such as the penalty weight λ, the dropout rate or the maximum depth. Too little and the model overfits. Too much and it underfits. Choosing that strength is a central part of hyperparameter tuning.

Where it came from

The mathematical root is in the solution of ill-posed problems. Andrey Tikhonov worked on stabilising such problems by adding a penalty on the size of the solution, which is why L2 regularisation is also called Tikhonov regularisation. In statistics, Arthur Hoerl and Robert Kennard published "Ridge regression: biased estimation for nonorthogonal problems" in Technometrics in 1970. They showed that accepting a small bias in the coefficients can greatly reduce their variance when features are correlated.

Robert Tibshirani introduced the lasso in 1996 in the Journal of the Royal Statistical Society, Series B, in a paper titled "Regression shrinkage and selection via the lasso". The L1 penalty both shrinks coefficients and sets some of them exactly to zero, so it performs feature selection. Hui Zou and Trevor Hastie combined L1 and L2 into the elastic net in 2005, in the same journal, to handle groups of correlated features better than the lasso alone.

In neural networks, weight decay has been used since the early days of backpropagation. Dropout was proposed by Geoffrey Hinton's group in 2012 and published in full by Nitish Srivastava and colleagues in the Journal of Machine Learning Research in 2014; that paper also recommended max-norm constraints alongside dropout. Label smoothing was introduced by Christian Szegedy and colleagues in "Rethinking the Inception Architecture for Computer Vision" (arXiv 2015, CVPR 2016), and studied in depth by Rafael Müller, Simon Kornblith and Geoffrey Hinton in "When does label smoothing help?" at NeurIPS 2019. Mixup was introduced by Hongyi Zhang and colleagues at ICLR 2018.

For adaptive optimisers, Ilya Loshchilov and Frank Hutter showed in "Decoupled Weight Decay Regularization" (arXiv 2017, ICLR 2019) that L2 regularisation and weight decay are not the same thing inside Adam. Their fix, AdamW, is now widely used for training transformers. In boosted trees, Jerome Friedman's 2001 paper "Greedy function approximation: a gradient boosting machine" introduced shrinkage, the learning rate that scales each new tree.

How it works

Penalties on the loss

The most common form adds a penalty to the training loss. The optimiser now minimises "error plus complexity". The key formulas are below.

L2 (ridge):    J(w) = L(w) + λ · Σ wᵢ²
L1 (lasso):    J(w) = L(w) + λ · Σ |wᵢ|
Elastic net:   J(w) = L(w) + λ · ( α · Σ |wᵢ| + (1 − α)/2 · Σ wᵢ² )
Weight decay:  w ← w − η · ∇L(w) − η · λ · w
AdamW step:    w ← w − η · ( m̂ / (√v̂ + ε) + λ · w )
Label smooth:  y_smooth = (1 − ε) · y_onehot + ε / K

Here J is the regularised objective, L is the ordinary loss (for example squared error or cross-entropy), w is the vector of weights and wᵢ one weight, λ is the regularisation strength, α in the elastic net sets the mix between L1 and L2, η is the learning rate, ∇L(w) is the gradient of the loss, m̂ and v̂ are Adam's bias-corrected running averages of the gradient and squared gradient, ε in the AdamW line is a tiny constant for numerical safety, ε in the label-smoothing line is the smoothing amount, and K is the number of classes. Biases and normalisation parameters are usually left out of the penalty.

Why L2 shrinks and L1 selects

L2 punishes large weights much more than small ones, because the penalty grows with the square. Its pull on a weight is proportional to the weight itself, so it pushes weights towards zero smoothly but rarely all the way. Correlated features end up sharing the credit. L1 pulls every non-zero weight towards zero with the same constant force, whatever its size. Small weights get pushed exactly to zero and stay there, which gives a sparse model. Geometrically, the L1 constraint region is a diamond with corners on the axes, and the best solution often lands on a corner. The elastic net keeps the sparsity of L1 while borrowing L2's habit of keeping correlated features together.

Penalties only make sense when features are on comparable scales. A weight on "income in euros" and a weight on "age in years" are not comparable until both features are standardised, which is one reason data preprocessing comes before fitting a penalised model.

Weight decay and AdamW

For plain stochastic gradient descent, adding an L2 penalty and applying weight decay give the same update (after rescaling λ). For adaptive methods such as Adam they differ. With L2 inside Adam, the penalty's gradient is divided by the same per-weight scaling as the loss gradient, so weights with large historic gradients are barely regularised. AdamW applies the decay directly to the weights, outside the adaptive scaling, so every weight shrinks at the same relative rate. In PyTorch, torch.optim.AdamW has a default weight_decay of 0.01, while torch.optim.Adam treats weight_decay as an L2 term. See optimisers for the full picture.

Early stopping

During training, the training loss keeps falling, but the validation loss usually falls, flattens and then rises as the model starts memorising. Early stopping watches a validation score and stops when it has not improved for a set number of checks, called the patience, then restores the best checkpoint. For simple models trained by gradient descent, stopping early has an effect similar to an L2 penalty: the weights have not had time to grow large. Many libraries build it in; scikit-learn's histogram gradient boosting turns it on automatically when there are more than 10,000 samples.

Dropout, noise and max-norm

Dropout switches off a random fraction of units on every training step and rescales the rest, so the network cannot depend on any single unit. At test time all units are used. It behaves like training a huge ensemble of thinned networks that share weights. Adding Gaussian noise to inputs or activations has a related effect. Max-norm caps the length of each unit's incoming weight vector at a constant c, projecting it back whenever an update makes it longer. It works well with dropout and high learning rates because weights cannot blow up.

Data augmentation and mixup

Augmentation creates new training examples by applying changes that should not alter the label: flips, crops and colour shifts for images, time shifts for audio, synonym swaps or back-translation for text. It encodes prior knowledge about what does not matter. Mixup blends two examples and their labels in a random proportion, which encourages the model to behave linearly between training points. Augmentation is often the single strongest regulariser for vision models.

Label smoothing

With one-hot targets, cross-entropy keeps rewarding the model for pushing the correct class probability closer to 1, which drives logits to extreme values. Label smoothing replaces the hard target with a slightly softened one, for example 0.9 for the true class and the remaining 0.1 spread over all classes. In PyTorch this is the label_smoothing argument of CrossEntropyLoss. It usually improves generalisation and calibration, though Müller and colleagues found it can make the network a worse teacher for knowledge distillation.

Regularisation in trees

Decision trees overfit by splitting until each leaf holds a handful of examples. They are regularised by limiting maximum depth, the minimum number of samples per leaf or split, and the number of leaves, and by pruning. Gradient boosting adds shrinkage: each new tree's contribution is multiplied by a learning rate below 1, so many small steps replace a few large ones. Row and column subsampling add randomness like bagging. XGBoost also puts L2 (lambda, default 1) and L1 (alpha, default 0) penalties on leaf values and a minimum gain for a split (gamma, default 0); its default learning rate eta is 0.3 and default max_depth is 6. LightGBM defaults to 31 leaves, a learning rate of 0.1 and at least 20 samples per leaf. Random forests regularise mainly through averaging many decorrelated trees.

Choosing the strength

Strength is chosen on held-out data, usually with cross-validation. Search penalty weights on a logarithmic grid, such as 10⁻⁴ to 10², because the effect is multiplicative. Plot the validation score against strength: the curve usually rises, peaks and falls, and the peak is the choice. Libraries offer shortcuts, such as scikit-learn's RidgeCV, LassoCV and LogisticRegressionCV, which search the path efficiently.

from sklearn.linear_model import ElasticNetCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

model = make_pipeline(
    StandardScaler(),
    ElasticNetCV(l1_ratio=[0.1, 0.5, 0.9, 1.0], alphas=50, cv=5),
)
model.fit(X_train, y_train)
enet = model[-1]
print(enet.alpha_, enet.l1_ratio_, (enet.coef_ != 0).sum())

Key concepts and terms

A worked example

This is an illustrative scenario, not a report about a real company. A retailer wants to predict weekly sales for each store from 400 features: past sales, promotions, local weather, holidays and many product-mix ratios. There are only 1,500 store-weeks of history.

Step 1: an unregularised baseline. Ordinary least squares on standardised features reaches an R² of 0.97 on training data but only 0.41 on a held-out quarter. Many coefficients are huge and have opposite signs for near-duplicate features, a classic symptom of overfitting with correlated inputs.

Step 2: ridge. Five-fold cross-validation over α from 10⁻³ to 10³ peaks at α = 30. Validation R² rises to 0.68. Every feature still has a non-zero weight, but the weights are small and stable from fold to fold.

Step 3: lasso and elastic net. The lasso peaks at a validation R² of 0.66 with only 38 non-zero features, but it keeps one of each correlated pair at random, and the chosen set changes between folds. The elastic net with an L1 ratio of 0.5 reaches 0.69 with 71 features and a much more stable selection.

Step 4: a tree model. A LightGBM model with defaults reaches 0.64. Lowering the learning rate to 0.03, limiting leaves to 15, requiring 40 samples per leaf, using 80 per cent column sampling and early stopping on a validation fold gives 0.72.

Step 5: decide. The team keeps the elastic net as an explainable baseline and the regularised boosting model for production, and records both in the experiment log. Neither choice was possible without measuring on data the model had not seen.

Strengths and limitations

Strengths:

• Cheap: most methods add one line of code and little training time.
• Reliable: some regularisation almost always helps when data is limited relative to model size.
• Flexible: methods can be combined, such as weight decay plus dropout plus augmentation.
• Interpretable side effects: L1 gives feature selection, and shrinkage stabilises coefficients.
• Applies to every model family, from linear models to transformers.

Limitations:

• Each method adds a strength setting that must be tuned on held-out data.
• Too much regularisation causes underfitting, which can look like "the model is just weak".
• Penalties assume comparable feature scales; forgetting to standardise silently changes their effect.
• Regularisation reduces variance but cannot fix biased data, wrong labels or leakage.
• Methods interact: strong dropout plus strong weight decay plus heavy augmentation can over-constrain a model.

How it compares with alternatives

These are not exclusive. A typical modern recipe combines a pre-trained model, data augmentation, weight decay and early stopping.

Practical guidance and best practices

1. Always standardise numeric features before using L1, L2 or elastic net penalties, and fit the scaler on training data only.
2. Search penalty strengths on a logarithmic scale and choose them with cross-validation, not on the test set.
3. For neural networks trained with Adam, use AdamW and exclude biases and normalisation parameters from decay.
4. Use early stopping almost everywhere: it is cheap and it doubles as a guard against wasting compute.
5. For boosted trees, lower the learning rate, raise the number of trees and let early stopping choose when to stop.
6. Prefer the elastic net over the lasso when features come in correlated groups.
7. Treat augmentation as the first regulariser for images, audio and text; it often matters more than any penalty.
8. Change one regulariser at a time and keep a record, so you know which one actually helped.
9. Re-tune strength when the dataset size changes a lot: more data usually needs less regularisation.

Common mistakes

• Applying a penalty to unscaled features, so that the penalty falls mostly on features measured in small units.
• Using L2 inside Adam and believing it behaves like weight decay.
• Selecting λ on the test set, which leaks information and gives an optimistic final score.
• Penalising the intercept or bias terms, which shifts predictions without reducing complexity.
• Stacking every regulariser at maximum strength and then concluding that the architecture is too weak.
• Reading lasso's selected features as "the true causes", when the choice among correlated features is partly arbitrary.
• Turning off dropout or augmentation in training by accident, or leaving them on at evaluation time.

Frequently asked questions

Is weight decay the same as L2 regularisation?

For plain stochastic gradient descent they are equivalent up to a rescaling of the strength. For adaptive optimisers such as Adam they are not. With an L2 term, Adam rescales the penalty gradient per weight, so frequently updated weights are barely regularised. Decoupled weight decay, as in AdamW, shrinks every weight directly and usually generalises better.

Should I use L1 or L2?

Use L2 when you expect many features to matter a little and want stable coefficients. Use L1 when you expect only a few features to matter and want a sparse, easy-to-read model. When in doubt, use the elastic net and let cross-validation choose the mix, because it covers both ends.

Does regularisation help if I have a lot of data?

It helps less, because a large dataset already limits variance. But large models trained on large data still benefit from some weight decay, augmentation and early stopping, and large language models are commonly trained with weight decay. The optimal strength usually falls as data grows.

Why does dropout make training loss higher?

Dropout deliberately damages the network during training, so the training loss is measured on a weakened model. At evaluation time dropout is switched off and the full network is used. A gap where training loss is above validation loss is therefore normal with dropout and is not a bug.

How do I regularise a gradient-boosted tree model?

Lower the learning rate, limit depth or number of leaves, require a minimum number of samples or hessian weight per leaf, use row and column subsampling, add L1 or L2 penalties on leaf values if the library supports them, and use early stopping on a validation set. Tune these together, because a lower learning rate needs more trees.

Is early stopping a form of regularisation?

Yes. It limits how far the weights can travel from their starting point, which restricts the effective capacity of the model. For linear models trained by gradient descent it behaves much like an L2 penalty. It is also the cheapest regulariser, because it saves training time instead of adding to it.

What does label smoothing do to calibration?

It usually improves calibration by stopping the model from pushing probabilities to 0 and 1. But it changes what the probabilities mean, because the training target itself is no longer one-hot. If you need well-calibrated probabilities, check them with a reliability diagram and the evaluation metrics for calibration, and recalibrate after training if needed.

Glossary

• Regularisation: any technique that improves generalisation by limiting or guiding what a model learns.
• Ridge regression: linear regression with an L2 penalty on the coefficients.
• Lasso: linear regression with an L1 penalty, which can set coefficients exactly to zero.
• Elastic net: a penalty that mixes L1 and L2 with a chosen ratio.
• AdamW: a version of Adam with weight decay applied separately from the adaptive gradient step.
• Max-norm constraint: a cap on the length of each unit's incoming weight vector.
• Dropout: randomly switching off units during training.
• Mixup: training on blended pairs of examples and their labels.
• Shrinkage: multiplying each boosting step by a learning rate below 1.
• Pruning: removing branches of a decision tree that do not improve validation performance.

Summary

Regularisation is how machine learning turns flexible models into reliable ones. It covers explicit penalties such as L2, L1 and the elastic net; weight decay, which inside Adam should be decoupled as in AdamW; procedural methods such as early stopping and dropout; data-side methods such as augmentation, mixup and label smoothing; and structural limits such as tree depth, leaf size and shrinkage in boosting. All of them trade a little training accuracy for better accuracy on new data, and all of them have a strength that must be chosen on held-out data, usually by cross-validation on a logarithmic grid. Standardise before penalising, change one regulariser at a time, and remember that no amount of regularisation can rescue leaked, mislabelled or unrepresentative data.

Related

It belongs to Training fundamentals in my stack, under Machine learning. Others in the same group: