# Generalisation: overfitting, regularisation, normalisation and honest evaluation

A model is only useful on data it has never seen. This page covers why networks overfit and the techniques that keep them general.

## Overview

Large networks can memorise their training data, noise included. Generalisation is the gap between training performance and performance on new data, and most of the craft of deep learning is keeping that gap small without making the model too weak.

## Signs of overfitting

Training loss keeps falling while validation loss rises; the model is very confident on wrong answers; performance collapses on data from a slightly different source. The [overfitting simulation](/ai-simulations/overfitting-and-regularisation) shows the gap opening as model capacity grows. See also the [bias-variance trade-off](/tools/bias-variance-trade-off).

## The techniques

| Technique | How it helps | Read more |
|---|---|---|
| More and more varied data | The best regulariser | [Data preprocessing](/tools/data-preprocessing) |
| Data augmentation | Creates realistic variations | [Synthetic data](/tools/synthetic-data) |
| Weight decay | Penalises large weights | [Regularization](/tools/regularization) |
| Dropout | Randomly drops units so none dominates | [Dropout](/tools/dropout) |
| Normalisation layers | Stabilise training, smooth the loss surface | [Normalization layers](/tools/normalization-layers) |
| Early stopping | Stop when validation stops improving | |
| Transfer learning | Start from a model trained on more data | [Transfer learning](/tools/transfer-learning) |
| Ensembles | Average several models' errors | [Ensemble learning](/tools/ensemble-learning) |

## Normalisation

Batch normalisation rescales activations across a batch; layer normalisation and RMSNorm rescale within each example and are standard in transformers. They let deeper networks train with higher learning rates. Try the [normalisation and dropout simulation](/ai-simulations/normalisation-and-dropout).

## Honest evaluation

Split data into training, validation and test sets before any modelling, and split the way the model will be used: by time for forecasting, by user or patient for personal data. Use the test set once. [Cross-validation](/tools/cross-validation) gives steadier estimates on small data. For large models, check benchmarks for contamination; the [evaluation simulation](/ai-simulations/train-evaluation) shows how leaked test items inflate scores.

## Large models generalise differently

Very large models trained on enormous data often keep improving with more parameters, even past the point where classical theory expects overfitting. Their failures move from memorising the training set to gaps in what the data covered, which is why data quality and fresh evaluation sets matter so much.

## Frequently asked questions

### How much dropout should I use?

Values between 0.1 and 0.3 are common for medium models; very large pre-trained language models often use little or none.

### Is a bigger model always worse at generalising?

No. With enough data and regularisation, bigger models usually generalise better. Overfitting is about capacity relative to data.

## Summary

Use more varied data, regularise with weight decay, dropout and normalisation, stop early, and evaluate on splits that match real use. The [deep learning essentials path](/paths/deep-learning-essentials) puts these together.

## Back to the topic

This page is part of **[Deep learning](/topics/deep-learning)**, which maps every concept, tool, simulation and article on the subject.
