Glossary · Machine learning
What is Hyperparameter tuning? Definition and meaning
The search for the training settings, such as learning rate or tree depth, that give a model its best performance on unseen data.
Definition
Hyperparameter tuning: The search for the training settings, such as learning rate or tree depth, that give a model its best performance on unseen data.
In short: Finding the right model settings.
When it is used
Almost every model, from gradient-boosted trees and SVMs to deep networks and fine-tuning runs.
Where it fits
Hyperparameter tuning belongs to Training fundamentals in the Machine learning topic. These are the other terms in the same group:
- Gradient descent: Gradient descent is an iterative method that lowers a loss by repeatedly moving parameters in the direction of steepest decrease.
- Loss functions: A loss function is a formula that scores how far a model's predictions are from the targets, giving training something to minimise.
- Regularization: Any technique that limits or guides what a model learns so that it performs better on unseen data, not just on its training set.
- Bias-variance trade-off: The idea that a model's error on new data comes from being too simple (bias), too sensitive to its training sample (variance), and unavoidable noise.
- Evaluation metrics: Numbers that summarise how well a model's predictions match the truth, chosen to reflect what matters for the task.
- Feature engineering: The craft of creating, transforming and selecting input variables so that a model can learn the pattern more easily and more reliably.
- Data preprocessing: The steps that turn raw, messy data into a clean, consistent numeric form that a model can be trained and evaluated on without leakage.
- Synthetic data: Data produced by a model, program or simulator rather than recorded from real events, used to train, test or share machine learning systems.
Learn more
The full guide to Hyperparameter tuning explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.