Özer Cem KELAHMETKELAHMET
Glossary · Machine learning

What is Gradient boosting? Definition and meaning

An ensemble method that adds decision trees one at a time, each fitted to the remaining errors of the current model.

Definition

Gradient boosting: An ensemble method that adds decision trees one at a time, each fitted to the remaining errors of the current model.

In short: Many small trees, each fixing the last.

When it is used

Classification and regression on tabular data.

Where it fits

Gradient boosting belongs to ML foundations in the Machine learning topic. These are the other terms in the same group:

  • Linear regression: A model that predicts a numeric target as a weighted sum of input features, fitted by minimising squared error.
  • Logistic regression: A linear model that outputs class probabilities through the logistic (sigmoid) function.
  • Decision trees: A model that splits data step by step on feature thresholds, forming a tree of decisions that ends in predictions.
  • k-nearest neighbors: A method that predicts by finding the k closest training examples and using their labels or values.
  • Naive Bayes: A probabilistic classifier that applies Bayes' theorem assuming features are independent given the class.
  • Principal component analysis: A linear method that finds orthogonal directions of maximum variance and projects data onto the top ones.
  • t-SNE: A non-linear method that projects high-dimensional data to 2D or 3D, preserving local neighbourhoods for visualisation.
  • Cross-validation: A method that rotates which part of the data is held out for testing, averaging results across folds.

Learn more

The full guide to Gradient boosting explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: GPTQ · Next term: Gradient checkpointing

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open