# What is Gradient boosting? Definition and meaning

An ensemble method that adds decision trees one at a time, each fitted to the remaining errors of the current model.

## Definition

**Gradient boosting**: An ensemble method that adds decision trees one at a time, each fitted to the remaining errors of the current model.

In short: Many small trees, each fixing the last.

## When it is used

Classification and regression on tabular data.

## Where it fits

Gradient boosting belongs to **ML foundations** in the [Machine learning](/topics/machine-learning) topic. These are the other terms in the same group:

- **[Linear regression](/glossary/linear-regression)**: A model that predicts a numeric target as a weighted sum of input features, fitted by minimising squared error.
- **[Logistic regression](/glossary/logistic-regression)**: A linear model that outputs class probabilities through the logistic (sigmoid) function.
- **[Decision trees](/glossary/decision-trees)**: A model that splits data step by step on feature thresholds, forming a tree of decisions that ends in predictions.
- **[k-nearest neighbors](/glossary/k-nearest-neighbors)**: A method that predicts by finding the k closest training examples and using their labels or values.
- **[Naive Bayes](/glossary/naive-bayes)**: A probabilistic classifier that applies Bayes' theorem assuming features are independent given the class.
- **[Principal component analysis](/glossary/principal-component-analysis)**: A linear method that finds orthogonal directions of maximum variance and projects data onto the top ones.
- **[t-SNE](/glossary/t-sne)**: A non-linear method that projects high-dimensional data to 2D or 3D, preserving local neighbourhoods for visualisation.
- **[Cross-validation](/glossary/cross-validation)**: A method that rotates which part of the data is held out for testing, averaging results across folds.

## Learn more

The full guide to **[Gradient boosting](/tools/gradient-boosting)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [GPTQ](/glossary/gptq) · Next term: [Gradient checkpointing](/glossary/gradient-checkpointing)
