Gradient Boosting

← Back to the article

Gradient boosting builds one small model after another. Each new model tries to fix the last mistakes.

What it does

Gradient boosting builds one small model after another. Each new model tries to fix the last mistakes.

Boosting: Making a model stronger • Gradient: Direction of the error

How it works

  • Start with a simple guess.
  • Check where the guess was wrong.
  • Add a new small model that focuses on those mistakes.

Uses and applications

  • Search engines can use it to rank better results higher.
  • A store can predict which products may sell next week.
  • A delivery company can predict which packages might arrive late.

In short

It is a chain of models where each one fixes mistakes from the one before it.

Related methods

  • Decision Tree: The small model that boosting adds at each step.
  • Random Forest: Another tree ensemble, built from independent trees that vote.

Back to Prediction and classification · All topics

The interactive demonstration and the Python walkthrough for this topic need JavaScript.