Gradient Boosting

Gradient Boosting Q&A

1What is gradient boosting?
Answer: Sequentially adds weak learners to correct previous errors.
2Boosting vs bagging?
Answer: Bagging builds independently; boosting builds sequentially.
3What is learning rate?
Answer: Shrinks each tree’s contribution to improve generalization.
4Why shallow trees in boosting?
Answer: Weak learners reduce overfitting and improve incremental learning.
5Key hyperparameters?
Answer: n_estimators, learning_rate, max_depth, subsample.
6What is XGBoost advantage?
Answer: Regularization, handling sparsity, fast optimized implementation.
7What is LightGBM advantage?
Answer: Faster training on large data using histogram-based algorithms.
8What is CatBoost good for?
Answer: Strong performance with categorical variables out-of-the-box.
9How prevent overfitting?
Answer: Early stopping, lower depth, subsampling, regularization.
10Why boosting strong on tabular data?
Answer: Captures complex nonlinear interactions effectively.
11Any downside?
Answer: More tuning and longer training than simple baselines.
12One-line summary?
Answer: Gradient boosting is often a top performer for structured data.