Which ensemble method builds trees sequentially, with each new tree focusing on the errors of previous trees?

Which ensemble method builds trees sequentially, with each new tree focusing on the errors of previous trees?

• A. Boosting
• B. Random Forests
• C. K-Nearest Neighbors
• D. Support Vector Machines

Answer: (B)

Boosting is about building models sequentially where each new tree tries to fix the mistakes of what came before. You start with a first tree that captures some patterns, then fit a second tree to the errors the first one made, then a third to the errors of the combined first two, and so on. This approach pushes the ensemble to pay more attention to difficult cases. In practice, methods like AdaBoost increase the weights of misclassified instances so later trees focus on them, while gradient boosting fits each new tree to the residuals or the negative gradient of the loss, directly learning what the previous ensemble got wrong. Random Forests, in contrast, grow many trees independently on bootstrap samples and then aggregate their outputs, so they don’t target prior errors sequentially. K-Nearest Neighbors and Support Vector Machines aren’t ensembles built in this sequential error-correcting way.

Boosting is about building models sequentially where each new tree tries to fix the mistakes of what came before. You start with a first tree that captures some patterns, then fit a second tree to the errors the first one made, then a third to the errors of the combined first two, and so on. This approach pushes the ensemble to pay more attention to difficult cases. In practice, methods like AdaBoost increase the weights of misclassified instances so later trees focus on them, while gradient boosting fits each new tree to the residuals or the negative gradient of the loss, directly learning what the previous ensemble got wrong.

Random Forests, in contrast, grow many trees independently on bootstrap samples and then aggregate their outputs, so they don’t target prior errors sequentially. K-Nearest Neighbors and Support Vector Machines aren’t ensembles built in this sequential error-correcting way.