Which method constructs an ensemble by training on bootstrapped samples and aggregating predictions?

Which method constructs an ensemble by training on bootstrapped samples and aggregating predictions?

• A. AdaBoost
• B. Random Forests
• C. Gradient Boosting
• D. SVMs

Answer: (B)

Ensemble learning that uses bootstrap samples and then aggregates predictions is what defines Random Forests. In this approach, many decision trees are grown, each on a bootstrap sample drawn with replacement from the original data. After training, their outputs are combined to make the final prediction—majority vote for classification or averaging for regression. This aggregation smooths out the noise of any single tree and reduces overall model variance because the trees are trained on different subsets and, with random feature selection at each split, become more diverse. AdaBoost and Gradient Boosting, by contrast, build trees sequentially to correct the mistakes of earlier trees, rather than training each on bootstrap samples and averaging. SVMs are not an ensemble method by default; they’re typically a single model unless used within an ensemble framework.

Ensemble learning that uses bootstrap samples and then aggregates predictions is what defines Random Forests. In this approach, many decision trees are grown, each on a bootstrap sample drawn with replacement from the original data. After training, their outputs are combined to make the final prediction—majority vote for classification or averaging for regression. This aggregation smooths out the noise of any single tree and reduces overall model variance because the trees are trained on different subsets and, with random feature selection at each split, become more diverse. AdaBoost and Gradient Boosting, by contrast, build trees sequentially to correct the mistakes of earlier trees, rather than training each on bootstrap samples and averaging. SVMs are not an ensemble method by default; they’re typically a single model unless used within an ensemble framework.