Which pruning method uses a separate validation set to prune branches?

Which pruning method uses a separate validation set to prune branches?

• A. Pre-Pruning
• B. Post-Pruning
• C. Reduced error pruning
• D. Cost complexity pruning

Answer: (C)

The key idea is using validation data to judge how pruning affects generalization. Reduced error pruning does exactly that: it starts with the fully grown tree and, for each subtree, tries replacing it with a single leaf. It then checks how this change impacts accuracy on a separate validation set. If pruning a subtree doesn’t reduce (and may even improve) validation accuracy, that subtree is pruned away. This direct use of a held-out validation set to guide pruning decisions is what distinguishes this approach from others. Pre-pruning stops growth during tree construction based on criteria observed on the training process, not by evaluating performance on a separate validation set. Other post-pruning methods exist that use different criteria (like a complexity penalty) and may involve cross-validation to choose parameters, but the explicit use of a separate validation set to determine whether pruning improves generalization points to reduced error pruning.

The key idea is using validation data to judge how pruning affects generalization. Reduced error pruning does exactly that: it starts with the fully grown tree and, for each subtree, tries replacing it with a single leaf. It then checks how this change impacts accuracy on a separate validation set. If pruning a subtree doesn’t reduce (and may even improve) validation accuracy, that subtree is pruned away. This direct use of a held-out validation set to guide pruning decisions is what distinguishes this approach from others.

Pre-pruning stops growth during tree construction based on criteria observed on the training process, not by evaluating performance on a separate validation set. Other post-pruning methods exist that use different criteria (like a complexity penalty) and may involve cross-validation to choose parameters, but the explicit use of a separate validation set to determine whether pruning improves generalization points to reduced error pruning.