Which regularization keeps all features but prevents any single feature from becoming too influential?

Which regularization keeps all features but prevents any single feature from becoming too influential?

• A. LASSO (L1 penalty)
• B. Ridge Regression (L2 penalty)
• C. Elastic Net (L1 + L2 penalties)
• D. K-fold Cross-Validation

Answer: (B)

Regularization that keeps all features but prevents any single feature from becoming too influential uses the L2 penalty, known as Ridge Regression. The L2 penalty adds the sum of squared coefficients to the loss, which discourages large coefficients but, unlike L1, does not drive them to zero. This means every feature remains in the model, just with its influence shrunk toward zero so no single feature can dominate. In contrast, LASSO (L1 penalty) tends to push some coefficients exactly to zero, effectively dropping those features from the model. Elastic Net blends L1 and L2, so it can both shrink coefficients and, depending on tuning, drop some features. K-fold cross-validation is a validation method used to assess model performance and choose hyperparameters; it is not a regularization method itself.

Regularization that keeps all features but prevents any single feature from becoming too influential uses the L2 penalty, known as Ridge Regression. The L2 penalty adds the sum of squared coefficients to the loss, which discourages large coefficients but, unlike L1, does not drive them to zero. This means every feature remains in the model, just with its influence shrunk toward zero so no single feature can dominate.

In contrast, LASSO (L1 penalty) tends to push some coefficients exactly to zero, effectively dropping those features from the model. Elastic Net blends L1 and L2, so it can both shrink coefficients and, depending on tuning, drop some features. K-fold cross-validation is a validation method used to assess model performance and choose hyperparameters; it is not a regularization method itself.