Which method systematically tests multiple combinations of hyperparameters to find the best model?

Which method systematically tests multiple combinations of hyperparameters to find the best model?

• A. Random Search
• B. Grid Search
• C. Bayesian Optimization
• D. Manual Tuning

Answer: (B)

Systematically exploring a predefined set of hyperparameter values by evaluating every combination is grid search. By constructing a grid for each hyperparameter and training the model on every possible point in that grid, you obtain a complete view of how those specific values interact and which combo yields the best performance within that defined space. It’s straightforward and reproducible, which makes it a common baseline for hyperparameter tuning. The trade-off is workload: as you add more hyperparameters or more candidate values, the number of configurations grows rapidly and can become very expensive to run. Other approaches work differently. Random search selects configurations at random rather than exhaustively covering a grid, which can be more efficient in high-dimensional spaces but doesn’t guarantee that all grid combinations are tested. Bayesian optimization uses a probabilistic model to focus on promising areas of the space, often needing far fewer evaluations but not evaluating every possible combination. Manual tuning relies on intuition and ad hoc adjustments rather than systematic exploration.

Systematically exploring a predefined set of hyperparameter values by evaluating every combination is grid search. By constructing a grid for each hyperparameter and training the model on every possible point in that grid, you obtain a complete view of how those specific values interact and which combo yields the best performance within that defined space. It’s straightforward and reproducible, which makes it a common baseline for hyperparameter tuning. The trade-off is workload: as you add more hyperparameters or more candidate values, the number of configurations grows rapidly and can become very expensive to run.

Other approaches work differently. Random search selects configurations at random rather than exhaustively covering a grid, which can be more efficient in high-dimensional spaces but doesn’t guarantee that all grid combinations are tested. Bayesian optimization uses a probabilistic model to focus on promising areas of the space, often needing far fewer evaluations but not evaluating every possible combination. Manual tuning relies on intuition and ad hoc adjustments rather than systematic exploration.