Hyperparameter Tuning

Hyperparameter Tuning Q&A

1What are hyperparameters?
Answer: External model settings chosen before training.
2Parameters vs hyperparameters?
Answer: Parameters are learned from data; hyperparameters are configured.
3Grid search?
Answer: Exhaustively tests predefined combinations.
4Random search?
Answer: Samples random combinations; often more efficient than full grid.
5Bayesian optimization?
Answer: Builds surrogate model to intelligently pick promising settings.
6What is search space?
Answer: Range/distribution of candidate hyperparameter values.
7Why use CV in tuning?
Answer: Stable estimate for comparing configurations.
8Early stopping role?
Answer: Stops training when validation score no longer improves.
9Pitfall during tuning?
Answer: Overfitting to validation sets by excessive iteration.
10One-line summary?
Answer: Good tuning improves performance without changing core algorithm.