Hyperparameter Tuning

machine-learning optimisation

Definition

Hyperparameter Tuning

Hyperparameter tuning is the process of searching for the optimal set of hyperparameters ${\lambda }^{\ast }$ that maximises model performance on a validation set.

Common search strategies include:

1. Grid Search: An exhaustive search over a specified subset of the hyperparameter space. While thorough, it suffers from the curse of dimensionality as the number of combinations grows exponentially.
2. Random Search: Randomly samples the hyperparameter space for a fixed number of iterations. Research suggests this is more efficient than grid search for identifying important hyperparameters in high-dimensional spaces.