Hypothesis Class

machine-learning learning-theory

Definition

Hypothesis Class

A hypothesis class $\mathcal{H}$ is the set of all candidate functions (models) that a learning algorithm is permitted to select. It defines the search space of the learner. For supervised learning, the class is a subset of all possible mappings from instances to labels:

$\mathcal{H}\subseteq \{h\mid h:\mathcal{X}\to \mathcal{Y}\}$

Inductive Bias and Capacity

The selection of $\mathcal{H}$ represents the inductive bias of the learner, as it restricts the model to certain structural forms (e.g., linear functions, decision trees of depth $k$).

Expressiveness: The “richness” of the class is quantified by its VC dimension. A class with higher capacity can represent more complex patterns but requires a higher sample complexity to ensure generalisation and avoid overfitting.