Hinge Loss

machine-learning optimisation statistics

Definition

Hinge Loss

The hinge loss is a loss function primarily utilised for training maximum-margin classifiers, such as the Support Vector Machine. For a predicted output $\hat{y}$ and a true label $y\in \{-1,+1\}$, the hinge loss $L$ is defined as:

$L(y,\hat{y})=\max (0,1-y\hat{y})$

Properties

Convex Approximation: The hinge loss provides a convex upper bound on the non-convex 0-1 loss function. This allows for efficient optimisation using gradient-based methods while still penalising misclassifications and instances that fall within the margin.

Margin Maximisation: By requiring $y\hat{y}\ge 1$ for a zero loss, the function encourages the model to not only classify points correctly but to do so with a minimum level of confidence, directly leading to the identification of a maximum-margin hyperplane.