Hinge Loss

machine-learning optimisation statistics

Definition

Hinge Loss

The hinge loss is a loss function primarily utilised for training maximum-margin classifiers, such as the Support Vector Machine. For a predicted output and a true label , the hinge loss is defined as:

Properties

Convex Approximation: The hinge loss provides a convex upper bound on the non-convex 0-1 loss function. This allows for efficient optimisation using gradient-based methods while still penalising misclassifications and instances that fall within the margin.

Margin Maximisation: By requiring for a zero loss, the function encourages the model to not only classify points correctly but to do so with a minimum level of confidence, directly leading to the identification of a maximum-margin hyperplane.