Loss

machine-learning

Definition

Loss Function

A loss function $L:\mathcal{Y}\times \mathcal{Y}\to {\mathbb{R}}_{\ge 0}$ is a non-negative function that quantifies the discrepancy between the predicted label $\hat{y}=h(x)$ and the true label $y\in \mathcal{Y}$. In supervised learning, the objective is to find a hypothesis $h\in \mathcal{H}$ that minimises the expected loss, also known as the risk. Common examples include:

• 0-1 Loss: $L(y,\hat{y})=\mathbb{I}(y\ne \hat{y})$ for classification.
• Squared Loss: $L(y,\hat{y})=(y-\hat{y}{)}^2$ for regression.