Distance-based Learning Algorithm

machine-learning

Definition

Distance-based Learning Algorithm

A distance-based learning algorithm is a procedure that derives its predictions primarily by evaluating the distance between elements of the instance space $\mathcal{X}$. Formally, given a metric $d:\mathcal{X}\times \mathcal{X}\to {\mathbb{R}}_{\ge 0}$, the algorithm utilises the geometric proximity of data points to infer labels or structural properties.

Geometric Dependency

The predictive performance of these methods is inherently dependent on the choice of the metric $d$ and its alignment with the underlying data geometry. Common applications include $k$-nearest neighbours ($k$NN) for classification and $k$-means for clustering, where the proximity in $\mathcal{X}$ is assumed to reflect similarity in the label space $\mathcal{Y}$.