Semi-Supervised Learning

machine-learning

Definition

Semi-Supervised Learning

Semi-supervised learning is a paradigm of machine learning that combines a small amount of labelled data with a large amount of unlabelled data during training. Formally, given a labelled dataset $D_L=\{(x_i,y_i){\}}_{i=1}^l$ and an unlabelled dataset $D_U=\{x_j{\}}_{j=l+1}^{l+u}$ where $u\gg l$, the learner aims to find a hypothesis $h\in \mathcal{H}$ that leverages the distribution of $D_U$ to improve the predictive performance on the label space $\mathcal{Y}$ compared to learning from $D_L$ alone.

Fundamental Assumptions

The success of semi-supervised methods relies on assumptions about the underlying data geometry. The cluster assumption posits that decision boundaries should reside in low-density regions, ensuring that points in the same cluster share labels. Complementarily, the manifold assumption states that high-dimensional data is supported on a lower-dimensional manifold $\mathcal{M}$, which can be characterised using the unlabelled portion of the dataset.