Self-Supervised Learning

machine-learning unsupervised-learning representation-learning

Definition

Self-Supervised Learning

Self-supervised learning is a form of unsupervised learning where the model generates its own supervision signal from the input data. Formally, a pretext task is defined where part of the input $x$ is hidden or transformed, and the learner must predict or reconstruct it (e.g., predicting the next word in a sequence or the missing patch in an image).

Learning Objectives

The primary goal is representation learning, where the model identifies an encoding function $f(x)$ that captures the intrinsic structure of the data. Unlike supervised learning, which requires external human-provided labels $y$, self-supervised learning derives targets directly from the unlabelled data itself. The resulting features are typically transferred to downstream tasks to improve generalisation or label efficiency.