Representation Learning

machine-learning unsupervised-learning

Definition

Representation Learning

Representation learning is a paradigm concerned with the automatic discovery of feature representations $z=f(x)$ from raw data that are useful for downstream tasks. Formally, given an instance space $\mathcal{X}$, the learner identifies an encoding function $f:\mathcal{X}\to \mathcal{Z}$ that maps input to a latent space $\mathcal{Z}\subseteq {\mathbb{R}}^k$.

Characteristics

Information Preservation: The mapping $f$ must preserve the salient information of the original distribution while potentially satisfying constraints such as lower dimensionality, disentanglement, or invariance to nuisance variables.

Utility: The discovered features are typically transferred to supervised tasks where they often yield improved generalisation or faster convergence compared to learning from raw features.