Dimensionality Reduction

machine-learning unsupervised-learning

Definition

Dimensionality Reduction

Dimensionality reduction is the process of transforming data from a high-dimensional instance space into a lower-dimensional manifold or subspace while preserving essential structural properties. Formally, given a dataset $D=\{x_i{\}}_{i=1}^n$ where $x_i\in {\mathbb{R}}^D$, the process seeks a mapping function $f:{\mathbb{R}}^D\to {\mathbb{R}}^d$ such that $d\ll D$.

Methodological Approaches

Feature Selection: The identification and retention of a subset ${\mathcal{X}}^{\prime }\subset \mathcal{X}$ of the original features based on their relevance or information gain.

Feature Projection: The construction of a new latent feature space $\mathcal{Z}$ via linear transformations (e.g., PCA) or non-linear architectures (e.g., Autoencoders).

Objectives

The primary goals are to mitigate the curse of dimensionality, improve computational efficiency during training, and facilitate the visualisation of high-dimensional distributions while maintaining intrinsic information content.