One-Hot Encoding

machine-learning preprocessing

Definition

One-Hot Encoding

One-hot encoding is a preprocessing technique used to represent nominal variables as numerical vectors. Formally, for a categorical feature with a finite alphabet of $k$ unique symbols $\Sigma =\{s_1,s_2,\dots ,s_k\}$, each symbol $s_j$ is mapped to a binary vector $\mathbf{v}\in \{0,1{\}}^k$ where:

$$v_i=\{\begin{array}{ll}1& \text{if }i=j\\ 0& \text{else}\end{array}$$

This transformation ensures that the learning algorithm does not erroneously infer an ordinal relationship between purely nominal categories.

Applications

String Vectorisation: Sequences can be transformed into sparse matrices by concatenating the one-hot vectors of their constituent characters or words.

Neural Networks: Frequently used for the representation of discrete targets in classification tasks.

Example

A categorical feature representing the colour of an object (e.g., Red, Green, Blue) can be encoded as:

$$\left[\begin{array}{c}\text{Red}\\ \text{Green}\\ \text{Blue}\end{array}\right]\to \text{Red}=\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right],\text{Green}=\left[\begin{array}{c}0\\ 1\\ 0\end{array}\right],\text{Blue}=\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right]$$

where the active category is indicated by $1$ and others by $0$.