L1 Regularisation

machine-learning

Definition

L1 Regularisation

L1 Regularisation, also known as Lasso (Least Absolute Shrinkage and Selection Operator), is a regularisation technique that adds a penalty equal to the absolute value of the magnitude of coefficients to the loss function. This encourages the model to keep the coefficients of less important features exactly zero, effectively performing feature selection.

$${\mathcal{L}}_{L_1}(\mathcal{L})=\mathcal{L}+\lambda \sum _{j=1}^p\mid {\beta }_j\mid$$

where:

• $\lambda$ is the regularisation parameter controlling the strength of penalty.
• ${\beta }_j$ are the coefficients of the model.

Explanation

The term $\lambda {\sum }_{j=1}^p\mid {\beta }_j\mid$ penalises large coefficients. The higher $\lambda$ is, the more coefficients are pushed towards zero.

Feature Selection

Since this method allows coefficients to be zero, a simpler model can be found. Not all parameters were used → smaller model.