Gini Index

machine-learning statistics

Definition

Gini Index

The Gini index (or Gini impurity) is a measure used in decision tree algorithms to quantify the frequency with which a randomly chosen element from a set would be incorrectly labelled if it were randomly labelled according to the distribution of labels in the subset. Formally, for a set with $k$ classes and class probabilities $p_i$, the Gini impurity $I_G$ is:

$$I_G=1-\sum _{i=1}^k{p}_i^2$$

For a split on an attribute, the resulting impurity is the weighted average of the impurities of the child nodes. A Gini index of $0$ indicates a pure node where all elements belong to a single class.