Generative Learning

machine-learning statistics

Definition

Generative Learning

Generative learning is a strategy in supervised learning that models the joint probability distribution $P(\mathbf{x},y)$ of the input features and the labels. Formally, the learner estimates $P(y)$ (the class prior) and $P(\mathbf{x}|y)$ (the class-conditional distribution) to derive the posterior $P(y|\mathbf{x})$ via Bayes’ theorem:

$P(y|\mathbf{x})=\frac{P(\mathbf{x}|y)P(y)}{P(\mathbf{x})}$

Characteristics

Capability: Unlike discriminative models, generative models can be used to generate new data points by sampling from the learned distribution $P(\mathbf{x}|y)$.

Examples: A classic example is the Naive Bayes classifier, which assumes conditional independence between features to simplify the estimation of $P(\mathbf{x}|y)$.