Posterior Predictive Distribution

machine-learning statistics

Definition

Posterior Predictive Distribution

The posterior predictive distribution is the distribution over a new, unobserved data point $\tilde{D}$ given a set of observed (training) data $D$. Formally, it integrates the likelihood of the new data across the entire parameter space $\Theta$, weighted by the posterior probability of the parameters:

$P(\tilde{D}|D)={\int }_{\Theta }P(\tilde{D}|\theta )P(\theta |D)d\theta$

where $P(\theta |D)$ is the posterior distribution derived from observed data.

Bayesian Prediction

Unlike point estimation methods (MLE or MAP) which select a single optimal ${\theta }^{\ast }$, the posterior predictive distribution accounts for the uncertainty in the parameter estimation. It represents the full Bayesian approach to prediction, where every possible model in the hypothesis class contributes to the final output according to its consistency with the observed evidence.