Bayesian Learning Algorithm

machine-learning statistics

Definition

Bayesian Learning Algorithm

A Bayesian learning algorithm is a procedure that treats model parameters as random variables and utilises Bayes’ theorem to update the probability distribution over the hypothesis class based on observed data . Formally, the learner computes the posterior distribution :

where is the prior distribution, is the likelihood function, and is the evidence.

Inference and Prediction

Posterior Predictive Distribution: Predictions for a new instance are made by integrating over the parameter space , weighting each possible model by its posterior probability:

Uncertainty Quantification: Unlike frequentist methods that produce point estimates, Bayesian learning provides a full distribution over the parameters, allowing for the explicit modelling of epistemic uncertainty.