Naive Bayes

machine-learning classification statistics

Definition

Naive Bayes

Naive Bayes is a probabilistic generative classifier based on Bayes’ theorem with the “naive” assumption of conditional independence between features given the class label. Formally, for an instance and class , the joint probability is modelled as:

Inference and Training

Classification Rule: The predicted label is identified by maximising the numerator of the posterior (since the evidence is constant for all classes):

Parameter Estimation: The class priors and class-conditional probabilities are typically estimated via MLE (relative frequencies) or MAP (to avoid zero probabilities through smoothing).