Quadratic Discriminant Analysis

Definition

Quadratic Discriminant Analysis

Quadratic discriminant analysis (QDA) is a classifier that models each class by a multivariate Gaussian distribution with its own covariance matrix. For two classes,

where each is positive definite. The class scores and prediction are

A fixed convention resolves ties. With the true parameters, this is the Bayes optimal classifier under zero–one loss; in practice the means, covariances and priors are estimated from labelled data.

From Gaussian densities

Let be the density of . Bayes’ theorem gives

The denominator is common to both classes. Taking logarithms preserves which numerator is larger:

Only the class-independent constant is removed.

Boundary coefficients

Use , so selects class . Expanding the two quadratic expressions gives

The zero set is generally a quadratic decision boundary. Equal covariances make , yielding linear discriminant analysis.