Multiclass Classification

machine-learning classification statistics

Definition

Multiclass Classification

Multiclass classification is a supervised learning task where the objective is to categorise instances into one of more than two distinct classes. Formally, given an instance space , the learner seeks a mapping where the label space has cardinality .

Reduction Strategies

Since many foundational algorithms (e.g., SVM, Perceptron) are inherently binary, multiclass problems are often decomposed into multiple binary sub-problems.

One-vs-Rest (OvR): Also known as one-vs-all, this strategy trains separate binary classifiers, each designed to distinguish one class from all other classes combined. The final prediction is the class whose classifier produces the highest confidence score.

One-vs-One (OvO): This strategy trains binary classifiers, one for every possible pair of classes. The final label is determined by a majority vote among all pairs.

Softmax Regression: A direct multiclass extension of logistic regression that utilises the softmax function to produce a probability distribution over all classes, typically trained using categorical cross-entropy.