Linear Classifier

machine-learning classification

Definition

Linear Classifier

A linear classifier is a supervised learning model that makes classification decisions based on a linear combination of input features. Formally, for an input , the predicted class is determined by a hyperplane in the instance space:

where is the weight vector (normal to the hyperplane) and is the bias. Geometrically, the decision boundary is the set , which partitions the space into two half-spaces.

Training Strategies

Various algorithms exist to identify the optimal parameters and depending on the objective function and data arrival paradigm.

Least Squares Classification: This approach approximates the discrete sign function with a linear identity and solves the resulting system using the normal equations with target labels .

Perceptron Algorithm: An iterative online learning procedure that updates weights specifically based on misclassification errors, providing a hard decision boundary.

Logistic Regression: A probabilistic framework that maps the linear output to a confidence interval using the sigmoid function, allowing for uncertainty estimation.