Logistic Regression

machine-learning classification statistics

Definition

Logistic Regression

Logistic regression is a probabilistic linear classifier for binary classification that uses a linear score to parameterise a conditional probability for the class label.

Given an input , a weight vector , and a bias , the model first computes

It then maps this score through the sigmoid function and interprets the result as a conditional probability:

Equivalently, follows a Bernoulli distribution with parameter , so . The log-odds are linear,

and the decision boundary at threshold is the hyperplane .

Optimisation

The parameters and are usually estimated by maximum likelihood estimation, equivalently by minimising the cross-entropy loss on labelled data with :

For ordinary binary logistic regression, this objective is convex in and and is commonly optimised with gradient descent or related methods.