Linear Decision Boundary

Definition

Linear Decision Boundary

A linear decision boundary is the hyperplane where the score of a binary linear classifier is zero. In a feature space , let and . Then

The normal vector determines the orientation of , and the bias determines its offset. Positive and negative scores assign points to the two classes; points on require a tie-breaking convention.

Here, “linear” allows an offset: is an affine hyperplane and need not pass through the origin. It need not separate all training examples correctly.

In the input space

For a feature map , the corresponding boundary in the input space is

For vector-valued inputs, a non-linear feature map can give a non-linear decision boundary even though is a hyperplane in feature space.

For example, take and the same score as above. Then

The points and have positive scores; has a negative score. Colours match each point with its feature representation.