Polynomial Kernel Function

machine-learning statistics nlp

Definition

Polynomial Kernel Function

The polynomial kernel represents a feature mapping into a space of all possible monomial combinations of the input features up to a specified degree $d$. Formally, for a degree $d\in \mathbb{N}$ and a constant $c\ge 0$:

$k(\mathbf{x},{\mathbf{x}}^{\prime })=({\mathbf{x}}^{\top }{\mathbf{x}}^{\prime }+c{)}^d$

Expressivity and Complexity

Feature Expansion: This kernel implicitly computes the inner product in a feature space of dimension $\left(\genfrac{}{}{0ex}{}{n+d}{d}\right)$. For example, a second-degree polynomial kernel on 2D data expands the features to include $x_1^2,x_2^2,$ and $x_1{x}_2$.

Overfitting Risk: While providing higher capacity than the linear kernel, larger degrees $d$ significantly increase the risk of overfitting, particularly in domains like natural language processing. In practice, $d=2$ or $d=3$ are common choices to balance expressivity and generalisation.