Multiple Polynomial Regression

machine-learning regression statistics

Definition

Multiple Polynomial Regression

Multiple polynomial regression is an extension of polynomial regression to multivariate input features $\mathbf{x}\in {\mathbb{R}}^d$. The model expands the instance space into a higher-dimensional feature space containing all polynomial combinations of the original features up to degree $n$. Formally, the hypothesis is expressed as $h_{\theta }(\mathbf{x})={\theta }^{\top }\varphi (\mathbf{x})$, where $\varphi (\cdot )$ is a basis mapping including all monomial terms ${\prod }_{j=1}^d{x}_j^{k_j}$ such that $\sum k_j\le n$.

Dimensionality and Overfitting

As the number of features $d$ and the degree $n$ increase, the dimensionality of the expanded feature space $D=\left(\genfrac{}{}{0ex}{}{d+n}{n}\right)$ grows combinatorially. This rapid expansion increases the probability of overfitting, necessitating the application of regularisation techniques (e.g., Ridge or Lasso) to ensure the model generalises to unseen data.