Hessian Matrix

analysis linear-algebra

Definition

Hessian Matrix

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a function taking as input a vector $\mathbf{x}\in {\mathbb{R}}^n$ and outputting a scalar $f(\mathbf{x})\in \mathbb{R}$. If all second-order Partial derivatives of $f$ exist, then the Hessian matrix $\mathbf{H}$ of $f$ is a square matrix $n\times n$, defined as:

$${\mathbf{H}}_f=\left[\begin{array}{cccc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}& \dots & \frac{{\partial }^{2}f}{\partial x_1\partial x_n}\\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}& \dots & \frac{{\partial }^{2}f}{\partial x_2\partial x_n}\\ ⋮& ⋮& \ddots & ⋮\\ \frac{{\partial }^{2}f}{\partial x_n\partial x_1}& \frac{{\partial }^{2}f}{\partial x_n\partial x_2}& \dots & \frac{{\partial }^{2}f}{\partial x_n^2}\end{array}\right]$$

Or short:

$$({\mathbf{H}}_f{)}_{i,j}=\frac{{\partial }^{2}f}{\partial x_i\partial x_j}$$

The Hessian matrix of a function $f$ is the transpose of the Jacobian matrix of the gradient of the function $f$:

$$\mathbf{H}(f(\mathbf{x}))=\mathbf{J}(\nabla f(\mathbf{x}){)}^T$$