Directional Derivative

analysis

Definition

Directional Derivative

Let $D\subseteq {\mathbb{R}}^n$ be an open set, let $f:D\to \mathbb{R}$ be a totally differentiable function, let $\mathbf{x}\in D$, and let $\mathbf{u}\in {\mathbb{R}}^n$.

The directional derivative of $f$ at $\mathbf{x}$ in direction $\mathbf{u}$ is

$$D_{\mathbf{u}}f(\mathbf{x})=\underset{t\to 0}{\lim }\frac{f(\mathbf{x}+t\mathbf{u})-f(\mathbf{x})}{t},$$

provided the limit exists.

It measures the first-order rate of change of $f$ at $\mathbf{x}$ when one moves from $\mathbf{x}$ in direction $\mathbf{u}$.

Gradient form

If $f$ is totally differentiable at $\mathbf{x}$, then the directional derivative is given by:

$$D_{\mathbf{u}}f(\mathbf{x})=\nabla f(\mathbf{x}{)}^T\mathbf{u}.$$

So the directional derivative is the dot product of the gradient with the direction vector.

Unit directions

If $\mathbf{u}$ is a unit vector, then $D_{\mathbf{u}}f(\mathbf{x})$ is the rate of change per unit distance.

If $\mathbf{u}$ is not a unit vector, its length also scales the value:

$$D_{\mathbf{u}}f(\mathbf{x})=\Vert \mathbf{u}\Vert D_{\mathbf{u}/\Vert \mathbf{u}\Vert }f(\mathbf{x}).$$

Relation to partial derivatives

The partial derivatives are special cases of directional derivatives.

For the standard basis vectors ${\mathbf{e}}_1,\dots ,{\mathbf{e}}_n$,

$$D_{{\mathbf{e}}_i}f(\mathbf{x})=\frac{\partial f}{\partial x_i}(\mathbf{x}).$$

So partial derivatives only look along the coordinate axes, while directional derivatives allow arbitrary directions.

Relation to JVP

For a scalar-valued function $f:{\mathbb{R}}^n\to \mathbb{R}$, the Jacobian-vector product is exactly the directional derivative:

$$J_f(\mathbf{x})\mathbf{u}=D_{\mathbf{u}}f(\mathbf{x}).$$

This is because the Jacobian of a scalar-valued function is the transpose of the gradient.

Geometric meaning

The gradient points in the direction of steepest increase.

• If $D_{\mathbf{u}}f(\mathbf{x})>0$, then $f$ increases in direction $\mathbf{u}$.
• If $D_{\mathbf{u}}f(\mathbf{x})<0$, then $f$ decreases in direction $\mathbf{u}$.
• If $D_{\mathbf{u}}f(\mathbf{x})=0$, then $f$ has no first-order change in that direction.

For a unit vector $\mathbf{u}$,

$$D_{\mathbf{u}}f(\mathbf{x})=\Vert \nabla f(\mathbf{x})\Vert \cos \theta ,$$

where $\theta$ is the angle between $\nabla f(\mathbf{x})$ and $\mathbf{u}$.

So the directional derivative is largest when $\mathbf{u}$ points in the same direction as the gradient.

Examples

Quadratic function

Let

$$f(x,y)=x^2+xy.$$

Then

$$\nabla f(x,y)=\left[\begin{array}{c}2x+y\\ x\end{array}\right].$$

At the point $(1,2)$, we get

$$\nabla f(1,2)=\left[\begin{array}{c}4\\ 1\end{array}\right].$$

In direction

$$\mathbf{u}=\left[\begin{array}{c}3\\ 4\end{array}\right],$$

the directional derivative is

$$D_{\mathbf{u}}f(1,2)=\nabla f(1,2{)}^T\mathbf{u}=\left[\begin{array}{cc}4& 1\end{array}\right]\left[\begin{array}{c}3\\ 4\end{array}\right]=16.$$

If we instead use the associated unit vector

$$\hat{\mathbf{u}}=\left[\begin{array}{c}3/5\\ 4/5\end{array}\right],$$

then

$$D_{\hat{\mathbf{u}}}f(1,2)=\left[\begin{array}{cc}4& 1\end{array}\right]\left[\begin{array}{c}3/5\\ 4/5\end{array}\right]=\frac{16}{5}.$$

The two values differ by the length $\Vert \mathbf{u}\Vert =5$.

Partial derivative as a directional derivative

Let

$$f(x,y)=x^2+xy.$$

Choose the direction

$${\mathbf{e}}_1=\left[\begin{array}{c}1\\ 0\end{array}\right].$$

Then

$$D_{{\mathbf{e}}_1}f(x,y)=\nabla f(x,y{)}^T{\mathbf{e}}_1=\left[\begin{array}{cc}2x+y& x\end{array}\right]\left[\begin{array}{c}1\\ 0\end{array}\right]=2x+y.$$

This is exactly

$$\frac{\partial f}{\partial x}(x,y).$$