Subgradient

analysis optimisation convex-analysis

Definition

Subgradient

Let be a convex function on a convex set , and let . A subgradient of at is a vector such that

The affine function is a global lower supporting plane for at . The set of all such vectors is the subdifferential .

Derivation

For a differentiable convex function, the tangent line at lies below the whole function:

The gradient is therefore the slope of a global affine lower bound. At a corner, there is no single tangent slope, but there may be many supporting slopes. A subgradient keeps exactly the part of the gradient that survives the corner: it is any slope whose affine line still stays below the function everywhere.

This is why the formula is forced. Given convexity, the world is shaped like a bowl; asking for a first-order linear approximation that never cuts through the bowl gives the inequality

The vector is not merely a local hint. It defines a global supporting hyperplane to the epigraph of the function.

Relation to the Gradient

If is a differentiable convex function at , then the subdifferential contains exactly one vector:

So, for differentiable convex functions, the subgradient is the gradient.

The converse intuition needs care. A differentiable non-convex function has a gradient, but that gradient need not satisfy the global subgradient inequality. In that setting, the gradient is a local slope, not necessarily a subgradient in the convex-analysis sense.

Optimisation Meaning

For convex minimisation, a subgradient points toward increase in the linear lower model. Therefore is used as a descent direction in subgradient methods.

The optimality condition is especially clean:

At a smooth minimum, this says . At a corner minimum, it says that the zero vector lies among the supporting slopes.

Example

For on ,

At the corner , every slope between and defines a line that touches the graph at the origin and stays below everywhere.