Orthogonality Splits Space

An inner product space gives us a way to say that two directions do not overlap. That is what orthogonality means:

This one equation is the source of orthogonal complements, orthogonal projections, and the clean part of spectral representation. Orthogonality lets a vector be split into independent pieces.

Perpendicular means no shared component

Let be a subspace of an inner product space . The orthogonal complement is the set of all vectors perpendicular to every vector in :

So is not merely “another direction”. It is the collection of directions that have no component inside .

In finite dimensions, the cleanest case is

Each subspace stays separate: a vector in cannot be built from vectors in , except for the zero vector. Instead, the whole space is rebuilt by adding one component from each side.

The symbol also records uniqueness: every vector has exactly one decomposition

The two pieces do not interfere. One lives in ; the other is perpendicular to all of .

Projection chooses the kept part

An orthogonal projection onto is the operator that keeps the -part and removes the perpendicular part:

Here and . The vector is the shadow of on ; the vector is the leftover error, and it is perpendicular to .

So projection is not “dropping a vector down” by convention. It is the unique split

Orthogonality tells us which leftover is legitimate: the leftover must have no component in .

The closest point condition

The projection is also the closest point in to . The reason is geometric but algebraic.

Take any other . Since , the difference lies in . The residual lies in . Therefore the two pieces are orthogonal:

Then the squared distance from to splits by Pythagoras:

The second term is non-negative, so the smallest distance occurs at .

This is why the projected point is not arbitrary. It is the best approximation to inside .

Operator view: kept means eigenvalue 1, removed means eigenvalue 0

The projection is a linear operator. Its behaviour is completely simple on the two perpendicular parts:

So is the eigenspace for eigenvalue , and is the eigenspace for eigenvalue :

This is the spectral meaning of projection. Projection separates the space into two perpendicular eigenspaces and applies the scalars and .

Spectral representation is many projections at once

An orthogonal projection is the smallest spectral picture:

A spectral representation generalises this idea. Instead of only two parts, a nice operator splits the space into mutually orthogonal eigenspaces:

On each part, the operator is just scaling:

If denotes the orthogonal projection onto , then

The projections separate the vector into orthogonal spectral components. The eigenvalues say how much each component is scaled.

So orthogonal projection is the training-wheel version of spectral representation. It teaches the core move: split the vector into perpendicular components, then act on each component independently.