Linear Operations Are Index Wiring

Linear operations feel similar because they are built from the same small moves. An array has indices. An operation tells us which indices to keep, which indices to match, and which matched indices to sum away.

The bottom layer is not mysterious:

$$\text{choose entries}⟶\text{multiply them}⟶\text{optionally sum over a hidden index}.$$

Einstein summation notation makes this visible. Here we use the array convention: a repeated index is summed. An index that remains visible is a coordinate of the output.

Indices are handles

Think of each index as a handle attached to an array. A vector has one handle, a matrix has two, and a higher-order tensor has more.

An operation is a rule for these handles:

• keep an index: it appears in the output,
• pair indices: keep both as a joint coordinate,
• contract an index: match it across factors and sum over it.

The preconditions come from these handles. If an index is contracted, both sides must have the same range. If no index is contracted, the dimensions do not need to match.

The smallest moves

Start with a vector $x\in F^n$ and a scalar $\lambda \in F$.

Scalar multiplication has no hidden index:

$$(\lambda x{)}_i=\lambda x_i.$$

Each coordinate is scaled independently. The index $i$ is kept.

Dot product contracts one index:

$$x_i{y}_i=\sum _{i=1}^n{x}_i{y}_i\in F.$$

The index $i$ appears twice and disappears. The output has no index, so it is a scalar. This is why the dot product requires $x$ and $y$ to have the same length.

Over $\mathbb{C}$, the inner-product version conjugates one side, for example ${\sum }_i\overline{x_i}{y}_i$. The wiring is unchanged; only the entry rule changes.

Outer product keeps both indices:

$$(x{y}^T{)}_{ij}=x_i{y}_j.$$

Over $\mathbb{C}$, the bra-ket version is $(x{y}^{†}{)}_{ij}=x_i\overline{y_j}$.

Nothing is summed. The output has two handles, so it is a matrix. The vector lengths may differ.

The contrast is small but decisive:

$$\begin{array}{rlrlrl}\text{dot product:}& & x_i{y}_i& \to \text{sum over }i& & \to \text{scalar},\\ \text{outer product:}& & x_i{y}_j& \to \text{keep }i,j& & \to \text{matrix}.\end{array}$$

Matrix multiplication is a repeated dot product

Let $A\in F^{m\times n}$ and $B\in F^{n\times p}$. Matrix multiplication is

$$C_{ik}=A_{ij}{B}_{jk}=\sum _{j=1}^n{A}_{ij}{B}_{jk}.$$

The index $j$ is internal. It is the wire between the column handle of $A$ and the row handle of $B$. The indices $i$ and $k$ remain visible, so the output has shape $m\times p$.

So the shape rule is not arbitrary:

$$(m\times n)(n\times p)\to m\times p.$$

The middle $n$ is consumed by the sum. The outside dimensions survive.

Tensor product keeps the worlds separate

The tensor product does not contract. It keeps the indices from both factors.

For vectors,

$$(x\otimes y{)}_{ij}=x_i{y}_j.$$

For matrices,

$$(A\otimes B{)}_{ikjl}=A_{ij}{B}_{kl}.$$

There is no repeated index, so there is no hidden sum. The result remembers both coordinate choices. If $i$ chooses a coordinate in the first space and $k$ chooses one in the second, then $(i,k)$ chooses a coordinate in the combined space.

This is why tensor products are natural for composite systems. They do not say “line these two things up and sum”. They say “carry both coordinate choices at once”.

Vector products are not one product

The phrase “vector product” hides several different index recipes.

operation	index form	output	hidden sum?	main precondition
scalar multiplication	$(\lambda x{)}_i=\lambda x_i$	vector	no	$\lambda \in F$
dot product	$x_i{y}_i$	scalar	yes, over $i$	same length
outer product	$x_i{y}_j$	matrix	no	none between lengths
matrix multiplication	$A_{ij}{B}_{jk}$	matrix	yes, over $j$	inner dimensions match
tensor product	$A_{ij}{B}_{kl}$	tensor/block matrix	no	none between dimensions
cross product	${\epsilon }_{ijk}{x}_j{y}_k$	vector	yes, over $j,k$	dimension $3$

The cross product looks different because it uses the Levi-Civita symbol ${\epsilon }_{ijk}$. That extra tensor encodes orientation and antisymmetry. The result is still an index recipe: sum over hidden indices, keep the free index $i$.

Why the laws differ

The properties of an operation come from how the indices move.

Matrix multiplication is associative because two contractions can be performed in either grouping:

$$(A_{ij}{B}_{jk})C_{k\ell }=A_{ij}(B_{jk}{C}_{k\ell }).$$

Both expressions sum over $j$ and $k$ and keep $i,\ell$. The wiring is the same.

It is not generally commutative because swapping the factors changes the wiring:

$$A_{ij}{B}_{jk}\text{and}{B}_{ij}{A}_{jk}$$

contract different handles and may not even be well-typed.

Tensor product is bilinear because multiplication distributes over addition in each visible slot:

$$(x+x^{\prime }{)}_i{y}_j=x_i{y}_j+x_i^{\prime }{y}_j.$$

It is not a dot product because no index disappears.

The rule of thumb

When an operation feels unclear, rewrite it with indices.

1. Write the entries with their indices.
2. Mark repeated indices as summed.
3. Mark unrepeated indices as the output shape.
4. Check that every summed index has the same range wherever it appears.

The operation then becomes a small piece of wiring. Some wires are left visible. Some wires are tied together and summed away. Most of linear algebra is a disciplined way of doing this without drawing the wires every time.