Tensor Product

linear-algebra

Definition

Tensor Product

Let $A=(a_{ij})$ be an $m\times n$-matrix and $B=(b_{kl})$ a $p\times q$-matrix. Then the tensor product $A\otimes B$ is the $(mp)\times (nq)$-matrix obtained by replacing each entry $a_{ij}$ of $A$ with the block $a_{ij}B$.

Equivalently, the entries of $A\otimes B$ are given by

$$(A\otimes B{)}_{(i,k),(j,l)}=a_{ij}{b}_{kl}$$

for $1\le i\le m$, $1\le j\le n$, $1\le k\le p$, and $1\le l\le q$.

Intuition

The tensor product keeps two independent coordinate choices at once.

• A row index becomes a pair $(i,k)$.
• A column index becomes a pair $(j,l)$.
• The entry at the combined position is the product of the corresponding entries:

$$(A\otimes B{)}_{(i,k),(j,l)}=A_{ij}{B}_{kl}.$$

So $A\otimes B$ is not obtained by summing over a shared index, unlike matrix multiplication. It is obtained by pairing indices and multiplying entries.

Vector View

For vectors

$$v=\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right),w=\left(\begin{array}{c}{w}_1\\ w_2\end{array}\right),$$

the tensor product is

$$v\otimes w=\left(\begin{array}{c}{v}_1{w}_1\\ v_1{w}_2\\ v_2{w}_1\\ v_2{w}_2\end{array}\right).$$

For basis vectors, this means that two basis choices combine into one joint basis choice:

$$e_i\otimes f_k\text{has joint index}(i,k).$$

This is why tensor products are used for composite systems: if one system has basis states $e_i$ and another has basis states $f_k$, the combined system has basis states $e_i\otimes f_k$.

Properties

The map

$$\tau :V\times W\to V\otimes W,\tau (v,w)=v\otimes w$$

is bilinear. That means it is linear in each argument separately.

$$\begin{array}{rlrlrl}\text{Homogeneity:}& \lambda (v\otimes w)& =& (\lambda v)\otimes w& =& v\otimes (\lambda w)\\ \text{Additivity (left):}& (v+v^{\prime })\otimes w& =& v\otimes w+v^{\prime }\otimes w& & \\ \text{Additivity (right):}& v\otimes (w+w^{\prime })& =& v\otimes w+v\otimes w^{\prime }& & \end{array}$$

So the listed laws are homogeneity and additivity in each argument, not left-associativity. The associative law for tensor products is a separate statement:

$$(U\otimes V)\otimes W\cong U\otimes (V\otimes W),$$

usually via a canonical isomorphism, not literal equality of the constructed spaces.

Einsum

Einsum notation can express the tensor product by keeping both pairs of indices separate:

$$(A\otimes B{)}_{(i,k),(j,l)}=A_{ij}{B}_{kl}$$

In code, this is similar to

einsum('ij,kl->ikjl', A, B)

The result can then be reshaped into the usual matrix form of the tensor product.

Examples

Example

For

$$A=\left(\begin{array}{cc}1& 2\end{array}\right),B=\left(\begin{array}{cc}0& 1\\ 3& 4\end{array}\right),$$

the tensor product is

$$A\otimes B=\left(\begin{array}{cc}1B& 2B\end{array}\right)=\left(\begin{array}{cc}1\left(\begin{array}{cc}0& 1\\ 3& 4\end{array}\right)& 2\left(\begin{array}{cc}0& 1\\ 3& 4\end{array}\right)\end{array}\right)=\left(\begin{array}{cccc}0& 1& 0& 2\\ 3& 4& 6& 8\end{array}\right).$$