Tensor Product

linear-algebra

Definition

Tensor Product

Let $A=(a_{ij})$ be an $m\times n$-matrix and $B=(b_{kl})$ a $p\times q$-matrix. Then the tensor product $A\otimes B$ is the $(mp)\times (nq)$-matrix obtained by replacing each entry $a_{ij}$ of $A$ with the block $a_{ij}B$.

Equivalently, the entries of $A\otimes B$ are given by

$$(A\otimes B{)}_{(i,k),(j,l)}=a_{ij}{b}_{kl}$$

for $1\le i\le m$, $1\le j\le n$, $1\le k\le p$, and $1\le l\le q$.

Properties

\begin{alignat*}{3} \text{I.}\qquad &\lambda (v \otimes w) &=\;& (\lambda v) \otimes w &=\;& v \otimes (\lambda w) \qquad &\lambda \in \mathbb{C} \\ \text{II.}\qquad & (v + v') \otimes w &=\;& v \otimes w + v' \otimes w \\ \text{III.}\qquad & v \otimes (w + w') &=\;& v \otimes w + v \otimes w' \end{alignat*}

Einsum

Einsum notation can express the tensor product by keeping both pairs of indices separate:

$$(A\otimes B{)}_{(i,k),(j,l)}=A_{ij}{B}_{kl}$$

In code, this is similar to

einsum('ij,kl->ikjl', A, B)

The result can then be reshaped into the usual matrix form of the tensor product.

Example

Example

For

$$A=\left(\begin{array}{cc}1& 2\end{array}\right),B=\left(\begin{array}{cc}0& 1\\ 3& 4\end{array}\right),$$

the tensor product is

$$A\otimes B=\left(\begin{array}{cc}1B& 2B\end{array}\right)=\left(\begin{array}{cc}1\left(\begin{array}{cc}0& 1\\ 3& 4\end{array}\right)& 2\left(\begin{array}{cc}0& 1\\ 3& 4\end{array}\right)\end{array}\right)=\left(\begin{array}{cccc}0& 1& 0& 2\\ 3& 4& 6& 8\end{array}\right).$$