Kronecker product

In mathematics, the Kronecker product, sometimes denoted by ⊗, is an operation on two matrices of arbitrary size resulting in a block matrix. It is a generalization of the outer product (which is denoted by the same symbol) from vectors to matrices, and gives the matrix of the tensor product with respect to a standard choice of basis. The Kronecker product is to be distinguished from the usual matrix multiplication, which is an entirely different operation. The Kronecker product is also sometimes called matrix direct product.
The Kronecker product is named after the German mathematician Leopold Kronecker (1823–1891), even though there is little evidence that he was the first to define and use it. The Kronecker product has also been called the "Zehfuss matrix", and the "Zehfuss product", after , who in 1858 described this matrix operation, but Kronecker product is currently the most widely used.
Definition.
If A is an matrix and B is a matrix, then the Kronecker product is the block matrix:
more explicitly:
Using formula_3 and formula_4 to denote truncating integer division and remainder, respectively, and numbering the matrix elements starting from 0, one obtains formula_5 and formula_6 For the usual numbering starting from 1, one obtains
formula_7
and
formula_8
If A and B represent linear transformations and , respectively, then represents the tensor product of the two maps, .
Examples.
Similarly:
Properties.
Relations to other matrix operations.
) = \exp(\mathbf{N}) \otimes \exp(\mathbf{M}) </math>
Kronecker sums appear naturally in physics when considering ensembles of non-interacting systems. Let "Hi" be the Hamiltonian of the "i"th such system. Then the total Hamiltonian of the ensemble is
Matrix equations.
The Kronecker product can be used to get a convenient representation for some matrix equations. Consider for instance the equation , where A, B and C are given matrices and the matrix X is the unknown. We can use the "vec trick" to rewrite this equation as
Here, vec(X) denotes the vectorization of the matrix X, formed by stacking the columns of X into a single column vector.
It now follows from the properties of the Kronecker product that the equation has a unique solution, if and only if A and B are invertible .
If X and C are row-ordered into the column vectors u and v, respectively, then 
The reason is that
Applications.
For an example of the application of this formula, see the article on the Lyapunov equation. This formula also comes in handy in showing that the matrix normal distribution is a special case of the multivariate normal distribution. This formula is also useful for representing 2D image processing operations in matrix-vector form.
Another example is when a matrix can be factored as a Hadamard product, then matrix multiplication can be performed faster by using the above formula. This can be applied recursively, as done in the radix-2 FFT and the Fast Walsh–Hadamard transform. Splitting a known matrix into the Hadamard product of two smaller matrices is known as the "nearest Kronecker product" problem, and can be solved exactly by using the SVD. To split a matrix into the Hadamard product of more than two matrices, in an optimal fashion, is a difficult problem and the subject of ongoing research; some authors cast it as a tensor decomposition problem.
In conjunction with the least squares method, the Kronecker product can be used as an accurate solution to the hand eye calibration problem.
Related matrix operations.
Two related matrix operations are the Tracy–Singh and Khatri–Rao products, which operate on partitioned matrices. Let the matrix A be partitioned into the blocks A"ij" and matrix B into the blocks B"kl", with of course , , and .
Tracy–Singh product.
The Tracy–Singh product is defined as
which means that the ("ij")-th subblock of the product is the matrix , of which the ("k")-th subblock equals the matrix . Essentially the Tracy–Singh product is the pairwise Kronecker product for each pair of partitions in the two matrices.
For example, if A and B both are partitioned matrices e.g.:
we get:
Face-splitting product.
Mixed-products properties
where formula_19 denotes the Face-splitting product
Similarly:
where formula_23 and formula_24 are vectors,
where formula_23 and formula_24 are vectors, formula_28 denotes the Hadamard product
Similarly:
where formula_31 is vector convolution and formula_32 is the Fourier transform matrix (this result is an evolving of count sketch properties),
where formula_34 denotes the Column-wise Khatri–Rao product.
Similarly:
where formula_23 and formula_24 are vectors