Hadamard product (matrices)

In mathematics, the Hadamard product (also known as the element-wise, entrywise or Schur product) is a binary operation that takes two matrices of the same dimensions and produces another matrix of the same dimension as the operands, where each element is the product of elements of the original two matrices. It is to be distinguished from the more common matrix product. It is attributed to, and named after, either French mathematician Jacques Hadamard or German mathematician Issai Schur.
The Hadamard product is associative and distributive. Unlike the matrix product, it is also commutative.
Definition.
For two matrices and of the same dimension , the Hadamard product formula_1 (or formula_2) is a matrix of the same dimension as the operands, with elements given by
For matrices of different dimensions ( and , where or ), the Hadamard product is undefined.
Example.
For example, the Hadamard product for a 3 × 3 matrix with a 3 × 3 matrix is
Schur product theorem.
The Hadamard product of two positive-semidefinite matrices is positive-semidefinite. This is known as the Schur product theorem, after Russian mathematician Issai Schur. For two positive-semidefinite matrices and , it is also known that the determinant of their Hadamard product is greater than or equal to the product of their respective determinants:
In programming languages.
Hadamard multiplication is built into certain programming languages under various names. In MATLAB, GNU Octave, GAUSS and HP Prime, it is known as "array multiplication", or in Julia "broadcast multiplication", with the symbol codice_1. In Fortran, R, APL, J and Wolfram Language (Mathematica), it is done through simple multiplication operator codice_2, whereas the matrix product is done through the function codice_3, codice_4, codice_5, codice_6 and the codice_7 operators, respectively.
In Python with the NumPy numerical library, multiplication of objects as codice_8 produces the Hadamard product, and multiplication as codice_9 produces the matrix product. With the SymPy symbolic library, multiplication of objects as both codice_8 and codice_9 will produce the matrix product, the Hadamard product can be obtained with codice_12.
In C++, the Eigen library provides a codice_13 member function for the class (codice_14), while the Armadillo library uses the operator codice_15 to make compact expressions (codice_16; codice_17 is a matrix product). R package matrixcalc introduces function codice_18 for Hadamard Product of numeric matrices or vectors.
Applications.
The Hadamard product appears in lossy compression algorithms such as JPEG. The decoding step involves an entry-for-entry product, in other words the Hadamard product.
It is also used in the machine learning literature, for example to describe the architecture of recurrent neural networks as GRUs or LSTMs.
Analogous operations.
Other Hadamard operations are also seen in the mathematical literature, namely the "Hadamard root" and "Hadamard power" (which are in effect the same thing because of fractional indices), defined for a matrix such that:
For
and for
The "Hadamard inverse" reads:
A "Hadamard division" is defined as:
The penetrating face product.
According to the definition of V. Slyusar the penetrating face product of the "p"×"g" matrix formula_16 and "n"-dimensional matrix formula_17 ("n" > 1) with "p"×"g" blocks (formula_18) is a matrix of size formula_17 of the form:
Example.
If
then
Main properties.
where formula_25 denotes the face-splitting product of matrices,
Applications.
The penetrating face product is used in the tensor-matrix theory of digital antenna arrays. This operation can also be used in artificial neural network models, specifically convolutional layers.