differentiation of a matrix X with respect to vec(X) Assume $X$ is a $n\times n$ matrix, I am looking for the solution of
$\frac{\partial X}{\partial vec(X)}$
Anyone that can shed light on this?
$\def\bb{\mathbb}\def\v{{\rm vec}}\def\M{{\rm Mat}}\def\d{{\rm diag}}\def\D{{\rm Diag}}\def\e{\varepsilon}\def\o{{\tt1}}\def\vcal#1{\vec{\cal #1}}\def\m#1{\left[\begin{array}{r}#1\end{array}\right]}\def\p#1#2{\frac{\partial #1}{\partial #2}}$The
transformation between a matrix and its vectorized form can be indicated by functions
$$\eqalign{
x &amp;= \v(X) \quad&amp;\iff\quad &amp;X=\M(x) \\
}$$
or by dot products with special third-order tensors
$$\eqalign{
x &amp;= \vec\nu:X \quad&amp;\iff\quad X&amp;=\vec\mu\cdot x \\
x &amp;= X:\vec\mu \quad&amp;\iff\quad X&amp;=x\cdot\vec\nu \\
}$$
The gradient of a matrix with respect to itself yields the fourth-order identity tensor
$$\vcal E = \frac{\partial X}{\partial X}
\quad\implies\quad \vcal E:dX\;=\;dX\;=\;dX:\vcal E$$
just as the gradient of a vector wrt itself yields the second-order identity matrix
$$I = \frac{\partial x}{\partial x}
\quad\implies\quad I\cdot dx\;=\;dx\;=\;dx\cdot I\quad$$
Combining the above ideas yields
$$\eqalign{
\frac{\partial X}{\partial x} 
  &amp;= \left(\frac{\partial X}{\partial X}\right):\vec\mu 
  &amp;= (\vcal E):\vec\mu  = \vec\mu \\
\frac{\partial x}{\partial X} 
  &amp;= \vec\nu:\left(\frac{\partial X}{\partial X}\right) 
  &amp;= \vec\nu:(\vcal E)  = \vec\nu \\
}$$
This can also be derived using the identity matrix
$$\eqalign{
\frac{\partial X}{\partial x} 
  &amp;= \vec\mu\cdot\left(\frac{\partial x}{\partial x}\right) 
  &amp;= \vec\mu\cdot(I) = \vec\mu \\
\frac{\partial x}{\partial X} 
  &amp;= \left(\frac{\partial x}{\partial x}\right)\cdot\vec\nu 
  &amp;= (I)\cdot\vec\nu  = \vec\nu \\
}$$
For $(m\times n)$ matrices the components of these zero-one tensors are
$$\eqalign{
{\vec\mu}_{jk\ell} &amp;= {\vec\nu}_{\ell jk} \\
{\vec\nu}_{\ell jk} &amp;= \begin{cases}
1\quad{\rm if}\;\;\ell=j+mk-m \\
0\quad{\rm otherwise} \\
\end{cases}
\\
{\vcal E}_{jkpq} &amp;= \begin{cases}
1\quad{\rm if}\;\;j=p\;\;\&amp;\;\;k=q \\
0\quad{\rm otherwise} \\
\end{cases}
\\
I_{\ell r} &amp;= \begin{cases}
1\quad{\rm if}\;\;\ell=r \\
0\quad{\rm otherwise} \\
\end{cases}
\\
}$$
and the index ranges are
$$\eqalign{
1&amp;\le\; j,p  \;&amp;\le m
 &amp;\qquad\big({\rm row\,index}\big) \\
1&amp;\le\; k,q  \;&amp;\le n
 &amp;\qquad\big({\rm column\,index}\big) \\
1&amp;\le\; \ell,r \;&amp;\le mn
 &amp;\qquad\big({\rm vectorized\,index}\big) \\
}$$
In some sense the third-order tensors are the more fundamental quantities,
since given
$\big(\vec\mu,\vec\nu\big)$
the identity tensors can be calculated as
$$\eqalign{
\vcal E &amp;= \vec\mu\cdot\vec\nu
  \qquad&amp;\big({\rm contract\,over\,vectorized\,index}\big) \\
I &amp;= \vec\nu:\vec\mu
  \qquad&amp;\big({\rm contract\,over\,row/col\,indexes}\big) \\
}$$
$\def\bb{\mathbb}\def\v{{\rm vec}}\def\M{{\rm Mat}}\def\d{{\rm diag}}\def\D{{\rm Diag}}\def\e{\varepsilon}\def\o{{\tt1}}\def\vcal#1{\vec{\cal #1}}\def\m#1{\left[\begin{array}{r}#1\end{array}\right]}\def\p#1#2{\frac{\partial #1}{\partial #2}}$The
transformation between a matrix and its vectorized form can be indicated by functions
$$\eqalign{
x &amp;= \v(X) \quad&amp;\iff\quad &amp;X=\M(x) \\
}$$
or by dot products with special third-order tensors
$$\eqalign{
x &amp;= \vec\nu:X \quad&amp;\iff\quad X&amp;=\vec\mu\cdot x \\
x &amp;= X:\vec\mu \quad&amp;\iff\quad X&amp;=x\cdot\vec\nu \\
}$$
The gradient of a matrix with respect to itself yields the fourth-order identity tensor
$$\vcal E = \frac{\partial X}{\partial X}
\quad\implies\quad \vcal E:dX\;=\;dX\;=\;dX:\vcal E$$
just as the gradient of a vector wrt itself yields the second-order identity matrix
$$I = \frac{\partial x}{\partial x}
\quad\implies\quad I\cdot dx\;=\;dx\;=\;dx\cdot I\quad$$
Combining the above ideas yields
$$\eqalign{
\frac{\partial X}{\partial x} 
  &amp;= \left(\frac{\partial X}{\partial X}\right):\vec\mu 
  &amp;= (\vcal E):\vec\mu  = \vec\mu \\
\frac{\partial x}{\partial X} 
  &amp;= \vec\nu:\left(\frac{\partial X}{\partial X}\right) 
  &amp;= \vec\nu:(\vcal E)  = \vec\nu \\
}$$
This can also be derived using the identity matrix
$$\eqalign{
\frac{\partial X}{\partial x} 
  &amp;= \vec\mu\cdot\left(\frac{\partial x}{\partial x}\right) 
  &amp;= \vec\mu\cdot(I) = \vec\mu \\
\frac{\partial x}{\partial X} 
  &amp;= \left(\frac{\partial x}{\partial x}\right)\cdot\vec\nu 
  &amp;= (I)\cdot\vec\nu  = \vec\nu \\
}$$
For $(m\times n)$ matrices the components of these zero-one tensors are
$$\eqalign{
{\vec\mu}_{jk\ell} &amp;= {\vec\nu}_{\ell jk} \\
{\vec\nu}_{\ell jk} &amp;= \begin{cases}
1\quad{\rm if}\;\;\ell=j+mk-m \\
0\quad{\rm otherwise} \\
\end{cases}
\\
{\vcal E}_{jkpq} &amp;= \begin{cases}
1\quad{\rm if}\;\;j=p\;\;\&amp;\;\;k=q \\
0\quad{\rm otherwise} \\
\end{cases}
\\
I_{\ell r} &amp;= \begin{cases}
1\quad{\rm if}\;\;\ell=r \\
0\quad{\rm otherwise} \\
\end{cases}
\\
}$$
and the index ranges are
$$\eqalign{
1&amp;\le\; j,p  \;&amp;\le m
 &amp;\qquad\big({\rm row\,index}\big) \\
1&amp;\le\; k,q  \;&amp;\le n
 &amp;\qquad\big({\rm column\,index}\big) \\
1&amp;\le\; \ell,r \;&amp;\le mn
 &amp;\qquad\big({\rm vectorized\,index}\big) \\
}$$
In some sense the third-order tensors are the more fundamental quantities,
since given
$\big(\vec\mu,\vec\nu\big)$
the identity tensors can be calculated as
$$\eqalign{
\vcal E &amp;= \vec\mu\cdot\vec\nu
  \qquad&amp;\big({\rm contract\,over\,vectorized\,index}\big) \\
I &amp;= \vec\nu:\vec\mu
  \qquad&amp;\big({\rm contract\,over\,row/col\,indexes}\big) \\
}$$