How does one evaluate the derivative of a matrix with a tensor $\frac{\partial \operatorname{Tr}[A(\mathrm{Id}\otimes w)]}{\partial w}$? I am stuck on the following:
$$\frac{\partial \operatorname{Tr}[A(\mathrm{Id}\otimes w)]}{\partial w}=\text{ ?}$$
with $A$ a $d\times d^2$ matrix, $\mathrm{Id}$ the identity matrix of $d\times d$ dimension and $w$ a $d\times 1$ vector.  
THANKS!!!

Using the technique from this question, first calculate the Kronecker factorization $$A^T=Y\otimes z$$where $(Y,z)$ are shaped like $(I,w)$ respectively.
Using this factorization and the Frobenius (:) Inner Product you can rewrite the function, then find its differential and gradient 
$$\eqalign{
  f &= A^T:(I\otimes w) \cr\cr
 df &= (Y\otimes z):(I\otimes dw) \cr
    &= (I:Y)\,(z:dw) \cr
    &= {\rm tr}(Y)\,z:dw \cr
\frac{\partial f}{\partial w} &= z\,{\rm tr}(Y) \cr\cr
}$$
In most cases, you'll need a sum of Kronecker factors
$$A^T = \sum_{k=1}^r Y_k\otimes z_k$$
which changes the result slightly
$$\eqalign{
\frac{\partial f}{\partial w} &= \sum_{k=1}^r z_k\,{\rm tr}(Y_k) \cr\cr
}$$
