Matrices derivative I have a linear product of matrices, I did solve most of it, however, I stop at this component $(X^T W^T D W X)^{-1}$.
    Given that $X$ is $n \times p$ matrix and $D$ is $n\times n$ matrix. $W$ is a diagonal matrix $n\times n$
what is the derivative of this component with respect of $W$.
$\frac{\partial}{\partial W}(X^T W^T D W X)^{-1}$ = ?

For convenience, define $G=X^TWDWX$. 
Since {$W,D$} are diagonal, they are symmetric and therefore $G$ is symmetric, too.
Then your matrix function and its differential are
$$ \eqalign{
  F &= G^{-1} \cr
 dF &= -F\,dG\,F \cr
    &= -FX^T\,d(WDW)\,XF \cr
    &= -FX^T\,(dW)\,DWXF - FX^TWD\,(dW)\,XF \cr
}$$
Apply the vec operation to both sides of the differential expression
$$ \eqalign{
 {\rm vec}(dF) &= -(FX^TWD\otimes FX^T)\,\,{\rm vec}(dW) - (FX^T\otimes FX^TWD)\,\,{\rm vec}(dW) \cr
 df &= -\Big((FX^TWD\otimes FX^T) + (FX^T\otimes FX^TWD)\Big)\,dw \cr
 \frac{\partial f}{\partial w} &= -(FX^TWD\otimes FX^T) - (FX^T\otimes FX^TWD) \cr
}$$
This sort of vec/vec solution is typical for matrix-by-matrix derivatives, unless you're willing to consider $4^{th}$ order tensors.
