matrix differentiation - derivative of matrix vector dot product with respect to matrix Given the function
$$f(N) = x_1^T M x_2 $$
where 


*

*$x_1 = Nv_1 $

*$x_2 = Nv_2 $

*$x_1, x_2, v_1, v_2$ are vectors with dimension $n \times 1$

*$M$ and $N$ are matrices with dimension $n \times n$


what's the derivative of $f(N)$ with respect to $N$?

Generalizing the problem slightly, to use matrix (instead of vector) variables, $X_k = N\cdot V_k$.
We can write the function as
$$ \eqalign {
  f &= X_1 : M\cdot X_2 \cr
} $$
Now take the differential and expand
$$ \eqalign {
 df &= dX_1 : M\cdot X_2 + X_1 : M\cdot dX_2 \cr
    &= d(N\cdot V_1) : M\cdot X_2 + X_1 : M\cdot d(N\cdot V_2) \cr
    &= dN\cdot V_1 : M\cdot X_2 + X_1 : M\cdot dN\cdot V_2 \cr
    &= dN : M\cdot X_2\cdot V_1^T + M^T\cdot X_1\cdot V_2^T : dN \cr
    &= [M\cdot X_2\cdot V_1^T + M^T\cdot X_1\cdot V_2^T] : dN \cr
    &= [M\cdot N\cdot V_2\cdot V_1^T + M^T\cdot N\cdot V_1\cdot V_2^T] : dN \cr
} $$
The derivative is therefore
$$ \eqalign {
 \frac {\partial f} {\partial N} &= M\cdot N\cdot V_2\cdot V_1^T + M^T\cdot N\cdot V_1\cdot V_2^T \cr
} $$
If you dislike the Frobenius product, you can change the above derivation to use the trace instead
$$ \eqalign {
 A : B &= \text{tr}(A^T\cdot B) \cr
} $$
