Derivative of matrix inversion function? Let's say I have a function $f$ which maps any invertible $n\times n$ matrix to its inverse. How do I calculate the derivative of this function?

To calculate the directional derivatives of the map, at a fixed matrix $A$:
Suppose you change entry $i,j$ of your matrix $A$ by adding a small $t$ (small enough that the result is still invertible). Let $E_{i,j}$ be the matrix with a $1$ in the $i,j$th position and $0$ elsewhere. 
Note that 
$$
(A + tE_{i,j})(A + tE_{i,j})^{-1} = I
$$
so on one hand, the derivative with respect to $t$ is zero (as the identity is constant), on the other hand, the derivative can be written
$$
\frac{d(A + tE_{i,j})}{dt}(A + tE_{i,j})^{-1} + (A + tE_{i,j})\frac{d(A + tE_{i,j})^{-1}}{dt} = 0
$$
so 
$$
\frac{d(A + tE_{i,j})^{-1}}{dt} = -(A + tE_{i,j})^{-1}\frac{d(A + tE_{i,j})}{dt}(A + tE_{i,j})^{-1}\\ = -(A + tE_{i,j})^{-1}E_{i,j}(A + tE_{i,j})^{-1}
$$
