Derivative of matrix product: is it true that $\frac{d}{dt}(A^TA) = 2A^T \frac{dA}{dt}$? $A$ is a square matrix.  All elements of $A$ depend on a parameter $t$, that is, $a_{ij}=a_{ij}(t)$.  Let $S(A):=A^TA$, and take the derivative of $S$ w.r.t. $t$:
$\displaystyle \frac{dS}{dt}$
Now, pretty clearly
$\displaystyle \frac{dS}{dt} = \frac{dA^T}{dt}A + A^T\frac{dA}{dt}$
But now, can this also be written $\displaystyle \frac{dS}{dt} = 2 A^T \frac{dA}{dt}$  ?
A math text I am working though right now -- if I am reading it right -- implies that this is the case, but I haven't been able to prove it myself.
Thanks.

Short answer : no. 
Think about the $ij$ entry of $A^T A$; it's 
$$
s_{ij} = \sum_k a_{ki} a_{kj}
$$
Take the derivative with respect to $t$ (using primes to denote that) to get
$$
s'_{ij} = \sum_k a_{ki}' a_{kj} + \sum_k a_{ki} a_{kj}'
$$
The claim is that this is just $2 \sum_k a_{ki}' a_{kj}$, after some index-shuffling, and that's true if the matrix is symmetric, and not necessarily true otherwise. 
Details: let's write that out in the case of a $3 \times 3$ matrix, with $i = 1$ and $j = 2$. We have
\begin{align}
s'_{1,2} 
&= \sum_k a_{k1}' a_{k2} + \sum_k a_{k1} a_{k2}'\\
&= (a_{1,1}' a_{1,2} +a_{2,1}' a_{2,2} +a_{3,1}' a_{3,2}) + 
(a_{1,1} a_{1,2}' +a_{2,1} a_{2,2}' +a_{3,1} a_{3,2}') \\ 
\end{align}
Those two don't look equal, do they? Let's try a concrete example. 
\begin{align}
A &= \begin{bmatrix} 1 & t \\ 0 & 2\end{bmatrix} \\
A^t A &= \begin{bmatrix} 1 & 0 \\ t & 2\end{bmatrix} \begin{bmatrix} 1 & t \\ 0 & 2\end{bmatrix} = \begin{bmatrix} 1 & t \\ t & t^2 +4\end{bmatrix} \\
(A^t A)' &= \begin{bmatrix} 0 & 1 \\ 1 & 2t\end{bmatrix} \\
A'^t A &= \begin{bmatrix} 0 & 0 \\ 1 & 0\end{bmatrix} \begin{bmatrix} 1 & t \\ 0 & 2\end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 1 & t\end{bmatrix} \\
A^t A' &= \begin{bmatrix} 1 & 0 \\ t & 2\end{bmatrix} \begin{bmatrix} 0 & 1 \\ 0 & 0\end{bmatrix} = \begin{bmatrix} 0 & 1 \\ 0 & t\end{bmatrix} \\
A'^t A + A^t A' &= \begin{bmatrix} 0 & 1 \\ 1 & 2t\end{bmatrix} 
\end{align}
Clearly the last expression is different from  $2A^t A'$. So the formula is not correct. 
(My apologies for the glib response earlier; I hope my working out the details makes up for it.) 
On the other hand, if the matrix is symmetric, I'm pretty sure everything works out OK. :)
{Actually not -- see comments. The example of $s_{12}'$ makes that pretty clear: $a_{3,1}'$ appears in the left group of three, but neither it nor $a_{1,3}'$ appears in the right group of three.}  
