Show that $\text{rank}(Df)(A) = \frac{n(n+1)}{2}$ for all $A$ such that $A^TA = I_n$ 
We identify $\mathbb R^{n \times n}$ with $\mathbb R^{n^2}$ and define $f:\mathbb R^{n^2} \to \mathbb R^{n^2}, A \mapsto A^TA$. 
Show that $\text{rank}(Df)(A) = \frac{n(n+1)}{2}$ for all $A$ such that $A^TA = I_n$.

The solution by Omnomnomnom written in a more accessible (for me) way:
We firstly note that $$f(A+H)=(A + H)^T(A + H) = A^TA  + H^TA + A^TH +H^TH$$
Consider the following homomorphismus $\Phi:\mathbb R^{n^2} \to \mathbb R^{n^2}, H \mapsto H^TA + A^TH$
Thus the equation above can be written as:
$$f(A+H)= f(A)  + \Phi(H) +H^TH$$
Remember that according to the definition $(Df)(A)$ is a homomorphismus from $\mathbb R^{n^2}$ to $\mathbb R^{n^2}$ such that $$\lim_{H\to 0, H \ne 0} \frac{f(A+H)-f(A)-(Df)(A)H}{\|H\|}=0$$
Because $\lim_{H\to 0, H \ne 0}\|\frac{H^TH}{\|H\|}\| \le \lim_{H\to 0, H \ne 0}\frac{\|H^T\|\|H\|}{\|H\|}=\lim_{H\to 0, H \ne 0}\|H^T\|=0$, it follows that:
$$\lim_{H\to 0, H \ne 0} \frac{f(A+H)-f(A)-\Phi (H)}{\|H\|}=\lim_{H\to 0, H \ne 0} \frac{H^TH}{\|H\|}=0$$ and consequently  $Df(A)=\Phi$.
Now we note that $\{\Phi(H):H \in \mathbb R^{n^2}\}=\{X\in \mathbb R^{n^2} : X^T = X\}$, because:
1) $(H^TA + A^TH)^T = A^TH^{TT} + H^TA^{TT} = H^TA + A^TH$
2) $S$ is symmetric, then $\Phi(\frac{1}{2}AS)=\frac{1}{2}(SA^TA+A^TAS)=S$
Now it follows:
$$\frac{n(n+1)}{2}=\dim \{X\in \mathbb R^{n^2} : X^T = X\}=\dim\{\Phi(H):H \in \mathbb R^{n^2}\}=\text{rank}\Phi=\text{rank}(Df)(A)$$

A geometrical proof. It suffices to prove that, if $A\in O(n)$, then $dim\ker(Df_A)=n(n-1)/2$. Note that $Df_A(H)=0$ iff $H^TA+A^TH=0$ (Serguey, learn to differentiate a product...), that is, $H$ is in the tangent space $T_A$ to $O(n)$ in $A$. Thus $\ker(Df_A)=T_A$ (it is because $O(n)$ is a level curve of $f$). Since $O(n)$ is a group, $T_A\sim T_I$, the set of skew-symmetric matrices -which is a vector space of dimension $n(n-1)/2$-; and we are done...
EDIT. We can generalize this result; in fact, $O(n)$ has nothing to do here.
Proposition. Let $A\in GL_n(\mathbb{R})$; then $rank(Df_A)=n(n+1)/2$, that is, the function $A\in GL_n(\mathbb{R})\rightarrow A^TA\in S_n^{++}$ is a submersion in each invertible matrix.
Proof. As above, $H\in \ker(Df_A)$ iff $A^TH\in SK_n$, the set of skew-symmetric matrices. Then $\ker(Df_A)=A^{-T}(SK_n)$, a vector space of dimension $n(n-1)/2$ and we are done.
