Find critical points of $\langle Av,v\rangle$ 
Let $A$ be an $n \times n$ real symmmetric matrix. Find the critical points of the function $\langle Av,v\rangle$ restricted to the unit sphere in $\mathbb{R}^n$.

I would think you just use Lagrange multipliers, and $\nabla\langle Av,v\rangle=2Ax$, since $A$ is symmetric. But this only gives the possible locations for the local max/min. So the gradient is zero at these points, but doesn't necessarily include all points were the gradient is zero. Is there something I'm missing here?

You forgot the constraint that $v$ should be on the unit sphere. The Lagrangian is $L(v,\lambda)=\langle Av,v\rangle -\lambda(\langle v,v\rangle -1)$. The gradient wrt $v$ is $2Av-2\lambda v$ and it is 0 when $v$ is an eigenvector to A with eigenvalue $\lambda$. The constraint then gives that eigenvector should be normalized to unit length. The function value is then the eigenvalue. Thus the max is the largest eigenvalue and is obtained in the eigenspace restricted to the unit sphere, in case eigenspace is onedimensional it will just be two antipodal points. Similarly for min and smallest eigenvalue. Any intermediate eigenvalue will be saddle point because moving along a great circle in the plane spanned by the eigenvector with the intermediate eigenvalue and an eigenvector with a larger/smaller eigenvalue will increase/decrease the function value towards the larger/smaller eigenvalue. 
