Gradient descent with constraints In order to find the local minima of a scalar function $p(x), x\in \mathbb{R}^3$, I know we can use the gradient descent method:
$$x_{k+1}=x_k-\alpha_k \nabla_xp(x)$$
where $\alpha_k$ is the step size and $\nabla_xp(x)$ is the gradient of $p(x)$.
My question is: what if $x$ must be constrained on a sphere, i.e.,  $\|x_k\|=1$? Then we are actually to find the local minima of $p(x)$ on a sphere. Can gradient descent be applied to constrained optimizations? Can anyone give any suggestions? Thanks.

The sphere is a particular example of a (very nice) Riemannian manifold.
Most classical nonlinear optimization methods designed for unconstrained optimization of smooth functions (such as gradient descent which you mentioned, nonlinear conjugate gradients, BFGS, Newton, trust-regions, etc.) work just as well when the search space is a Riemannian manifold (a smooth manifold with a metric) rather than (classically) a Euclidean space.
The branch of optimization concerned with that topic is called Riemannian optimization or optimization on manifolds. There is a great reference book on the topic that you can access online for free:
Optimization algorithms on matrix manifolds,
P.-A. Absil, R. Mahony, R. Sepulchre,
Princeton university press, 2008.
https://press.princeton.edu/absil
Some of the theory presented in that book (and some more) is implemented in a Matlab toolbox called Manopt (which I develop with a number of collaborators), also available freely online:
http://www.manopt.org
The tutorial happens to start with an example on the sphere, which could be a handy starting point for the question raised by the OP (assuming Matlab is an option).
EDIT:
I wrote an introduction to optimization on manifolds, available here:
http://www.nicolasboumal.net/book
