Saddle point escape with access to deterministic gradient. While standard gradient descent can escape saddle points asymptotically [1]}, [2]}, it is known that standard gradient descent may take exponential time to escape saddle points [3]}. Hence, when access to deterministic gradient is available, research has centered on escaping saddle points with adding perturbation [4]}, momentum/acceleration based methods [5]}, [6]}, [7]}, or gradient-based robust Hessian power/curvature exploitation methods [8]}, [9]}. In addition, there has also been work on escaping saddle points devoted to specific optimization settings, such as constrained optimization [10]}, [11]}, optimization of weakly convex functions [12]}, bilevel optimization [13]}, as well as on general manifolds [14]}, [15]}, [16]}.
