One of the future works is to apply our Hessian averaging technique on constrained nonlinear optimization problems. [1]}, [2]} designed various stochastic second-order methods based on sequential quadratic programming (SQP) for solving constrained problems, where the Hessian of the Lagrangian function was estimated by subsampling. These works established the global convergence for stochastic SQP methods, while the local convergence rate of these methods remains unknown. On the other hand, based on our analysis, the local rate of Hessian averaging schemes is induced by the central limit theorem, which we can also apply on the noise of the Lagrangian Hessian oracles. Therefore, it is possible to design a stochastic SQP with Hessian averaging and show a similar local superlinear rate for solving constrained nonlinear optimization problems. Furthermore, for unconstrained convex optimization problems, generalizing our analysis to enable stochastic gradients and function evaluations as well as inexact Newton directions is also an important future work, which can further improve the applicability of the algorithm.
