It is also worth mentioning that this paper only considers a basic stochastic Newton scheme based on (REF ), where we suppose to know exact function and gradient information and solve the Newton system exactly. Some literature allows one to access inexact function values and/or gradients, and/or apply Newton-CG or MINRES to solve the linear system inexactly [1]}, [2]}, [3]}, [4]}. Applying our sample aggregation technique under these setups is promising, but we defer it to future work. The basic scheme purely reflects the benefits of Hessian averaging, which is the main interest of this work. Additionally, some literature deals with non-convex objectives via stochastic trust region methods [5]}, [6]} or stochastic Levenberg-Marquardt methods [7]}. The averaging scheme may not directly apply for these methods due to potential bias brought by Hessian modifications for addressing non-convexity, while our sample aggregation idea is still inspiring. We leave the generalization to non-convex objectives to the future work as well.
