First, different from [1]}, [2]} but following from [3]}, [4]}, [5]}, [6]}, we study a fully stochastic setup where only a single sample is generated in each iteration for estimating the objective gradient. However, those fully stochastic schemes did not estimate the objective Hessian, and simply let the modified Hessian \(B_t\)  be identity matrix in the experiments. Differently, we estimate the objective Hessian (with the same sample). This extra computation is necessary for local analysis even for deterministic problems [7]}. We emphasize that our modified Hessian \(B_t\)  is constructed based on the averaged Hessian \(\frac{1}{t}\sum _{i=0}^{t-1}{\bar{\nabla }}_{{x}}^2\mathcal {L}_i\)  excluding the \(t\) -th iteration, i.e., not based on a single noisy Hessian estimate. This choice of Hessian is more effective in practice (e.g., a recent study on stochastic Newton method has demonstrated the effectiveness of such Hessian averaging [8]}), and it is critical for our analysis. In particular, \(B_t\)  is deterministic conditional on \(({x}_t, {\lambda }_t)\) , and converges to \(\nabla _{{x}}^2\mathcal {L}^\star \)  if \(({x}_t, {\lambda }_t)\)  converges to \(({x}^\star , {\lambda }^\star )\) .
