Beyond reducing the computional cost, stochastic optimization can improve on classical methods in non-convex settings. For instance, suboptimal solutions aren't fixed points for sgd  [1]}, [2]}. Though sgd  requires \(\gamma _k \rightarrow 0\) , it is common to use \(\gamma _k = \gamma \)  fixed in practice. [3]} consider \(Q_n\)  quadratic with \(\gamma \)  and \(m\)  fixed and find that it leads to estimation bias which needs to be corrected. Few results are available for inference, notably [4]} consider the computation of standard errors for sgd. This paper considers \(\gamma \)  fixed with \(Q_n\)  non-quadratic but requires \(m \rightarrow \infty \) . For inference, \(P_k\)  is either a Hessian or the new quasi-Newton update. The connection with bootstrap inference and the AR(1) representation are new.
