We should point out that the condition (REF ) excludes the promising stochastic line search method, as studied in [1]}, [2]}.
This is because there is no clear decaying trend for random stepsize that is selected by line search.
On the other hand, line search step requires to generate more and more samples to have a precise estimation for the gradient and Hessian, which is inapplicable under our fully stochastic setup (i.e., we only generate a single sample per iteration).
