As we see from Theorem REF , when using constant stepsize in Algorithm REF , the convergence bias has geometric rate while the variance is a constant with size \(\mathcal {O}(\alpha \log (1/\alpha ))\) . This phenomenon is well observed in SA literature [1]}.
