In another thread of works, convergence analyses of existing asynchronous
methods have been developed. In [1]}, the authors
analyze the convergence of distributed optimization of smooth convex
loss functions using delayed gradients under the assumption of bounded
gradient delay. They prove that the effect of gradient delay is negligible
asymptotically in the number of processed data points and that the
optimal rate is achievable. In [2]}, the
convergence analysis is extended to general regularized convex cost
functions. In [3]}, the authors study the
convergence of stochastic gradient methods for nonconvex objective
functions and establish an ergodic convergence rate of \({\cal O}(1/\sqrt{T})\) ,
where \(T\)  is the number of iterations.
