Distributed offline learning: The study of distributed offline algorithm is relatively mature, such as subgradient-push [1]}, primal-dual [2]}, [3]}, gradient tracking [4]}, [5]}, [6]}, subgradient rescaling [7]}, asynchronous optimization [8]}, [9]}, etc. Such algorithms often use residual error\(\Vert \mathbf {x}\left( k \right) -\mathbf {x}^{\ast } \Vert \)  or \(f\left( \mathbf {x}\left( t \right) \right) -f\left( \mathbf {x}^{\ast } \right)\) , where \(\mathbf {x}\left( k \right)\)  is decision variable, \(\mathbf {x}^{\ast }\)  represents the global optimal point, and \(f\)  is the objective function. as the performance measurement. Generally, the algorithm is said to be elegant if it achieves a linear convergence rate \(\mathcal {O}\left( a^t \right)\) , where \(t\)  denotes the update counter and \(0<a<1\)  is a constant, is established for strongly convex and smooth objective functions while it reach a sublinear rate \(\mathcal {O}( 1/t^2 )\)  for convex objective functions.
