theo1 shows that COLD converges at a linear rate \(\mathcal {O}\big (\max \lbrace \frac{L}{\mu },\frac{1}{(1-\delta )^2\rho }\rbrace \log \frac{1}{\epsilon }\big )\) . In comparison with the (centralized) gradient method of the convergence rate \(\mathcal {O}\big (\frac{L}{\mu }\log \frac{1}{\epsilon }\big )\)  [1]}, we can explicitly quantify how the network and compression affect the rates in terms of two simple quantities, i.e, \(\delta \)  and \(\rho \) .
