If \(\alpha = 30 L\sqrt{\frac{m}{P}}\) , we get the statement in Theorem REF . Throughout the proof, we utilize similar techniques as in SCAFFOLD [1]} convergence. We define a set of variables which are useful in the analysis. Algorithm  freezes \({\theta }_k\)  and its gradients if the device is not active. Let's define virtual \(\lbrace \tilde{{\theta }}_k^t\rbrace \)  variables as
\(\tilde{{\theta }}_k^t = \underset{{\theta }}{\mbox{arg}\min }\ L_k({\theta }) -\langle \nabla L_k({\theta }_k^{t-1}),{\theta }\rangle +\frac{\alpha }{2}\Vert {\theta }-{\theta }^{t-1}\Vert ^2 \ \ \forall k\in [m], t > 0\) 
