FedDyn vs. SCAFFOLD [1]}. While SCAFFOLD appears to be similar to our method, there are fundamental differences. Practically, SCAFFOLD communicates twice as many bits as FedDyn or Federated Dynamic Regularizer, transmitting back and forth, both a model and its gradient. The \(2\times \)  increase in bits can be substantial for many low-power IoT applications, since energy consumption for communication dominates computation. Conceptually, we attribute the increased bit-rate to algorithmic differences. At the device-level, our modified risk incorporates a linear term, \(\nabla L_k({\theta }^t_k)\)  (which we can compute readily (Eq. REF )). Applying our perspective to SCAFFOLD, in full participation setting, we see SCAFFOLD as replacing our linear term \(\left\langle \nabla L_k({\theta }^t_k),{\theta }\right\rangle \)  with \(\left\langle \nabla L_k({\theta }^t) - \frac{1}{m} \sum _{k \in [m]} \nabla L_k({\theta }_k^t),{\theta }\right\rangle \) . While \(\nabla L_k({\theta }^t)\)  can be locally computed, after \({\theta }^t\)  is received, the term \(\frac{1}{m}\sum _{k\in [m]} \nabla L_k({\theta }_k^t)\)  is unknown and must be transmitted by the server, leading to increased bit-rate. Note that this is unavoidable, since ignoring this term, leads to freezing device updates (optimizing \(L_k({\theta }) - \left\langle \nabla L_k({\theta }^t),{\theta }-{\theta }^t\right\rangle + \frac{\alpha }{2} \Vert {\theta }- {\theta }^t\Vert ^2\)  results in \({\theta }={\theta }^t\) ). This extra term is a surrogate for \(\nabla \ell ({\theta }^t)\) , which is unavailable. As such we believe that these differences are responsible for FedDyn's improved rate (in rounds) in theory as well as practice.
