The objective of federated learning [1]} is to train a machine learning model in a distributed fashion without centralized collection of training data. In detail, let \(f_{\mathbf {w}}\)  be the global model parameterized by \(\mathbf {w}\) , and consider a supervised learning setting that optimizes \(\mathbf {w}\)  by minimizing a loss function \(\ell \)  over the training set \(\mathcal {D}_{\rm train}\) : \(\sum _{(\mathbf {x}, y)\in \mathcal {D}_{\rm train}}\ell (f_{\mathbf {w}}(\mathbf {x}), y)\) . In centralized learning this is typically done by computing a stochastic gradient \(\frac{1}{B} \sum _{i=1}^B \nabla _{\mathbf {w}}\ell (f_{\mathbf {w}}(\mathbf {x}_i), y_i)\)  over a randomly drawn batch of data \((\mathbf {x}_1,y_1),\ldots ,(\mathbf {x}_B,y_B)\)  and minimizing \(\ell \)  using gradient descent.
