In FL, instead of centrally collecting \(\mathcal {D}_{\rm train}\)  to draw a random batch during training, the training set \(\mathcal {D}_{\rm train}\)  is distributed across multiple clients and the model \(f_{\mathbf {w}}\)  is stored on a central server. At each iteration, the model parameter \(\mathbf {w}\)  is transmitted to each client to compute the per-sample gradients \(\lbrace  \nabla _\mathbf {w}\ell (f_{\mathbf {w}}(\mathbf {x}_i), y_i) \rbrace _{i=1}^B\)  locally over a set of clients. The server and clients then execute a federated aggregation protocol to compute the average gradient for the gradient descent update. A major advantage of FL is data privacy since clients do not need to disclose their data explicitly, but rather only send their gradient \(\nabla _\mathbf {w}\ell (f_{\mathbf {w}}(\mathbf {x}_i), y_i)\)  to the server. Techniques such as secure aggregation [1]} and differential privacy [2]}, [3]} can further reduce the privacy leakage from sending this gradient update.
