where \(m\)  is the number of clients and \(f_i(\theta )=\tfrac{\alpha _i}{n_i} \sum _{k=1}^{n_i}l_{ik}(\theta )\)  captures the weighted local training loss. Common choices of weights include \(\alpha _i=\frac{n_i}{n}\) , where \(n=\sum _{i=1}^m n_i\)  (i.e., weighting clients proportionally to their data volumes) or \(\alpha _i = 1\)  (i.e., equal weights, which serves to avoid overfitting to clients with more data, and is also the choice in our experiments). The goal of FL is to solve (REF ) over model parameter \(\theta \)  using data distributed across clients. In contrast to data-center distributed machine learning,
communication is the bottleneck for FL applications. The large number of clients in federated systems renders infeasible synchronous communication (i.e., all clients communicating with the server at a given round) with large message sizes; this is due to bandwidth limitations and the existence of stragglers. Moreover, the large data volumes involved along with the leakage of personal information from raw data prevents the server from collecting local data for centralized processing. Thus, a candidate algorithm for FL should meet the following requirements: (i) communication efficiency; (ii) low client participation rate per round; (iii) handling both system and statistical heterogeneity; (iv) no transmission of client raw data. We proceed to present FedADMM whose design meets all these guidelinesAlthough not the focus of this paper, we note that standard privacy-preserving methods, such as differential privacy and secure multi-party computation [1]}, [2]}, [3]} can be combined with FedADMM..
