Federated learning involves jointly learning over massively distributed partitions of data generated on remote devices. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by resource allocation strategies in wireless networks that encourages a more fair accuracy distribution across devices in federated networks. To solve q-FFL, we devise a scalable method, q-FedAvg, that can run in federated networks. We validate both the improved fairness and flexibility of q-FFL and the efficiency of q-FedAvg through simulations on federated datasets. With the growing prevalence of IoT-type devices, data is frequently collected and processed outside of the data center and directly on distributed devices, such as wearable devices or mobile phones. Federated learning is a promising learning paradigm in this setting that pushes statistical model training to the edge (McMahan et al., 2017) .The number of devices in federated networks is generally large-ranging from hundreds to millions. While one can naturally view federated learning as a multi-task learning problem where each device corresponds to a task (Smith et al., 2017) , the focus is often instead to fit a single global model over these distributed devices/tasks via some empirical risk minimization objective (McMahan et al., 2017) . Naively minimizing the average loss via such an objective may disproportionately advantage or disadvantage some of the devices, which is exacerbated by the fact that the data are often heterogeneous across devices both in terms of size and distribution. In this work, we therefore ask: Can we devise an efficient optimization method to encourage a more fair distribution of the model performance across devices in federated networks?There has been tremendous recent interest in developing fair methods for machine learning. However, current methods that could help to improve the fairness of the accuracy distribution in federated networks are typically proposed for a much smaller number of devices, and may be impractical in federated settings due to the number of involved constraints BID5 . Recent work that has been proposed specifically for the federated setting has also only been applied at small scales (2-3 groups/devices), and lacks flexibility by optimizing only the performance of the single worst device (Mohri et al., 2019) .In this work , we propose q-FFL, a novel optimization objective that addresses fairness issues in federated learning. Inspired by work in fair resource allocation for wireless networks, q-FFL minimizes an aggregate reweighted loss parameterized by q such that the devices with higher loss are given higher relative weight to encourage less variance in the accuracy distribution. In addition , we propose a lightweight and scalable distributed method, qFedAvg, to efficiently solve q-FFL, which carefully accounts for important characteristics of the federated setting such as communication-efficiency and low participation of devices BID3 McMahan et al., 2017) . We empirically demonstrate the fairness, efficiency, and flexibility of q-FFL and q-FedAvg compared with existing baselines. On average, q-FFL is able to reduce the variance of accuracies across devices by 45% while maintaining the same overall average accuracy.
We propose a novel optimization objective that encourages fairness in heterogeneous federated networks, and develop a scalable method to solve it.
{"src": "scitldr-AIC_train", "paper_id": "BJzvEqrj2V"}