Federated Learning (FL) refers to learning a high quality global model based on decentralized data storage, without ever copying the raw data. A natural scenario arises with data created on mobile phones by the activity of their users. Given the typical data heterogeneity in such situations, it is natural to ask how can the global model be personalized for every such device, individually. In this work, we point out that the setting of Model Agnostic Meta Learning (MAML), where one optimizes for a fast, gradient-based, few-shot adaptation to a heterogeneous distribution of tasks, has a number of similarities with the objective of personalization for FL. We present FL as a natural source of practical applications for MAML algorithms, and make the following observations. 1) The popular FL algorithm, Federated Averaging, can be interpreted as a meta learning algorithm. 2) Careful fine-tuning can yield a global model with higher accuracy, which is at the same time easier to personalize. However, solely optimizing for the global model accuracy yields a weaker personalization result. 3) A model trained using a standard datacenter optimization method is much harder to personalize, compared to one trained using Federated Averaging, supporting the first claim. These results raise new questions for FL, MAML, and broader ML research. In recent years, the growth of machine learning applications was driven by aggregation of large amounts of data in a datacenter, where a model can be trained using large scale distributed system (Dean et al., 2012; LeCun et al., 2015) . Both the research community and general public are becoming increasingly aware that there is a variety of scenarios where this kind of data collection comes with significant risks, mainly related to notions of privacy and trust. In the presence of user generated data, such as activity on mobile phones, Federated Learning (FL) proposes an alternative approach for training a high quality global model without ever sending raw data to the cloud. The FL system proposed by Google (Bonawitz et al., 2019 ) selects a sample of available devices and sends them a model to be trained. The devices compute an update to the model based on an optimization procedure with locally available data, and the central system aggregates the updates from different devices. Such iteration is repeated many times until the model has converged. The users' training data does not leave their devices. The basic FL algorithm, Federated Averaging (FedAvg) , has been used in production applications, for instance for next word prediction in mobile keyboard (Hard et al., 2018) , which shows that Federated Learning can outperform the best model trained in a datacenter. Successful algorithmic extensions to the central idea include training a differential private model (McMahan et al., 2018) , compression (Konečný et al., 2016b; Caldas et al., 2018a) , secure aggregation (Bonawitz et al., 2017) , and a smaller number of always-participating nodes (Yang et al., 2019) . FL applications generally face non-i.i.d and unbalanced data available to devices, which makes it challenging to ensure good performance across different devices with a FL-trained global model. Theoretical guarantees are only available under restrictive assumptions and for convex objectives, cf. Li et al. (2019b) . In this work, we are interested in personalization methods that adapt the model for data available on each device, individually. We refer to a trained global model as the initial model, and the locally adapted model as the personalized model. Existing FL personalization work directly takes a converged initial model and conducts personalization evaluation via gradient descent . However, in this approach, the training and personalization procedures are completely disconnected, which results in potentially suboptimal personalized models. Meta Learning optimizes the performance after adaptation given few-shot adaptation examples on heterogeneous tasks, and has increasing applications in the context of Supervised Learning and Reinforcement Learning. Model Agnostic Meta Learning (MAML) introduced by Finn et al. (2017) is a solely gradient-based Meta Learning algorithm, which runs in two connected stages; metatraining and meta-testing. Meta-training learns a sensitive initial model which can conduct fast adaptation on a range of tasks, and meta-testing adapts the initial model for a particular task. Both tasks for MAML, and clients for FL, are heterogeneous. For each task in MAML and client in FL, existing algorithms use a variant of gradient descent locally, and send an overall update to a coordinator to update the global model. If we present the FL training process as meta-training in the MAML language, and the FL personalization via gradient descent as meta-testing, we show in Section 2 that FedAvg (McMahan et al., 2017) and Reptile (Nichol et al., 2018) , two popular FL and MAML algorithms, are very similar to each other; see also Khodak et al. (2019) . In order to make FL personalization useful in practice, we propose that the following objectives must all be addressed, simultaneously. (1) Improved Personalized Model -for a large majority of the clients. (2) Solid Initial Model -some clients have limited or even no data for personalization. (3) Fast Convergence -reach a high quality model in small number of training rounds. Typically, the MAML algorithms only focus on objective (1); that was the original motivation in Finn et al. (2017) . Existing FL works usually focus on objectives (2) and (3), and take the personalized performance as secondary. This is largely due to the fact that it was not obvious that getting a solid initial model is feasible or practical if devices are available occasionally and with limited resources. In this work, we study these three objectives jointly, and our main contributions are: • We point out the connection between two widely used FL and MAML algorithms, and interpret existing FL algorithm in the light of existing MAML algorithms. • We propose a novel modification of FedAvg, with two stages of training and fine-tuning, for optimizing the three above objectives. • We empirically demonstrate that FedAvg is already a meta learning algorithm, optimizing for personalized performance, as opposed to quality of the global model. Furthermore, we show that the fine tuning stage enables better and more stable personalized performance. • We observe that different global models with the same accuracy, can exhibit very different capacity for personalization. • We highlight that these results challenge the existing objectives in the FL literature, and motivate new problems for the broader Machine Learning research community. It this work, we argue that in the context of Federated Learning, the accuracy of the global model after personalization should be of much greater interest than it has been. Investigation of the topic reveals close similarities between the fields of Federated Learning and Model Agnostic Meta Learning, and raises new questions for these areas, as well as for the broader Machine Learning community. Challenges for Federated Learning. Framing papers in the area of Federated Learning Konečný et al., 2016a; Li et al., 2019a) , formulate the objective as training of a shared global model, based on a decentralized data storage where each node / client has access to a non-i.i.d sample from the overall distribution. The objective is identical to one the broader ML community would optimize for, had all the data been available in a centralized location. We argue that in this setting, the primary objective should be the adaptation to the statistical heterogeneity present at different data nodes, and demonstrate that the popular FL algorithm, Federated Averaging, does in fact optimize the personalized performance, and while doing so, also improves the performance of the global model. Experiments we perform demonstrate that the algorithm used to train the model has major influence on its capacity to personalize. Moreover, solely optimizing the accuracy of the global model tends to have negative impact on its capacity to personalize, which further questions the correctness of the commonly presented objectives of Federated Learning. Challenges for Model Agnostic Meta Learning. The objectives in the Model Agnostic Meta Learning literature are usually only the model performance after adaptation to given task (Finn et al., 2017) . In this work, we present the setting of Federated Learning as a good source of practical applications for MAML algorithms. However, to have impact in FL, these methods need to also consider the performance of the initial model, 4 as in practice there will be many clients without data available for personalization. In addition, the connectivity constraints in a production deployment emphasize the importance of fast convergence in terms of number of communication rounds. We suggest these objectives become the subject of MAML works, in addition to the performance after adaptation, and to consider the datasets with a natural user/client structure being established for Federated Learning (Caldas et al., 2018b) as the source of experiments for supervised learning. Challenges for broader Machine Learning. The empirical evaluation in this work raises a number of questions of relevance to Machine Learning research in general. In particular, Figure 2 clearly shows that models with similar initial accuracy can have very different capacity to personalize to a task of the same type as it was trained on. This observation raises obvious questions for which we currently cannot provide an answer. How does the training algorithm impact personalization ability of the trained model? Is there something we can measure that will predict the adaptability of the model? Is it something we can directly optimize for, potentially leading to novel optimization methods? These questions can relate to a gap highlighted in Table 2 . While the common measures could suggest the global model is overfitting the training data, this is not true of the personalized model. Transfer Learning is another technique for which our result could inspire a novel solution. It is very common for machine learning practitioners to take a trained model from the research community, replace the final layer with a different output class of interest, and retrain for the new task (Oquab et al., 2014) . We conjecture that the algorithms proposed in the FL and MAML communities, could yield base models for which this kind of domain adaptation would yield better results. Finally, we believe that a systematic analysis of optimization algorithms of the inner-outer structure presented in Algorithm 1 could provide novel insights into the connections between optimization and generalization. Apart from the FL and MAML algorithms, Zhang et al. (2019) recently proposed a method that can be interpreted as outer optimizer in the general algorithm, which improves the stability of a variety of existing optimization methods used as the inner optimizer. A APPENDIX This Appendix contains further details referenced from the main body of the paper. Table 3 summarizes the attempts at fine tuning the model user in main body with different server optimizers. We see that comparing the same client optimizers, Adam consistently provides better and more stable results in terms of initial accuracy. A.2 PER-CLIENT PERSONALIZATION RESULTS Figure 4 visualizes the distribution of initial and personalized accuracies on a per-client basis. Each dot represents a random sample of the test clients used for personalization experiments. Studying this distribution is of great importance, as in practical deployment, degrading a user's experience might incur disproportionate cost, compared to the benefit of comparable improvement. Designing methods that robustly identify the clients below the diagonal line and at least revert to the initial model is worth of future investigation.
Federated Averaging already is a Meta Learning algorithm, while datacenter-trained methods are significantly harder to personalize.
{"src": "scitldr-AIC_train", "paper_id": "BkeaEyBYDB"}