FedML is structurally similar to distributed optimization for statistical learning [1]}, [2]}, [3]} but differs in a number of important ways. Foremost, in FedML there is no control over how the data is distributed over the participating computational nodes. Hence, data cannot be assumed to be balanced or IID across nodes. This is in stark contrast to distributed learning where the modelling party is in full control of the data partitioning and is able to devise optimal partitions for convergence and load balancing. Moreover, whereas distributed learning typically occurs on reliable cluster infrastructure with high network performance and low failure rate, in FedML clients can become unavailable at any time, and communication between client and server takes place over a high-latency network (the internet). Consequently, how to optimally balance local training iterations with global synchronization to avoid poor convergence and to minimize communication rounds are central research questions [4]}, [5]}, [6]}. Benchmark suits targeted at the exploration of federated learning algorithms have also been proposed [7]}. In addition to new algorithms, real-world federated learning requires a robust, resilient, and highly scalable distributed system capable of handling both the dimension of scaling to a large number of edge-clients and scaling to large model sizes. The design and implementation of such a framework is the main contribution of this paper. FEDn aims to be easy-to-use and agnostic when it comes to the ML-framework used by clients while supporting high-performance federated training in real distributed settings. It is thus well-suited as a tool for research that bridges the machine learning aspects of the problem and the systems aspect of the problem, as well as for running federated learning deployments in production.
