A Novel Optimized Asynchronous Federated Learning Framework

Federated Learning (FL) since proposed has been applied in many fields, such as credit assessment, medical, etc. Because of the difference in the network or computing resource, the clients may not update their gradients at the same time that may take a lot of time to wait or idle. That's why Asynchronous Federated Learning (AFL) method is needed. The main bottleneck in AFL is communication. How to find a balance between the model performance and the communication cost is a challenge in AFL. This paper proposed a novel AFL framework VAFL. And we verified the performance of the algorithm through sufficient experiments. The experiments show that VAFL can reduce the communication times about 51.02\% with 48.23\% average communication compression rate and allow the model to be converged faster. The code is available at \url{https://github.com/RobAI-Lab/VAFL}


I. INTRODUCTION
Machine learning and artificial intelligence need to be fed lots of data to get a well-performed model, which cased the demand of data that is stored in every device. For instance, smart city [1], medical [2], and credit assessment [3]. But, this data is sensitive. At the same time, the relationship between data is often isolated and fragmented. This limits its application for processing data.
In this context, federated learning was proposed. Federated learning is a privacy-preserving distributed machine learning technique. The data holders only need to provide a local model which was trained using their local data to interact with the central server. In this paradigm, only gradients need to be transmitted rather than the raw data, which can effectively reduce privacy risks. In federated learning, the client downloads the global model from the central server and trains the local data under the coordination of the central server. After a round of training is completed, the trained local model is uploaded to the central server for weighted aggregation to complete the update. After some iterations, the performance of the model is similar to the results of centralized machine learning. However, there are some challenges in federated learning [3] [4]. Traditional federated learning has obvious shortcomings in flexibility and scalability. These challenges are shown below: • Heterogeneous device: Since training data is stored on the client-side, heterogeneous device client used to have different data sizes, computing power, network, etc., and are affected by unstable communication conditions, the efficiency of traditional synchronous federated learning can be affected by the presence of dropped users. For example, when a few clients are disconnected due to network problems, other clients and server have to wait for them, causing the federated learning process slowly and spend lots of time idle. • Heterogeneous data: Since federated learning data are only stored locally, the inconsistency of this distribution among data in practical applications leads to significant degradation of model performance, such as feature distribution skew, label distribution skew, quantity skew [5], [6]. • Security risk: Even though federated learning does not involve the client raw data in the transmission, there is still a considerable security risk. For example, attackers may use GAN to learn potential information in the gradients [7]. In order to address these issues and challenges, FedAvg [8], FedProx [9], SkewScout [10], and other algorithms have been proposed one after another. Besides, asynchronous federated learning was proposed. Asynchronous federated learning mainly addresses the problem of how the central server parameters should be aggregated in the presence of any heterogeneous clients (heterogeneous data distribution, computing resources, network conditions, etc.). In this paper, we propose a new asynchronous federated learning optimization algorithm (VAFL) for the above challenges (mainly the first two). To address the first challenge, we consider the number of participating clients and provide an improved method for judging clients by checking obsolescence, introducing a function for evaluating the communication value of clients so that random selection is no longer performed. By selecting high-quality clients, the global model accuracy is improved. For the second challenge, we make a division between IID and Non-IID on the datasets, aiming to cope with practical federated learning scenarios, and explore applications in Non-IID environments.
In summary, our contributions are as follows: • We proposed a novel asynchronous federated learning algorithm VAFL, which can reduce the communication cost and the idle time and make the model converge faster. • We set up evaluation formulas for client selection to determine which clients upload the model. Experiment results show that this method accelerates the accuracy obtained at the beginning of training, and it also ensures that the communication compression rate can be improved to a certain extent. • We perform extensive experiments and validate the performance of VAFL.

II. RELATED WORK
In recent years, the Federated Learning framework proposed by Google has attracted great attention from academic scholars. The standard federated learning model needs to learn a single global statistical model from the stored data of a large number of remote devices. However, the standard model faces challenges such as high communication costs, system heterogeneity, statistical heterogeneity, and privacy issues [11].
In response to these challenges, McMahan [8]proposed a method for synchronous training of the Federated learning Average Algorithm (FedAvg), and recently Q Li [12] proposed MOON based on FedAvg. These algorithms are updated locally. The cost of communication is reduced; not only that, privacy protection is based on synchronization operations such as differential privacy [13]and secure aggregation [14] can solve the privacy problem to a certain extent.
However, many types of research on federated learning today are mainly about Non-IID data sets and data imbalances, and most of the researches are based on synchronization algorithms. However, federated learning based on synchronization algorithms cannot solve the problem of excessive communication costs. Issues such as differences in terminal performance due to different equipment terminals.
In response to the above challenges, many researchers have begun to try asynchronous algorithms for federated learning. For example, EAFLM proposed by Lu [15], this method can achieve the goal of reducing communication costs without searching for the optimal solution; SS Diwangkara [16]implemented it through an asynchronous aggregation algorithm The rapid convergence of each node and the reduction of the update frequency of the parameters can not only reduce the communication cost, but the method also achieves good results in the Non-IID data set. Although the above methods can reduce the communication cost better, they did not consider the serious delay caused by the lagging party. For this reason, McMahan B [8] proposed the use of multiple SGD updates and batch processing clients to alleviate the delay. Problem; Subsequently, Nishio T [17] suggested using the deadline method to solve the customer selection problem. Although this method can filter out some slow-response customers, it did not consider how to solve the model accuracy rate caused by the backwardness in training. Too low customers, for this reason, Chai [18] proposed a hierarchical-based federated learning system (TIFL), which can divide customers into different levels according to the user's response delay in the federated system. To select customers based on the system, the system can not only solve the problem of the user falling behind but also further alleviate the problem of Non-IID data. However, the TIFL system adaptively adjusts the client's participation based on the response delay of the client's machine, and it cannot solve some clients with high participation but low data quality.
Aiming at the excessively high communication cost in the federated learning training and the lagging customers, we propose an optimized federated learning asynchronous algorithm based on the communication value calculation of the gradient, the number of clients, and the accuracy of the model(VAFL). Different from the method of Chai [18], we use the client's gradient change, the accuracy rate on the test set, and the number of clients to evaluate the client's communication value, thereby determining whether the client can continue to participate in federated optimization. At the same time, considering the problem of low performance of client devices in reality, when we set up the experimental platform, we used a Raspberry Pi with poor performance to simulate client devices.

A. Communication Value Calculation
This section illustrates how VAFL is derived and worked. The communication value calculation is based on gradient, the number of clients, and model accuracy. The notations covered in this section are shown in Tab. I. The ith client's communication value The ith client's gradient ∇ k i Gradient at the kth round of training for the ith client Acc Accuracy of client models on the testset Acc i Accuracy of the ith client model on the testset n

Number of clients involved in federated learning
We refer to the work of Chen et al. [19], where the main idea of Chen et al.'s work is that not all communication back and forth between the server and the clients is equally important. Selecting only the important clients for communication reduces the communication cost and also maintains the convergence speed. Instead, our idea is to let the more communication-valuable clients communicate with the server, which we define as V , where the communication value of the ith client is denoted as V i .
How to determine the communication value of the client then becomes an important factor in the design of the algorithm. After following the prior work about the calculation of communication value, we decided to use the gradient of the clients' model as a variable in VAFL design. We use ∇ i to represent it. Considering that the model performance of each client reflects some extent the performance of the client and the combination of the datasets it has (in general, the more quality datasets it has, the better the performance of the model obtained by local training of the client will have better performance). For convenience, we choose an easy metric: the accuracy of the client model on the test set, as a variable for VAFL design. We define it as Acc, and The accuracy of the ith client model is Acc i . Recently, it has also been demonstrated that special client selection methods can optimize the federated learning process and improve the performance of the final model [20] [21]. Therefore, we choose the number of clients involved in federated learning as another variable in the design of VAFL. We define it as N ; Ultimately, we designed the evaluation formula for V i : Where ∇ k−1 i −∇ k i is the gradient difference of the model in the last two training processes of the ith client model. We use it to check whether the model is "old". This difference is smaller, indicating that the model is older, which means its V is smaller; n is the number of clients involved in federated learning, we divide it by 10 3 and plus 1 as the base of the power function with Acc i as the variable. Thus, as the number of clients involved in federated learning increases, the relative communication value of individual clients is further differentiated. The more valuable the client, the further the value will increase, while the less valuable the client, the further the value will decrease. To facilitate the design of VAFL.

B. An Asynchronous Federated Optimization Algorithm
This section introduce the design idea and algorithm flow of our asynchronous federated optimization algorithm based on client communication value.
We hope to compress the communication by measuring the communication value of each client participating in federated learning to decide whether they participate in federated optimization or not, so as to deal with the problem of communication bottleneck and unbalanced in asynchronous federated learning.
For the clients involved in federated learning, we calculate their communication value V by using Equation 1, and the server will be able to learn the V of each client. Add it to the asynchronous FedAvg only if the client's V satisfies the Formula 2.
The specific algorithm pseudo-code is shown in Algorithm 1

IV. EXPERIMENT SETTING
In this section, we will introduce how we built our experiment system and how VAFL is implemented.

A. Hardware Platform
VAFL are designed for varying edge devices. Following this principle, we used five Raspberry Pi devices and two laptops to build our experiment system for federated learning for edge computing, which is more realistic, compared to the simulated asynchronous environment. Among them, the laptop as the central server is configured with a 6core Intel(R) Core(TM) i7-9750H CPU running at 2.59GHz

Algorithm 1 V based Asynchronous FedAvg
N is the total number of clients , θ t is the parameters of current model, θ t i is the parameters of current model with index i, B is the batch size, E is the number of local epochs, S is a set of all clients model and η is the learning rate.
1: Server Update 2: Initialize θ 0 ; 3: for each round t = 1, 2, · · · do 4: for each i ∈ S do 5: // clients update model on local device and upload the V i to server; 26: return V to server with 8GB of memory, and the rest of the devices as edge clients, where the laptop is configured with a 4-core Intel(R) Core(TM) i5-9300H CPU running at 2.40GHz and 8GB of memory, one Raspberry Pi device is Raspberry Pi 4B, which is configured with a 4-core ARM Cortex-A72 running at 1.50GHz and 4GB of memory, and the remaining four Raspberry Pi devices are Raspberry Pi 4B with 8GB of memory.
All devices are connected to the same LAN with a network bandwidth of 2.4GHz and the speed for receiving and transmitting are 216 Mbps and 120 Mbps.

B. Framework
We use PySyft [22] as the basic framework to build our asynchronous federated learning experiment system. PySyft is a Python library for Federated Learning, Differential Privacy, and Encrypted Computation. We use the client class and server class provided by PySyft for connecting via WebSocket to facilitate the construction of our federated learning system. We use PyTorch as the deep learning backend which is supported by PySyft.
We run our client program on each client (a total of five Raspberry Pi devices and one laptop, which cannot be informed of each other's existence and ensures a certain degree of security and privacy between the clients). Each client is connected to the server, which can only be informed about the model of each client and the total number of clients. The server uses VAFL to update the model, and then returns the model obtained by the algorithm to the client, and the client uses the model to predict. The architecture of our experiment system is shown in Fig. 1.
We build our federated learning system using PySyft 0.2.4 with Python 3.7. We use ResNet [23] as the core part of model based on PyTorch 1.4.0, and its network structure is schematically shown in Fig. 2

C. Data
We constructed IID and Non-IID data distributed to each client using MNIST and conducted the experiments separately. For the IID case, the training set is equally distributed to all clients, each client contains samples with 10 labels. For the Non-IID dataset distribution, the training set varies in terms of labels and the number of samples with different labels. Some clients containing all labels and a large number of samples under each label, and some clients containing only a small number of labels and some samples under each label.
By constructing such a dataset, we hope to further explore the effect of whether the data is IID on the asynchronous federated learning system.

D. EAFLM
The main idea of EAFLM [15] is to ignore the "lazy" nodes in rounds and only communicate with the "hardworking" nodes. We were inspired by this and improved, so we choose Lu's algorithm for comparison. The difference between VAFL and EAFLM is that EAFLM selected the parameters and gradient of the model as the main factors for the design of the algorithm, whose algorithm is mainly as follows.
Where ∇ i θ k−1 is the k−1 th round gradient calculated by client i based on the k − 1 th round parameter. θ k−1 is the k − 1th round parameter of the server. β, ξ d and D are constant coefficients. α (0, 1) is the adjustable parameter in the parameter weights, which determines the decay rate. As α increases, the decay rate of the parameter weights increases. In the experiments of this paper, ξ d = 1 D , D = 1, α = 0.98.

V. EXPERIMENT AND ANALYSIS
In this section, we conduct four experiments, setting 3 or 7 clients with iid or non-iid data, to validate our proposed asynchronous federated optimization algorithm VAFL.

A. Experiment Metrics
We use the accuracy rate (Acc) and communication compression rate (CCR) as the evaluation metrics of the experiments. Where Acc is the highest Acc rate in multiple experiments and the communication compression rate (CCR) is calculated as: Where C t0 is the communication times before compression and C t1 is the communication times after compression.
The communication compression rate reflects how much the communication between the clients and the server is compressed, and the larger the compression rate, the higher the degree of compression. Although excessive compression rate reduces communication loss and saves time, it is usually accompanied by a decrease in model Acc.
Our experiments use two metrics to evaluate the effectiveness of VAFL, which is to achieve a certain communication compression rate while ensuring the loss of model Acc. The parameters for each of VAFL are shown in Tab. II.

B. Experiment
We conduct four experiments, setting 3 or 7 clients with iid or non-iid data.
• Experiment a: 3 clients with IID data. • Experiment b: 7 clients with data. • Experiment c: 3 clients with Non-IID data. • Experiment d: 7 clients with Non-IID data.
In the experiments of 3 client, we launch the client program to connect to the server on 3 Raspberry Pi devices, one of which has 4 GB of memory. And each client's training set have 20,000 samples. In experiments of 7 client, we launch the client program to connect to the server on 5 Raspberry Pi devices and 1 laptop (launched 2 processes). And each client's training set have 10,000 samples. The rest of the hardware configuration and software system have been mentioned in Section IV.
The neural network we use has been mentioned in Section IV also. We perform image classification experiments to verify whether our federated learning asynchronous training process can converge and the effectiveness of VAFL. We compare the three methods, ordinary asynchronous training, VAFL method, and EAFLM. The Acc of the federated model during the training period in four experiments are shown in Fig. 4.
We also compare the number of communications and the communication compression rate experienced by the three methods in training the model to achieve 94% Acc. The experimental results are shown in Tab. III.

C. VAFL Performance in Multiple Experiments
We also compared the performance of VAFL itself in different experiments. In the four experiments a, b, c, and d, the Acc of each client when executing VAFL are shown in Fig. 5. And the Acc when executing VAFL in different experiments is shown in the Fig. 6. The Communication times and the CCR are shown in the Tab. III. As the number of clients increases and the imbalance in the distribution of the dataset intensifies, the better VAFL performs.

D. Summary
Combining the results of the above experiments, it can be seen that compared with the basic asynchronous algorithm, VAFL enables the model to be trained faster, its Acc can be improved faster in a short period. Also, it can reduce the communication time of 51.02% and reach an average communication compression rate of 48.26%. Compared with EAFLM, VAFL reduces the number of communications  by about 3.23% and improves the compression rate of communications by an average of about 0.82%. In summary, VAFL is able to compress the communication to a large extent, reduce the communication burden between the client and the server, and improve model convergence speed compared to the common asynchronous training methods. Compared with EAFLM, VAFL is able to compress the communication to a better level and reduce the communication times during training period. And VAFL performs better as the number of clients increases and the imbalance in the distribution of the data set intensifies.

VI. CONCLUSION
In this paper. We proposed VAFL, a federated learning algorithm, focusing on the limitation and imbalance of the communication in asynchronous federated learning. The main idea of VAFL is that we expect to evaluate the communication values of each client to decide whether each one of them can participate in the optimization to compress communication values between server and clients. This study has designed a series of experiments aimed to identify the algorithm improving the efficiency of federated learning of edge devices compared with general algorithms and EAFLM, but also including some deficiencies. VAFL enables the model to be trained faster and can reduce the communication time of 51.02% and reach an average communication compression rate of 48.26%. But VAFL relies heavily on honest clients because the calculation of V i depends on the clients themselves. If the clients report V i incorrectly, this will exacerbate the unfairness between the data and clients.
Future work can introduce other reference factors to make the computed communication value more reflective of the client's real situation and make sure the value is trustworthy, such as local training for a certain amount of time before computing the communication value or setting up some mechanism to make the computation of the client communication value less frequent.
The approach proposed in this paper has potential applications in the fields of Internet of Things [24], robotics [25] [26] [27], and smart cities [1]. In addition, the idea of value assessment may be applied in the incentive mechanism of Federated learning [28] [29].