Federated learning is a recent advance in privacy protection. 
 In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to share the data. 
 However, the protocol is vulnerable to differential attacks, which could originate from any party contributing during federated optimization. In such an attack, a client's contribution during training and information about their data set is revealed through analyzing the distributed model. 
 We tackle this problem and propose an algorithm for client sided differential privacy preserving federated optimization. The aim is to hide clients' contributions during training, balancing the trade-off between privacy loss and model performance. 
 Empirical studies suggest that given a sufficiently large number of participating clients, our proposed procedure can maintain client-level differential privacy at only a minor cost in model performance. Lately, the topic of security in machine learning is enjoying increased interest. This can be largely attributed to the success of big data in conjunction with deep learning and the urge for creating and processing ever larger data sets for data mining. However, with the emergence of more and more machine learning services becoming part of our daily lives, making use of our data, special measures must be taken to protect privacy. Unfortunately, anonymization alone often is not sufficient BID12 ; BID1 and standard machine learning approaches largely disregard privacy aspects and are susceptible to a variety of adversarial attacks BID11 . In this regard, machine learning can be analyzed to recover private information about the participating user or employed data as well ; BID16 ; BID3 ; BID6 . BID2 propose a measure for to assess the memorization of privacy related data. All the aspects of privacy-preserving machine learning are aggravated when further restrictions apply such as a limited number of participating clients or restricted communication bandwidth such as mobile devices Google (2017) .In order to alleviate the need of explicitly sharing data for training machine learning models, decentralized approaches have been proposed, sometimes referred to as collaborative BID15 or federated learning BID9 . In federated learning BID9 a model is learned by multiple clients in decentralized fashion. Learning is shifted to the clients and only learned parameters are centralized by a trusted curator. This curator then distributes an aggregated model back to the clients. However, this alone is not sufficent to preserve privacy. In BID14 it is shown that clients be identified in a federated learning setting by the model updates alone, necessitating further steps.Clients not revealing their data is an advance in privacy protection. However, when a model is learned in conventional way, its parameters reveal information about the data that was used during training. In order to solve this issue, the concept of differential privacy (dp) BID4 for learning algorithms was proposed by BID0 . The aim is to ensure a learned model does not reveal whether a certain data point was used during training.We propose an algorithm that incorporates a dp-preserving mechanism into federated learning. However, opposed to BID0 we do not aim at protecting w.r.t. a single data point only. Rather, we want to ensure that a learned model does not reveal whether a client participated during decentralized training. This implies a client 's whole data set is protected against differential attacks from other clients.Our main contributions: First, we show that a client's participation can be hidden while model performance is kept high in federated learning. We demonstrate that our proposed algorithm can achieve client level differential privacy at a minor loss in model performance. An independent study BID10 , published at the same time, proposed a similar procedure for client level-dp. Experimental setups however differ and BID10 also includes elementlevel privacy measures. Second, we propose to dynamically adapt the dp-preserving mechanism during decentralized training. Empirical studies suggest that model performance is increased that way. This stands in contrast to latest advances in centralized training with differential privacy, were such adaptation was not beneficial. We can link this discrepancy to the fact that, compared to centralized learning, gradients in federated learning exhibit different sensibilities to noise and batch size throughout the course of training. As intuitively expected, the number of participating clients has a major impact on the achieved model performance. For 100 and 1000 clients, model accuracy does not converge and stays significantly below the non-differentially private performance. However, 78% and 92% accuracy for K ∈ {100, 1000} are still substantially better than anything clients would be able to achieve when only training on their own data. In domains where K lays in this order of magnitude and differential privacy is of utmost importance, such models would still substantially benefit any client participating. An example for such a domain are hospitals. Several hundred could jointly learn a model, while information about a specific hospital stays hidden. In addition, the jointly learned model could be used as an initialization for further client-side training.For K = 10000, the differentially private model almost reaches accuracies of the non-differential private one. This suggests that for scenarios where many parties are involved, differential privacy comes at almost no cost in model performance. These scenarios include mobile phones and other consumer devices.In the cross-validation grid search we also found that raising m t over the course of training improves model performance. When looking at a single early communication round, lowering both m t and σ t in a fashion such that σ 2 t /m t stays constant, has almost no impact on the accuracy gain during that round. however, privacy loss is reduced when both parameters are lowered. This means more communication rounds can be performed later on in training, before the privacy budget is drained. In subsequent communication rounds, a large m t is unavoidable to gain accuracy, and a higher privacy cost has to be embraced in order to improve the model. This observation can be linked to recent advances of information theory in learning algorithms. As observable in FIG3 , BID17 suggest, we can distinguish two different phases of training: label fitting and data fitting phase.During label fitting phase, updates by clients are similar and thus V c is low, as FIG3 shows. U c , however, is high during this initial phase, as big updates to the randomly initialized weights are performed. During data fitting phase V c rises. The individual updates w k look less alike, as each client optimizes on their data set. U c however drastically shrinks, as a local optima of the global model is approached, accuracy converges and the contributions cancel each other out to a certain extend. FIG3 shows these dependencies of V c and U c .We can conclude: i) At early communication rounds, small subsets of clients might still contribute an average update w t representative of the true data distribution ii ) At later stages a balanced (and therefore bigger) fraction of clients is needed to reach a certain representativity for an update. iii ) High U c makes early updates less vulnerable to noise. We were able to show through first empirical studies that differential privacy on a client level is feasible and high model accuracies can be reached when sufficiently many parties are involved. Furthermore, we showed that careful investigation of the data and update distribution can lead to optimized privacy budgeting. For future work, we plan to derive optimal bounds in terms of signal to noise ratio in dependence of communication round, data representativity and between-client variance as well as further investigate the connection to information theory. Additionally, we plan to further investigate the dataset dependency of the bounds. For assessing further applicability in bandwith-limited settings, we plan to investigate the applicability of proposed approach in context of compressed gradients such as proposed by BID8 .
Ensuring that models learned in federated fashion do not reveal a client's participation.
{"src": "scitldr-AIC_train", "paper_id": "SkVRTj0cYQ"}