FedEntropy: Efficient Device Grouping for Federated Learning Using Maximum Entropy Judgment
Along with the popularity of Artificial Intelligence (AI) and
Internet-of-Things (IoT), Federated Learning (FL) has attracted steadily
increasing attentions as a promising distributed machine learning paradigm,
which enables the training of a central model on for numerous decentralized
devices without exposing their privacy. However, due to the biased data
distributions on involved devices, FL inherently suffers from low
classification accuracy in non-IID scenarios. Although various device grouping
method have been proposed to address this problem, most of them neglect both i)
distinct data distribution characteristics of heterogeneous devices, and ii)
contributions and hazards of local models, which are extremely important in
determining the quality of global model aggregation. In this paper, we present
an effective FL method named FedEntropy with a novel dynamic device grouping
scheme, which makes full use of the above two factors based on our proposed
maximum entropy judgement heuristic.Unlike existing FL methods that directly
aggregate local models returned from all the selected devices, in one FL round
FedEntropy firstly makes a judgement based on the pre-collected soft labels of
selected devices and then only aggregates the local models that can maximize
the overall entropy of these soft labels. Without collecting local models that
are harmful for aggregation, FedEntropy can effectively improve global model
accuracy while reducing the overall communication overhead. Comprehensive
experimental results on well-known benchmarks show that, FedEntropy not only
outperforms state-of-the-art FL methods in terms of model accuracy and
communication overhead, but also can be integrated into them to enhance their
classification performance.