Hybrid Architectures for Distributed Machine Learning in Heterogeneous Wireless Networks
The ever-growing data privacy concerns have transformed machine learning (ML)
architectures from centralized to distributed, leading to federated learning
(FL) and split learning (SL) as the two most popular privacy-preserving ML
paradigms. However, implementing either conventional FL or SL alone with
diverse network conditions (e.g., device-to-device (D2D) and cellular
communications) and heterogeneous clients (e.g., heterogeneous
computation/communication/energy capabilities) may face significant challenges,
particularly poor architecture scalability and long training time. To this end,
this article proposes two novel hybrid distributed ML architectures, namely,
hybrid split FL (HSFL) and hybrid federated SL (HFSL), by combining the
advantages of both FL and SL in D2D-enabled heterogeneous wireless networks.
Specifically, the performance comparison and advantages of HSFL and HFSL are
analyzed generally. Promising open research directions are presented to offer
commendable reference for future research. Finally, primary simulations are
conducted upon considering three datasets under non-independent and identically
distributed settings, to verify the feasibility of our proposed architectures,
which can significantly reduce communication/computation cost and training
time, as compared with conventional FL and SL.