The abundance of data constitutes the fuel for the proliferation of Machine Learning (ML) in all aspects of life. In many applications, data are being generated in large quantities in a distributed fashion: for instance, crowdsourcing, mobile phones, autonomous vehicles, medical centers, distributed sensors in smart grids, to name but a few. The sheer size of the data alongside the urge to protect personal privacy prevent traditional distributed machine learning practices from being directly applied. The well-studied framework of data-center distributed learning [1]}, [2]} often involves maneuvering raw data from machine to machine, which not only violates privacy restrictions but also becomes infeasible when limited networking resources fail to cope with large data sizes. Besides, the rapid increase of the computational power of personal devices such as smartphones surges pushing computation to the edge as opposed to the cloud.
