Hierarchical Federated Learning with Privacy
Federated learning (FL), where data remains at the federated clients, and
where only gradient updates are shared with a central aggregator, was assumed
to be private. Recent work demonstrates that adversaries with gradient-level
access can mount successful inference and reconstruction attacks. In such
settings, differentially private (DP) learning is known to provide resilience.
However, approaches used in the status quo (\ie central and local DP) introduce
disparate utility vs. privacy trade-offs. In this work, we take the first step
towards mitigating such trade-offs through {\em hierarchical FL (HFL)}. We
demonstrate that by the introduction of a new intermediary level where
calibrated DP noise can be added, better privacy vs. utility trade-offs can be
obtained; we term this {\em hierarchical DP (HDP)}. Our experiments with 3
different datasets (commonly used as benchmarks for FL) suggest that HDP
produces models as accurate as those obtained using central DP, where noise is
added at a central aggregator. Such an approach also provides comparable
benefit against inference adversaries as in the local DP case, where noise is
added at the federated clients.