zPROBE: Zero Peek Robustness Checks for Federated Learning
Privacy-preserving federated learning allows multiple users to jointly train
a model with coordination of a central server. The server only learns the final
aggregation result, thereby preventing leakage of the users' (private) training
data from the individual model updates. However, keeping the individual updates
private allows malicious users to perform Byzantine attacks and degrade the
model accuracy without being detected. Best existing defenses against Byzantine
workers rely on robust rank-based statistics, e.g., the median, to find
malicious updates. However, implementing privacy-preserving rank-based
statistics is nontrivial and unscalable in the secure domain, as it requires
sorting of all individual updates. We establish the first private robustness
check that uses high break point rank-based statistics on aggregated model
updates. By exploiting randomized clustering, we significantly improve the
scalability of our defense without compromising privacy. We leverage the
derived statistical bounds in zero-knowledge proofs to detect and remove
malicious updates without revealing the private user updates. Our novel
framework, zPROBE, enables Byzantine resilient and secure federated learning.
Empirical evaluations demonstrate that zPROBE provides a low overhead solution
to defend against state-of-the-art Byzantine attacks while preserving privacy.