Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring
Attributes skew hinders the current federated learning (FL) frameworks from
consistent optimization directions among the clients, which inevitably leads to
performance reduction and unstable convergence. The core problems lie in that:
1) Domain-specific attributes, which are non-causal and only locally valid, are
indeliberately mixed into global aggregation. 2) The one-stage optimizations of
entangled attributes cannot simultaneously satisfy two conflicting objectives,
i.e., generalization and personalization. To cope with these, we proposed
disentangled federated learning (DFL) to disentangle the domain-specific and
cross-invariant attributes into two complementary branches, which are trained
by the proposed alternating local-global optimization independently.
Importantly, convergence analysis proves that the FL system can be stably
converged even if incomplete client models participate in the global
aggregation, which greatly expands the application scope of FL. Extensive
experiments verify that DFL facilitates FL with higher performance, better
interpretability, and faster convergence rate, compared with SOTA FL methods on
both manually synthesized and realistic attributes skew datasets.