Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data
We propose Compressed Vertical Federated Learning (C-VFL) for
communication-efficient training on vertically partitioned data. In C-VFL, a
server and multiple parties collaboratively train a model on their respective
features utilizing several local iterations and sharing compressed intermediate
results periodically. Our work provides the first theoretical analysis of the
effect message compression has on distributed training over vertically
partitioned data. We prove convergence of non-convex objectives at a rate of
$O(\frac{1}{\sqrt{T}})$ when the compression error is bounded over the course
of training. We provide specific requirements for convergence with common
compression techniques, such as quantization and top-$k$ sparsification.
Finally, we experimentally show compression can reduce communication by over
$90\%$ without a significant decrease in accuracy over VFL without compression.