Recovering Private Text in Federated Learning of Language Models
Federated learning allows distributed users to collaboratively train a model
while keeping each user's data private. Recently, a growing body of work has
demonstrated that an eavesdropping attacker can effectively recover image data
from gradients transmitted during federated learning. However, little progress
has been made in recovering text data. In this paper, we present a novel attack
method FILM for federated learning of language models -- for the first time, we
show the feasibility of recovering text from large batch sizes of up to 128
sentences. Different from image-recovery methods which are optimized to match
gradients, we take a distinct approach that first identifies a set of words
from gradients and then directly reconstructs sentences based on beam search
and a prior-based reordering strategy. The key insight of our attack is to
leverage either prior knowledge in pre-trained language models or memorization
during training. Despite its simplicity, we demonstrate that FILM can work well
with several large-scale datasets -- it can extract single sentences with high
fidelity even for large batch sizes and recover multiple sentences from the
batch successfully if the attack is applied iteratively. We hope our results
can motivate future work in developing stronger attacks as well as new defense
methods for training language models in federated learning. Our code is
publicly available at https://github.com/Princeton-SysML/FILM.