Few-Shot Unlearning by Model Inversion
We consider the problem of machine unlearning to erase a target dataset,
which causes an unwanted behavior, from the trained model when the training
dataset is not given. Previous works have assumed that the target dataset
indicates all the training data imposing the unwanted behavior. However, it is
often infeasible to obtain such a complete indication. We hence address a
practical scenario of unlearning provided a few samples of target data,
so-called few-shot unlearning. To this end, we devise a straightforward
framework, including a new model inversion technique to retrieve the training
data from the model, followed by filtering out samples similar to the target
samples and then relearning. We demonstrate that our method using only a subset
of target data can outperform the state-of-the-art methods with a full
indication of target data.