Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher
Machine unlearning has become an important field of research due to an
increasing focus on addressing the evolving data privacy rules and regulations
into the machine learning (ML) applications. It facilitates the request for
removal of certain set or class of data from the already trained ML model
without retraining from scratch. Recently, several efforts have been made to
perform unlearning in an effective and efficient manner. We propose a novel
machine unlearning method by exploring the utility of competent and incompetent
teachers in a student-teacher framework to induce forgetfulness. The knowledge
from the competent and incompetent teachers is selectively transferred to the
student to obtain a model that doesn't contain any information about the forget
data. We experimentally show that this method is well generalized, fast, and
effective. Furthermore, we introduce a zero retrain forgetting (ZRF) metric to
evaluate the unlearning method. Unlike the existing unlearning metrics, the ZRF
score does not depend on the availability of the expensive retrained model.
This makes it useful for analysis of the unlearned model after deployment as
well. The experiments are conducted for random subset forgetting and class
forgetting on various deep networks and across different application domains. A
use case of forgetting information about the patients' medical records is also
presented.