A Framework for Understanding Model Extraction Attack and Defense
The privacy of machine learning models has become a significant concern in
many emerging Machine-Learning-as-a-Service applications, where prediction
services based on well-trained models are offered to users via pay-per-query.
The lack of a defense mechanism can impose a high risk on the privacy of the
server's model since an adversary could efficiently steal the model by querying
only a few `good' data points. The interplay between a server's defense and an
adversary's attack inevitably leads to an arms race dilemma, as commonly seen
in Adversarial Machine Learning. To study the fundamental tradeoffs between
model utility from a benign user's view and privacy from an adversary's view,
we develop new metrics to quantify such tradeoffs, analyze their theoretical
properties, and develop an optimization problem to understand the optimal
adversarial attack and defense strategies. The developed concepts and theory
match the empirical findings on the `equilibrium' between privacy and utility.
In terms of optimization, the key ingredient that enables our results is a
unified representation of the attack-defense problem as a min-max bi-level
problem. The developed results will be demonstrated by examples and
experiments.