@inproceedings{
tsilivis2024the,
title={The Price of Implicit Bias in Adversarially Robust Generalization},
author={Nikolaos Tsilivis and Natalie Frank and Nathan Srebro and Julia Kempe},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=h1grUs6CjN}
}
We study the implicit bias of optimization in robust empirical risk minimization (robust ERM) and its connection with robust generalization.  In classification settings under adversarial perturbations with linear models, we study what type of regularization should ideally be applied for a given perturbation set to improve (robust) generalization. We then show that the implicit bias of optimization in robust ERM can significantly affect the robustness of the model and identify two ways this can happen; either through the optimization algorithm or the architecture. We verify our predictions in simulations with synthetic data and experimentally study the importance of implicit bias in robust ERM with deep neural networks.
The Price of Implicit Bias in Adversarially Robust Generalization
Nikolaos Tsilivis Natalie Frank Nathan Srebro Julia Kempe