Understanding Robust Learning through the Lens of Representation Similarities
Representation learning, i.e. the generation of representations useful for
downstream applications, is a task of fundamental importance that underlies
much of the success of deep neural networks (DNNs). Recently, robustness to
adversarial examples has emerged as a desirable property for DNNs, spurring the
development of robust training methods that account for adversarial examples.
In this paper, we aim to understand how the properties of representations
learned by robust training differ from those obtained from standard, non-robust
training. This is critical to diagnosing numerous salient pitfalls in robust
networks, such as, degradation of performance on benign inputs, poor
generalization of robustness, and increase in over-fitting. We utilize a
powerful set of tools known as representation similarity metrics, across three
vision datasets, to obtain layer-wise comparisons between robust and non-robust
DNNs with different architectures, training procedures and adversarial
constraints. Our experiments highlight hitherto unseen properties of robust
representations that we posit underlie the behavioral differences of robust
networks. We discover a lack of specialization in robust networks'
representations along with a disappearance of `block structure'. We also find
overfitting during robust training largely impacts deeper layers. These, along
with other findings, suggest ways forward for the design and training of better
robust networks.