Measuring Representational Robustness of Neural Networks Through Shared Invariances
A major challenge in studying robustness in deep learning is defining the set
of ``meaningless'' perturbations to which a given Neural Network (NN) should be
invariant. Most work on robustness implicitly uses a human as the reference
model to define such perturbations. Our work offers a new view on robustness by
using another reference NN to define the set of perturbations a given NN should
be invariant to, thus generalizing the reliance on a reference ``human NN'' to
any NN. This makes measuring robustness equivalent to measuring the extent to
which two NNs share invariances, for which we propose a measure called STIR.
STIR re-purposes existing representation similarity measures to make them
suitable for measuring shared invariances. Using our measure, we are able to
gain insights into how shared invariances vary with changes in weight
initialization, architecture, loss functions, and training dataset. Our
implementation is available at: \url{https://github.com/nvedant07/STIR}.