Synthetic adversarial examples that are crafted to fool a DNN can be used to study its failures [1]}.
These examples are typically generated by directly perturbing the input.
Some previous works offer examples of adversaries leading to generalizable interpretations (e.g. [2]}, [3]}, [4]}, [5]}).
However, there are limitations to what one can learn about a DNN's flaws from synthesized features (e.g. [6]}).
First, synthetic adversarial perturbations are often difficult for a human to describe because they do not resemble natural features.
Second, even when adversarial features are interpretable, it is unclear without additional testing whether they fool a DNN due to their interpretable features or due to hidden motifs (e.g. [7]}, [4]}).
Third, real world failures of DNNs are often due to atypical features or combinations thereof [9]}.
But synthetic adversaries cannot directly help us understand how networks make these kind of mistakes.
<FIGURE><FIGURE>