Comparing SNAFUE to benchmarks by reconstructing trojans: Finally, we test how SNAFUE compares to other interpretability/debugging tools.
To objectively evaluate how useful interpretability tools are [1]} and [2]} recommend attempting to rediscover known flaws in models.
We do this by implanting and searching for trojans [3]}.
We finetuned the ResNet18 on the ImageNet training set to introduce 4 source-class-conditional trojans.
Details for trojan implantation are in Appendix REF .
The trojan triggers were single smiley face, clownfish, green star, and strawberry images.
Figure REF  shows the four triggers and our attempts to rediscover them with feature visualization [4]}, [5]}, [6]}, adversarial patches [7]}, [8]}, and our copy/paste attacks.
No method reliably and identifiably reproduced the trojans.
Feature visualization with a Fourier parameterization and adversarial patches both failed to produce any visualizations that seem easily human-describable.
On the other hand, feature visualization with a PPN parameterization seemed to somewhat plausibly reproduce features resembling the green star trigger, and robust feature level adversaries did so with the strawberry trigger.
