As further verification of the above distortions we perform substitute model black box attacks on all three models. In this method we try to approximate the target model with a substitute and then generate white box adversaries from the substitute to attack the target model. The success of this method relies upon transferability of adversarial examples between models. We use the standard adversarially augmented training algorithm of Papernot et. al. [1]} to train the substitute. In the Supplementary Material we provide full details of the algorithm.
