Here, we compare NFM to adversarial training in the small noise regime, i.e., the situation where models do not show a significant drop on the clean test set. Specifically, we consider the projected gradient decent (PGD) method [1]} using 7 attack iterations and varying \(l_2\)  perturbation levels \(\epsilon \)  to train adversarial robust models.
First, we compare how resilient the different models are with respect to adversarial input perturbations during inference time (Fig. REF ; left). Again the adversarial examples are constructed using the PGD method with 7 attack iterations. Not very surprisingly, the adversarial trained model with \(\epsilon =0.01\)  features the best resilience while sacrificing about \(0.5\%\)  accuracy as compared to the baseline model (here not shown). In contrast, the models trained with NFM are less robust, while being about \(1-1.5\%\)  more accurate on clean data.
