The second assumption stated in Theorem REF  is similar to the one made in [1]}, [2]}, and is satisfied by linear models and deep neural networks with ReLU activation function and max-pooling. Theorem REF  shows that the NFM loss is approximately an upper bound of the adversarial loss with \(l_2\)  attack of size \(\epsilon ^{mix} = \min _{i \in [n]} \epsilon ^{mix}_i\) , plus a feature-dependent regularization term \(L_n^{reg}\) . Therefore, we see that minimizing the NFM loss not only results in a small adversarial loss, while retaining the robustness benefits of manifold mixup, but it also imposes additional smoothness (due to noise injection) on the adversarial loss. The latter can help mitigate robust overfitting and improve test performance [3]}, [4]}. This offers a plausible explanation for the remarkable performance of NFM (see next section).
