Non-\(\operatorname{\mathbf {\ell }}_p\)  adversarial robustness.
[1]} propose Lagrangian-style adversarial training in the input space and in the last layer of the network.
[2]} propose on-manifold adversarial training which is performed in the latent space of a VAE-GAN generative model. However, its success crucially depends on the quality of the generative model which could not be scaled beyond simple image recognition datasets.
[3]} derive generalization bounds that motivate adversarial training with respect to all network layers which they use to improve \(\operatorname{\mathbf {\ell }}_p\)  robustness.
Recently, [4]} provided algorithms for approximate perceptual adversarial training based on the LPIPS distance [5]} which is defined via activations of a neural network. They aim at improving robustness against new types of adversarial perturbations that were unseen during training.
