Motivated by the above robustness properties of 01 loss we propose a 01 loss dual layer neural network as a defense against adversarial attacks. Computationally 01 loss presents a considerable challenge because it is NP-hard to solve [1]}. Previous attempts [2]}, [3]}, [4]}, [5]}, [6]} lack on-par test accuracy with convex solvers and are slow and impractical for large image benchmarks. However, a recent stochastic coordinate descent method for linear 01 loss models [7]} has shown to attain comparable accuracies to state of the art linear solvers like the support vector machine. Thus we extend the coordinate descent as an optimizer to train our network.
