When tested on these adversarial examples, <cite>Jia and Liang (2017)</cite> showed that even the most 'robust' amongst published models (the Mnemonic Reader (Hu et al., 2017) ) only achieved 46.6% F1 (compared to 79.6% F1 on the regular task).