Neural reading comprehension models have recently achieved impressive gener- alisation results, yet still perform poorly when given adversarially selected input. Most prior work has studied semantically invariant text perturbations which cause a model’s prediction to change when it should not. In this work we focus on the complementary problem: excessive prediction undersensitivity where input text is meaningfully changed, and the model’s prediction does not change when it should. We formulate a noisy adversarial attack which searches among semantic variations of comprehension questions for which a model still erroneously pro- duces the same answer as the original question – and with an even higher prob- ability. We show that – despite comprising unanswerable questions – SQuAD2.0 and NewsQA models are vulnerable to this attack and commit a substantial frac- tion of errors on adversarially generated questions. This indicates that current models—even where they can correctly predict the answer—rely on spurious sur- face patterns and are not necessarily aware of all information provided in a given comprehension question. Developing this further, we experiment with both data augmentation and adversarial training as defence strategies: both are able to sub- stantially decrease a model’s vulnerability to undersensitivity attacks on held out evaluation data. Finally, we demonstrate that adversarially robust models gener- alise better in a biased data setting with a train/evaluation distribution mismatch; they are less prone to overly rely on predictive cues only present in the training set and outperform a conventional model in the biased data setting by up to 11% F1. Neural networks can be vulnerable to adversarial input perturbations (Szegedy et al., 2013; Kurakin et al., 2016) . In Natural Language Processing (NLP), which operates on discrete symbol sequences, adversarial attacks can take a variety of forms (Ettinger et al., 2017; Alzantot et al., 2018) including character perturbations (Ebrahimi et al., 2018) , semantically invariant reformulations (Ribeiro et al., 2018b; Iyyer et al., 2018b) or-specifically in Reading Comprehension (RC)-adversarial text insertions (Jia & Liang, 2017; Wang & Bansal, 2018) . A model's inability to handle adversarially chosen input text puts into perspective otherwise impressive generalisation results for in-distribution test sets (Seo et al. (2017) ; Yu et al. (2018) ; ; inter alia) and constitutes an important caveat to conclusions drawn regarding a model's language understanding abilities. While semantically invariant text transformations can remarkably alter a model's predictions, the converse problem of model undersensitivity is equally troublesome: a model's text input can often be drastically changed in meaning while retaining the original prediction. In particular, previous works (Feng et al., 2018; Ribeiro et al., 2018a; Sugawara et al., 2018) show that even after deletion of all but a small fraction of input words, models often produce the same output. However, such reduced inputs are usually unnatural to a human reader, and it is both unclear what behaviour we should expect from natural language models evaluated on unnatural text, and how to use such unnatural inputs to improve models. In this work, we show that in RC undersensitivity can be probed with automatically generated natural language questions. In turn, we use these to both make RC models more sensitive when they should be, and more robust in the presence of biased training data. Fig. 1 shows an examples for a BERT LARGE model ) trained on SQuAD2.0 (Rajpurkar et al., 2018) that is given a text and a comprehension question, i.e. "What was Fort Caroline renamed to after the Spanish attack?" which it correctly answers as "San Mateo" with 98% confidence. Altering this question, however, can increase model confidence for this same prediction to 99%, even though the new question is unanswerable given the same context. That is, we observe an increase in model probability, despite removing relevant question information and replacing it with irrelevant content. We formalise the process of finding such questions as an adversarial search in a discrete input space arising from perturbations of the original question. There are two types of discrete perturbations that we consider, based on part-of-speech and named entities, with the aim of obtaining grammatical and semantically consistent alternative questions that do not accidentally have the same correct answer. We find that SQuAD2.0 and NewsQA (Trischler et al., 2017 ) models can be attacked on a substantial proportion of samples, even with a limited computational adversarial search budget. The observed undersensitivity correlates negatively with standard performance metrics (EM/F 1 ), suggesting that this phenomenon -where present -is a reflection of a model's lack of question comprehension. When training models to defend against undersensitivity attacks with data augmentation and adversarial training, we observe that they can generalise their robustness to held out evaluation data without sacrificing standard performance. Furthermore, we notice they are also more robust in a learning scenario that has dataset bias with a train/evaluation distribution mismatch, increasing their performance by up to 11%F 1 . In summary, our contributions are as follows: • We propose a new type of adversarial attack targeting the undersensitivity of neural RC models, and show that current models are vulnerable to it. • We compare two defence strategies, data augmentation and adversarial training, and show their effectiveness at reducing undersensitivity errors on held-out data, without sacrificing standard performance. • We demonstrate that robust models generalise better in a biased data scenario, improving their ability to answer questions with many possible answers when trained on questions with only one. We have investigated a problematic behaviour of RC models -being overly stable in their predictions when given semantically altered questions. This undersensitivity can be drastically reduced with appropriate defences, such as adversarial training, and results in more robust models without sacrificing standard performance. Future work should study in more detail the causes and better defences to model undersensitivity, which we believe provides an alternative viewpoint on evaluating a model's RC capabilities. 5 approximate as we stratify by article 6 We also include an experiment with the setup used in (Lewis & Fan, 2019) Table 7 : Breakdown of undersensitivity error rate on NewsQA with a held-out attack space (lower is better). A APPENDIX: POS PERTURBATION DETAILS. We exclude these PoS-tags when computing perturbations:
We demonstrate vulnerability to undersensitivity attacks in SQuAD2.0 and NewsQA neural reading comprehension models, where the model predicts the same answer with increased confidence to adversarially chosen questions, and compare defence strategies.
{"src": "scitldr-AIC_train", "paper_id": "HkgxheBFDS"}