Despite alarm over the reliance of machine learning systems on so-called spurious patterns in training data, the term lacks coherent meaning in standard statistical frameworks. However, the language of causality offers clarity: spurious associations are those due to a common cause (confounding) vs direct or indirect effects. In this paper, we focus on NLP, introducing methods and resources for training models insensitive to spurious patterns. Given documents and their initial labels, we task humans with revise each document to accord with a counterfactual target label, asking that the revised documents be internally coherent while avoiding any gratuitous changes. Interestingly, on sentiment analysis and natural language inference tasks, classifiers trained on original data fail on their counterfactually-revised counterparts and vice versa. Classifiers trained on combined datasets perform remarkably well, just shy of those specialized to either domain. While classifiers trained on either original or manipulated data alone  are sensitive to spurious features (e.g., mentions of genre), models trained on the combined data are insensitive to this signal. We will publicly release both datasets. What makes a document's sentiment positive? What makes a loan applicant creditworthy? What makes a job candidate qualified? What about a photograph truly makes it depict a dolphin? Moreover, what does it mean for a feature to be relevant to such a determination? Statistical learning offers one framework for approaching these questions. First, we swap out the semantic question for a more readily answerable associative question. For example, instead of asking what comprises a document's sentiment, we recast the question as which documents are likely to be labeled as positive (or negative)? Then, in this associative framing, we interpret as relevant, those features that are most predictive of the label. However, despite the rapid adoption and undeniable commercial success of associative learning, this framing seems unsatisfying. Alongside deep learning's predictive wins, critical questions have piled up concerning spuriousness, artifacts, reliability, and discrimination, that the purely associative perspective appears ill-equipped to answer. For example, in computer vision, researchers have found that deep neural networks rely on surface-level texture (Jo & Bengio, 2017; Geirhos et al., 2018) or clues in the image's background to recognize foreground objects even when that seems both unnecessary and somehow wrong: the beach is not what makes a seagull a seagull. And yet researchers struggle to articulate precisely why models should not rely on such patterns. In NLP, these issues have emerged as central concerns in the literature on annotation artifacts and bias (in the societal sense). Across myriad tasks, researchers have demonstrated that models tend to rely on spurious associations (Poliak et al., 2018; Gururangan et al., 2018; Kaushik & Lipton, 2018; Kiritchenko & Mohammad, 2018) . Notably, some models for question-answering tasks may not actually be sensitive to the choice of the question (Kaushik & Lipton, 2018) , while in Natural Language Inference (NLI), classifiers trained on hypotheses only (vs hypotheses and premises) perform surprisingly well (Poliak et al., 2018; Gururangan et al., 2018) . However, papers seldom make clear what, if anything, spuriousness means within the standard supervised learning framework. ML systems are trained to exploit the mutual information between features and a label to make accurate predictions. Statistical learning does not offer a conceptual distinction between between spurious and non-spurious associations. Causality, however, offers a coherent notion of spuriousness. Spurious associations owe to common cause rather than to a (direct or indirect) causal path. We might consider a factor of variation to be spuriously correlated with a label of interest if intervening upon it (counterfactually) would not impact the applicability of the label or vice versa. While our paper does not rely on the mathematical machinery of causality, we draw inspiration from the underlying philosophy to design a new dataset creation procedure in which humans counterfactually augment datasets. Returning to NLP, even though the raw data does not come neatly disentangled into manipulable factors, people nevertheless speak colloquially of editing documents to manipulate specific aspects (Hovy, 1987) . For example, the following interventions seem natural: (i) Revise the letter to make it more positive; (ii) Edit the second sentence so that it appears to contradict the first. The very notion of targeted revisions like (i) suggests a generative process in which the sentiment is but one (manipulable) cause of the final document. These edits might be thought of as intervening on sentiment while holding all upstream features constant. However even if some other factor has no influence on sentiment, if they share some underlying common cause (confounding), then we might expect aspects of the final document to be predictive of sentiment owing to spurious association. In this exploratory paper, we design a human-in-the-loop system for counterfactually manipulating documents. Our hope is that by intervening only upon the factor of interest, we might disentangle the spurious and non-spurious associations, yielding classifiers that hold up better when spurious associations do not transport out of sample. We employ crowd workers not to label documents, but rather to edit them, manipulating the text to make a targeted (counterfactual) class apply. For sentiment analysis, we direct the worker: revise this negative movie review to make it positive, without making any gratuitous changes. We might regard the second part of this directive as a sort of least action principle, ensuring that we perturb only those spans necessary to alter the applicability of the label. For NLI, a 3-class classification task (entailment, contradiction, neutral), we ask the workers to modify the premise while keeping the hypothesis intact, and vice versa, seeking two sets of edits corresponding to each of the (two) counterfactual classes. Using this platform, we collect thousands of counterfactually-manipulated examples for both sentiment analysis and NLI, extending the IMDb (Maas et al., 2011) and SNLI (Bowman et al., 2015) datasets, respectively. The result is two new datasets (each an extension of a standard resource) that enable us to both probe fundamental properties of language and train classifiers less reliant on spurious signal. We show that classifiers trained on original IMDb reviews fail on counterfactually-revised data and vice versa. We further show that spurious correlations in these datasets are picked up by even linear models, however, augmenting the revised examples breaks up these correlations (e.g., genre ceases to be predictive of sentiment). For a Bidirectional LSTM (Graves & Schmidhuber, 2005 ) trained on IMDb reviews, classification accuracy goes down from 79.3% to 55.7% when evaluated on original vs revised reviews. The same classifier trained on revised reviews achieves an accuracy of 62.5% on original reviews compared to 89.1% on their revised counterparts. These numbers go to 81.7% and 92.0% respectively when the classifier is retrained on the combined dataset. Similar behavior is observed for linear classifiers. We discovered that BERT (Devlin et al., 2019 ) is more resilient to such drops in performance on sentiment analysis. Despite that, it appears to rely on spurious associations in SNLI hypotheses identified by Gururangan et al. (2018) . We show that if fine-tuned on SNLI sentence pairs, BERT fails on pairs with revised premise and vice versa, experiencing more than a 30 point drop in accuracy. However, fine-tuned on the combined set, it performs much better across all datasets. Similarly, a Bi-LSTM trained on hypotheses alone can accurately classify 69% of the SNLI dataset but performs worse than the majority class baseline when evaluated on the revised dataset. When trained on hypotheses only from the combined dataset, its performance is expectedly worse than simply selecting the majority class on both SNLI as well as the revised dataset. By leveraging humans not only to provide labels but also to intervene upon the data, revising documents to alter the applicability of various labels, we are able to derive insights about the underlying semantic concepts. Moreover we can leverage the augmented data to train classifiers less dependent on spurious associations. Our study demonstrates the promise of leveraging human-in-the-loop feedback to disentangle the spurious and non-spurious associations, yielding classifiers that hold up better when spurious associations do not transport out of sample. Our methods appear useful on both sentiment analysis and NLI, two contrasting tasks. In sentiment analysis, expressions of opinion matter more than stated facts, while in NLI this is reversed. SNLI poses another challenge in that it is a 3-class classification task using two input sentences. In future work, we plan to extend these techniques, finding ways to leverage humans in the loop to build more robust systems for question answering and summarization (among others).
Humans in the loop revise documents to accord with counterfactual labels, resulting resource helps to reduce reliance on spurious associations.
{"src": "scitldr-AIC_train", "paper_id": "Sklgs0NFvr"}