To this end, we introduce the negated LAMA dataset. We construct it by simply inserting negation elements (e.g., “not”) in LAMA cloze statement
EXTRACTIVE
Create the negated LAMA dataset and  query the pretrained language models with both original LAMA and negated LAMA statements and compare their predictions.
ABSTRACTIVE
This work analyzes the understanding of pretrained language models of factual and commonsense knowledge stored in negated statements. To this end, we introduce the negated LAMA dataset. We construct it by simply inserting negation elements (e.g., “not”) in LAMA cloze statement (e.g., “The theory of relativity was not developed by [MASK].”). In our experiments, we query the pretrained language models with both original LAMA and negated LAMA statements and compare their predictions in terms of rank correlation and overlap of top predictions. We find that the predicted filler words often have high overlap. Thus, negating a cloze statement does not change the predictions in many cases – but of course it should as our example “birds can fly” vs. “birds cannot fly” shows. We identify and analyze a subset of cloze statements where predictions are different. We find that BERT handles negation best among pretrained language models, but it still fails badly on most negated statements.
This work analyzes the understanding of pretrained language models of factual and commonsense knowledge stored in negated statements. To this end, we introduce the negated LAMA dataset. We construct it by simply inserting negation elements (e.g., “not”) in LAMA cloze statement (e.g., “The theory of relativity was not developed by [MASK].”). In our experiments, we query the pretrained language models with both original LAMA and negated LAMA statements and compare their predictions in terms of rank correlation and overlap of top predictions. We find that the predicted filler words often have high overlap. Thus, negating a cloze statement does not change the predictions in many cases – but of course it should as our example “birds can fly” vs. “birds cannot fly” shows. We identify and analyze a subset of cloze statements where predictions are different. We find that BERT handles negation best among pretrained language models, but it still fails badly on most negated statements.
1911.03343
Introduction </s> P ret rained Ġlanguage Ġmodels Ġlike ĠTrans former - XL ĠB IB REF 1 , ĠEL Mo ĠB IB REF 2 Ġand ĠB ERT ĠB IB REF 3 Ġhave Ġemerged Ġas Ġuniversal Ġtools Ġthat Ġcapture Ġa Ġdiverse Ġrange Ġof Ġlinguistic Ġand Ġfactual Ġknowledge . </s> Recently , ĠB IB REF 0 Ġintroduced ĠL AMA Ġ( L Ang u age ĠModel ĠAnalysis ) Ġto Ġinvestigate Ġto Ġwhat Ġextent Ġpret rained Ġlanguage Ġmodels Ġhave Ġthe Ġcapacity Ġto Ġrecall Ġfactual Ġknowledge Ġwithout Ġthe Ġuse Ġof Ġfine - tun ing . ĠThe Ġtraining Ġobjective Ġof Ġpret rained Ġlanguage Ġmodels Ġis Ġto Ġpredict Ġmasked Ġtokens Ġin Ġa Ġsequence . ĠWith Ġthis ĠâĢ ľ fill - in - the - blank âĢ Ŀ Ġscheme , Ġquestion Ġanswering Ġtasks Ġcan Ġbe Ġreform ulated Ġas Ġclo ze Ġstatements . ĠFor Ġexample , ĠâĢ ľ Who Ġdeveloped Ġthe Ġtheory Ġof Ġrelativity ? âĢ Ŀ Ġis Ġreform ulated Ġas ĠâĢ ľ The Ġtheory Ġof Ġrelativity Ġwas Ġdeveloped Ġby Ġ[ MAS K ]. âĢ Ŀ . ĠThis Ġsetup Ġallows Ġfor Ġun super vised Ġopen Ġdomain Ġquestion Ġanswering . ĠB IB REF 0 Ġfind Ġthat , Ġon Ġthis Ġtask , Ġpret rained Ġlanguage Ġmodels Ġoutper form Ġsupervised Ġbas elines Ġusing Ġtraditional Ġknowledge Ġbases Ġwith Ġaccess Ġto Ġor acle Ġknowledge . </s> This Ġwork Ġanaly zes Ġthe Ġunderstanding Ġof Ġpret rained Ġlanguage Ġmodels Ġof Ġfactual Ġand Ġcommons ense Ġknowledge Ġstored Ġin Ġneg ated Ġstatements . ĠTo Ġthis Ġend , Ġwe Ġintroduce Ġthe Ġneg ated ĠL AMA Ġdataset . ĠWe Ġconstruct Ġit Ġby Ġsimply Ġinserting Ġneg ation Ġelements Ġ( e . g ., ĠâĢ ľ not âĢ Ŀ ) Ġin ĠL AMA Ġclo ze Ġstatement Ġ( e . g ., ĠâĢ ľ The Ġtheory Ġof Ġrelativity Ġwas Ġnot Ġdeveloped Ġby Ġ[ MAS K ]. âĢ Ŀ ). ĠIn Ġour Ġexperiments , Ġwe Ġquery Ġthe Ġpret rained Ġlanguage Ġmodels Ġwith Ġboth Ġoriginal ĠL AMA Ġand Ġneg ated ĠL AMA Ġstatements Ġand Ġcompare Ġtheir Ġpredictions Ġin Ġterms Ġof Ġrank Ġcorrelation Ġand Ġoverlap Ġof Ġtop Ġpredictions . ĠWe Ġfind Ġthat Ġthe Ġpredicted Ġfiller Ġwords Ġoften Ġhave Ġhigh Ġoverlap . ĠThus , Ġneg ating Ġa Ġclo ze Ġstatement Ġdoes Ġnot Ġchange Ġthe Ġpredictions Ġin Ġmany Ġcases ĠâĢĵ Ġbut Ġof Ġcourse Ġit Ġshould Ġas Ġour Ġexample ĠâĢ ľ birds Ġcan Ġfly âĢ Ŀ Ġvs . ĠâĢ ľ birds Ġcannot Ġfly âĢ Ŀ Ġshows . ĠWe Ġidentify Ġand Ġanalyze Ġa Ġsubset Ġof Ġclo ze Ġstatements Ġwhere Ġpredictions Ġare Ġdifferent . ĠWe Ġfind Ġthat ĠB ERT Ġhandles Ġneg ation Ġbest Ġamong Ġpret rained Ġlanguage Ġmodels , Ġbut Ġit Ġstill Ġfails Ġbadly Ġon Ġmost Ġneg ated Ġstatements . </s> Data </s> A Ġclo ze Ġstatement Ġis Ġgenerated Ġfrom Ġa Ġsubject - relation - object Ġtriple Ġfrom Ġa Ġknowledge Ġbase Ġand Ġfrom Ġa Ġtem pl atic Ġstatement Ġfor Ġthe Ġrelation Ġthat Ġcontains Ġvariables ĠX Ġand ĠY Ġfor Ġsubject Ġand Ġobject Ġ( e . g , ĠâĢ ľ X Ġwas Ġborn Ġin ĠY âĢ Ŀ ). ĠWe Ġthen Ġsubstitute Ġthe Ġsubject Ġfor ĠX Ġand ĠMAS K Ġfor ĠY . ĠThe Ġtri ples Ġare Ġchosen Ġsuch Ġthat ĠY Ġis Ġalways Ġa Ġsingle - token Ġanswer . </s> L AMA Ġcovers Ġdifferent Ġsources : ĠThe ĠGoogle - RE Ġset Ġcovers Ġthe Ġthree Ġrelations ĠâĢ ľ place Ġof Ġbirth âĢ Ŀ , ĠâĢ ľ date Ġof Ġbirth âĢ Ŀ Ġand ĠâĢ ľ place Ġof Ġdeath âĢ Ŀ . ĠT - RE x ĠB IB REF 4 Ġconsists Ġof Ġa Ġsubset Ġof ĠWik id ata Ġtri ples Ġcovering Ġ41 Ġrelations . ĠConcept Net ĠB IB REF 5 Ġcombines Ġ16 Ġcommons ense Ġrelationships Ġbetween Ġwords Ġand / or Ġphrases . ĠThe Ġunderlying ĠOpen ĠMind ĠCommon ĠSense Ġcorpus Ġprovides Ġmatching Ġstatements Ġto Ġquery Ġthe Ġlanguage Ġmodel . ĠS Qu AD ĠB IB REF 6 Ġis Ġa Ġstandard Ġquestion Ġanswering Ġdataset . ĠL AMA Ġcontains Ġa Ġsubset Ġof Ġ305 Ġcontext - ins ensitive Ġquestions Ġand Ġprovides Ġmanually Ġreform ulated Ġclo ze - style Ġstatements Ġto Ġquery Ġthe Ġmodel . </s> We Ġcreated Ġneg ated Ġversions Ġof ĠGoogle - RE , ĠT - RE x Ġand ĠS Qu AD Ġby Ġmanually Ġinserting Ġa Ġneg ation Ġelement Ġin Ġeach Ġtemplate Ġor Ġstatement . ĠWe Ġdid Ġthe Ġsame Ġfor Ġa Ġsubset Ġof ĠConcept Net Ġthat Ġis Ġeasy Ġto Ġnegate . ĠWe Ġselected Ġthis Ġsubset Ġby Ġfiltering Ġfor Ġsentence Ġlength Ġand Ġextracting Ġcommon Ġqueries . </s> Mod els </s> We Ġuse Ġthe Ġsource Ġcode Ġprovided Ġby ĠB IB REF 0 Ġand ĠB IB REF 7 Ġand Ġevaluate Ġusing ĠTrans former - XL Ġlarge Ġ( Tx l ), ĠEL Mo Ġoriginal Ġ( E b ), ĠEL Mo Ġ5 . 5 B Ġ( E 5 B ), ĠB ERT - base Ġ( B b ) Ġand ĠB ERT - large Ġ( Bl ). </s> Results </s> Table ĠT AB REF 1 Ġcompares Ġthe Ġpredictions Ġof Ġoriginal ĠL AMA Ġand Ġneg ated ĠL AMA . ĠAs Ġtrue Ġanswers Ġof Ġthe Ġneg ated Ġstatements Ġare Ġhighly Ġambiguous , Ġour Ġmeasures Ġare Ġspear man Ġrank Ġcorrelation Ġand Ġoverlap Ġin Ġrank Ġ1 Ġpredictions Ġbetween Ġthe Ġoriginal Ġand Ġneg ated Ġdataset . ĠTable ĠT AB REF 4 Ġgives Ġexamples Ġof ĠB ERT - large Ġpredictions . </s> We Ġobserve Ġrank Ġcorrelations Ġof Ġmore Ġthan Ġ0 . 85 Ġin Ġmost Ġcases Ġand Ġa Ġhigh Ġoverlap Ġin Ġfirst Ġranked Ġpredictions Ġlike : ĠâĢ ľ B irds Ġcan Ġfly . âĢ Ŀ Ġand ĠâĢ ľ B irds Ġcannot Ġfly . âĢ Ŀ . ĠB ERT Ġhas Ġslightly Ġbetter Ġresults Ġthan Ġthe Ġother Ġmodels . </s> Our Ġinterpretation Ġof Ġthe Ġresults Ġis Ġthat ĠB ERT Ġmostly Ġdid Ġnot Ġlearn Ġthe Ġmeaning Ġof Ġneg ation . ĠThe Ġimpressive Ġresults Ġin ĠQ A Ġsuggest Ġthat Ġpret rained Ġlanguage Ġmodels Ġare Ġable Ġto Ġmemor ize Ġaspects Ġof Ġspecific Ġfacts ; Ġbut , Ġapparently , Ġthey Ġignore Ġneg ation Ġmarkers Ġin Ġmany Ġcases Ġand Ġrely Ġon Ġthe Ġco - occ urrence Ġof Ġthe Ġsubject Ġwith Ġthe Ġoriginal Ġrelation Ġonly . ĠOne Ġreason Ġfor Ġthe Ġpoor Ġperformance Ġwe Ġobserve Ġprobably Ġis Ġthat Ġneg ated Ġstatements Ġoccur Ġmuch Ġless Ġin Ġtraining Ġcorpor a Ġthan Ġpositive Ġstatements . </s> A Ġkey Ġproblem Ġis Ġthat Ġthe ĠL AMA Ġsetup Ġdoes Ġnot Ġallow Ġto Ġrefrain Ġfrom Ġgiving Ġan Ġanswer . ĠGenerally , Ġprediction Ġprobabilities Ġdrop Ġin Ġthe Ġneg ated Ġstatements , Ġwhich Ġwould Ġsuggest Ġthe Ġexistence Ġof Ġa Ġthreshold Ġto Ġfilter Ġanswers . ĠBut Ġa Ġcloser Ġlook Ġat Ġthe Ġprobabilities Ġof Ġcorrect Ġand Ġincorrect Ġpredictions Ġshows Ġthat Ġthey Ġfall Ġinto Ġthe Ġsame Ġrange . ĠNo Ġcommon Ġthreshold Ġcan Ġbe Ġfound . </s> Given Ġthat Ġneg ation Ġhas Ġlittle Ġeffect Ġon Ġmost Ġqueries , Ġit Ġis Ġinteresting Ġto Ġlook Ġat Ġthe Ġsmall Ġnumber Ġof Ġqueries Ġwhere Ġpret rained Ġlanguage Ġmodels Ġmake Ġcorrect Ġpredictions , Ġi . e ., Ġthey Ġsolve Ġthe Ġclo ze Ġtask Ġas Ġa Ġhuman Ġsubject Ġwould Ġdo . ĠWe Ġgive Ġtwo Ġexamples Ġof Ġsuch Ġpatterns . ĠThe Ġpattern ĠâĢ ľ X Ġdid Ġnot Ġdie Ġin ĠY âĢ Ŀ Ġalways Ġresults Ġin Ġthe Ġgeneric Ġtop Ġranked Ġpredictions : ĠâĢ ľ battle âĢ Ŀ , ĠâĢ ľ office âĢ Ŀ , ĠâĢ ľ prison âĢ Ŀ Ġwhereas Ġthe Ġoriginal Ġpattern Ġis Ġlikely Ġto Ġrank Ġcities Ġfirst . ĠThis Ġseems Ġappropriate Ġsince Ġa Ġstatement Ġof Ġthe Ġform , Ġsay , ĠâĢ ľ X Ġdid Ġnot Ġdie Ġin ĠNew ĠYork âĢ Ŀ Ġis Ġrare Ġin Ġtext Ġcorpor a , Ġbut Ġstatements Ġcharacter izing Ġthe Ġsituation Ġin Ġwhich Ġthe Ġdeath Ġoccurred Ġ( âĢ ľ he Ġdid Ġnot Ġdie Ġin Ġprison âĢ Ŀ ) Ġsound Ġmore Ġnatural . ĠFor Ġthe Ġtemplate ĠâĢ ľ X Ġwas Ġborn Ġin ĠY âĢ Ŀ , Ġcities Ġare Ġpredicted . ĠIn Ġcontrast , Ġfor ĠâĢ ľ X Ġwas Ġnot Ġborn Ġin ĠY âĢ Ŀ , Ġcountries Ġare Ġpredicted . ĠBoth Ġtimes Ġit Ġrefers Ġto Ġa Ġmore Ġspecific Ġstatement , Ġmore Ġlikely Ġto Ġoccur Ġin Ġthe Ġtraining Ġcorpus . ĠPeople Ġwould Ġrefer Ġmore Ġoften Ġto Ġa Ġperson Ġbeing Ġborn Ġin Ġa Ġcity Ġand Ġnot Ġborn Ġin Ġa Ġcountry , Ġgiving Ġyou Ġin Ġboth Ġcases Ġmore Ġprecise Ġinformation . </s> Related ĠWork </s> P ret rained Ġembed d ings Ġhave Ġpushed Ġbas elines Ġon Ġa Ġvariety Ġof Ġquestion Ġanswering Ġdatasets ĠB IB REF 8 , ĠB IB REF 9 . ĠGenerally , Ġthe Ġpret rained Ġmodels Ġare Ġfine - tun ed Ġto Ġthe Ġspecific Ġtask ĠB IB REF 10 , ĠB IB REF 3 Ġbut Ġrecent Ġwork Ġhas Ġapplied Ġthe Ġmodels Ġwithout Ġthe Ġfine - tun ing Ġstep ĠB IB REF 11 , ĠB IB REF 0 . </s> There Ġis Ġa Ġwide Ġrange Ġof Ġliterature Ġanalyzing Ġlinguistic Ġknowledge Ġstored Ġin Ġpret rained Ġembed d ings ĠB IB REF 12 , ĠB IB REF 13 , ĠB IB REF 14 , ĠB IB REF 15 , ĠB IB REF 16 , ĠB IB REF 17 , ĠB IB REF 18 , ĠB IB REF 19 . </s> Con cerning Ġneg ation Ġthe Ġfollowing Ġpapers Ġare Ġof Ġinterest : ĠB IB REF 20 Ġanalyze Ġthe Ġgram matical Ġknowledge Ġcaptured Ġby ĠB ERT . ĠIn Ġa Ġcase Ġstudy , Ġthey Ġtest Ġfor Ġcorrect Ġlicensing Ġenvironments Ġfor Ġnegative Ġpol arity Ġitems . ĠThey Ġstudy Ġa Ġset Ġof Ġclass ifiers Ġdistinguishing Ġbetween Ġgram matically Ġcorrect Ġand Ġincorrect Ġsentences . ĠWe Ġtake Ġa Ġdifferent Ġapproach Ġby Ġfocusing Ġon Ġfactual Ġknowledge Ġstored Ġin Ġneg ated Ġstatements . ĠGram matically Ġcorrect Ġstatements Ġcan Ġstill Ġbe Ġfact ually Ġfalse Ġ( e . g ., ĠâĢ ľ General Ġrelativity , ĠNewton Ġdevelop âĢ Ŀ ). </s> BI B REF 21 Ġinvestigate Ġthe Ġunderstanding Ġof Ġfunction Ġwords ĠâĢĵ Ġamong Ġthem Ġneg ation Ġparticles ĠâĢĵ Ġusing Ġan Ġentail ment - Ġand Ġclassification - based Ġapproach . ĠThey Ġanalyze Ġthe Ġability Ġof Ġdifferent Ġmodel Ġarchitectures Ġand Ġtraining Ġobjectives Ġto Ġcapture Ġknowledge Ġof Ġsingle Ġsentences . ĠThe Ġmodels Ġare Ġfine - tun ed Ġto Ġthe Ġtask Ġof Ġinterest . ĠWe Ġon Ġthe Ġother Ġhand Ġquestion Ġto Ġwhat Ġextend Ġfactual Ġknowledge Ġpresent Ġin Ġneg ated Ġstatements Ġis Ġindirectly Ġacquired Ġduring Ġpret raining . </s> BI B REF 22 Ġdefines Ġthree Ġpsych ol ingu istic Ġdiagn ostics Ġfor Ġlanguage Ġmodels Ġand Ġapplies Ġthem Ġin Ġa Ġcase Ġstudy Ġto ĠB ERT . ĠNeg ation Ġis Ġexamined Ġusing Ġa Ġdataset Ġof Ġ72 Ġsimple Ġsentences Ġquer ying Ġfor Ġcategory Ġmembership . ĠA Ġsupplementary Ġdataset Ġof Ġ16 Ġsentences Ġqueries Ġagain Ġfor Ġthe Ġ" to Ġbe " Ġrelation Ġonly Ġbut Ġincluding Ġmore Ġnatural Ġsentence Ġstructure . ĠOur Ġwork Ġcovers Ġ51 , 329 Ġneg ated Ġstatements Ġcovering Ġa Ġwide Ġrange Ġof Ġtopics Ġand Ġrelations . ĠIn Ġthe ĠS Qu AD Ġbased Ġdataset Ġwe Ġcover Ġmore Ġnatural Ġlanguage Ġin Ġterms Ġof Ġcontext Ġand Ġrelation . ĠIn Ġcontrast Ġto ĠB IB REF 22 , Ġwe Ġdo Ġnot Ġsee Ġa Ġreliable Ġpreference Ġof Ġthe Ġtrue Ġcomple tions Ġto Ġfalse Ġin Ġthe Ġmore Ġnatural Ġneg ated Ġstatements . </s> BI B REF 23 Ġtest Ġfor Ġcomprehension Ġof Ġminim ally Ġmodified Ġstatements Ġin Ġan Ġadvers arial Ġsetup Ġwhile Ġtrying Ġto Ġkeep Ġthe Ġoverall Ġsemantics Ġthe Ġsame . ĠWe Ġtry Ġto Ġmaximize Ġthe Ġchange Ġin Ġsemantics Ġand Ġin vert Ġmeaning . </s> Conclusion </s> We Ġshow Ġthat Ġpret rained Ġlanguage Ġmodels Ġhave Ġproblems Ġhandling Ġneg ation . ĠOutput Ġpredictions Ġfor Ġthe Ġoriginal ĠL AMA Ġquery Ġand Ġthe Ġneg ated Ġstatement Ġare Ġhighly Ġcorrelated . </s> Even Ġthough Ġthis Ġelegant Ġapproach Ġof Ġquer ying Ġa Ġlanguage Ġmodel Ġwithout Ġfine - tun ing Ġallows Ġfor Ġtruly Ġopen Ġdomain Ġquestion Ġanswering , Ġpromoting Ġan Ġanswer Ġno Ġmatter Ġwhat Ġis Ġnot Ġalways Ġthe Ġbetter Ġsolution . ĠRef raining Ġfrom Ġgiving Ġan Ġanswer Ġcan Ġbe Ġmore Ġappropriate , Ġmaking Ġknowledge Ġgraphs Ġcurrently Ġa Ġmore Ġreliable Ġchoice Ġfor Ġquestion Ġanswering .
Introduction Pretrained language models like Transformer-XL BIBREF1, ELMo BIBREF2 and BERT BIBREF3 have emerged as universal tools that capture a diverse range of linguistic and factual knowledge. Recently, BIBREF0 introduced LAMA (LAnguage Model Analysis) to investigate to what extent pretrained language models have the capacity to recall factual knowledge without the use of fine-tuning. The training objective of pretrained language models is to predict masked tokens in a sequence. With this “fill-in-the-blank” scheme, question answering tasks can be reformulated as cloze statements. For example, “Who developed the theory of relativity?” is reformulated as “The theory of relativity was developed by [MASK].”. This setup allows for unsupervised open domain question answering. BIBREF0 find that, on this task, pretrained language models outperform supervised baselines using traditional knowledge bases with access to oracle knowledge. This work analyzes the understanding of pretrained language models of factual and commonsense knowledge stored in negated statements. To this end, we introduce the negated LAMA dataset. We construct it by simply inserting negation elements (e.g., “not”) in LAMA cloze statement (e.g., “The theory of relativity was not developed by [MASK].”). In our experiments, we query the pretrained language models with both original LAMA and negated LAMA statements and compare their predictions in terms of rank correlation and overlap of top predictions. We find that the predicted filler words often have high overlap. Thus, negating a cloze statement does not change the predictions in many cases – but of course it should as our example “birds can fly” vs. “birds cannot fly” shows. We identify and analyze a subset of cloze statements where predictions are different. We find that BERT handles negation best among pretrained language models, but it still fails badly on most negated statements. Data A cloze statement is generated from a subject-relation-object triple from a knowledge base and from a templatic statement for the relation that contains variables X and Y for subject and object (e.g, “X was born in Y”). We then substitute the subject for X and MASK for Y. The triples are chosen such that Y is always a single-token answer. LAMA covers different sources: The Google-RE set covers the three relations “place of birth”, “date of birth” and “place of death”. T-REx BIBREF4 consists of a subset of Wikidata triples covering 41 relations. ConceptNet BIBREF5 combines 16 commonsense relationships between words and/or phrases. The underlying Open Mind Common Sense corpus provides matching statements to query the language model. SQuAD BIBREF6 is a standard question answering dataset. LAMA contains a subset of 305 context-insensitive questions and provides manually reformulated cloze-style statements to query the model. We created negated versions of Google-RE, T-REx and SQuAD by manually inserting a negation element in each template or statement. We did the same for a subset of ConceptNet that is easy to negate. We selected this subset by filtering for sentence length and extracting common queries. Models We use the source code provided by BIBREF0 and BIBREF7 and evaluate using Transformer-XL large (Txl), ELMo original (Eb), ELMo 5.5B (E5B), BERT-base (Bb) and BERT-large (Bl). Results Table TABREF1 compares the predictions of original LAMA and negated LAMA. As true answers of the negated statements are highly ambiguous, our measures are spearman rank correlation and overlap in rank 1 predictions between the original and negated dataset. Table TABREF4 gives examples of BERT-large predictions. We observe rank correlations of more than 0.85 in most cases and a high overlap in first ranked predictions like: “Birds can fly.” and “Birds cannot fly.”. BERT has slightly better results than the other models. Our interpretation of the results is that BERT mostly did not learn the meaning of negation. The impressive results in QA suggest that pretrained language models are able to memorize aspects of specific facts; but, apparently, they ignore negation markers in many cases and rely on the co-occurrence of the subject with the original relation only. One reason for the poor performance we observe probably is that negated statements occur much less in training corpora than positive statements. A key problem is that the LAMA setup does not allow to refrain from giving an answer. Generally, prediction probabilities drop in the negated statements, which would suggest the existence of a threshold to filter answers. But a closer look at the probabilities of correct and incorrect predictions shows that they fall into the same range. No common threshold can be found. Given that negation has little effect on most queries, it is interesting to look at the small number of queries where pretrained language models make correct predictions, i.e., they solve the cloze task as a human subject would do. We give two examples of such patterns. The pattern “X did not die in Y” always results in the generic top ranked predictions: “battle”, “office”, “prison” whereas the original pattern is likely to rank cities first. This seems appropriate since a statement of the form, say, “X did not die in New York” is rare in text corpora, but statements characterizing the situation in which the death occurred (“he did not die in prison”) sound more natural. For the template “X was born in Y”, cities are predicted. In contrast, for “X was not born in Y”, countries are predicted. Both times it refers to a more specific statement, more likely to occur in the training corpus. People would refer more often to a person being born in a city and not born in a country, giving you in both cases more precise information. Related Work Pretrained embeddings have pushed baselines on a variety of question answering datasets BIBREF8, BIBREF9. Generally, the pretrained models are fine-tuned to the specific task BIBREF10, BIBREF3 but recent work has applied the models without the fine-tuning step BIBREF11, BIBREF0. There is a wide range of literature analyzing linguistic knowledge stored in pretrained embeddings BIBREF12, BIBREF13, BIBREF14, BIBREF15, BIBREF16, BIBREF17, BIBREF18, BIBREF19. Concerning negation the following papers are of interest: BIBREF20 analyze the grammatical knowledge captured by BERT. In a case study, they test for correct licensing environments for negative polarity items. They study a set of classifiers distinguishing between grammatically correct and incorrect sentences. We take a different approach by focusing on factual knowledge stored in negated statements. Grammatically correct statements can still be factually false (e.g., “General relativity, Newton develop”). BIBREF21 investigate the understanding of function words – among them negation particles – using an entailment- and classification-based approach. They analyze the ability of different model architectures and training objectives to capture knowledge of single sentences. The models are fine-tuned to the task of interest. We on the other hand question to what extend factual knowledge present in negated statements is indirectly acquired during pretraining. BIBREF22 defines three psycholinguistic diagnostics for language models and applies them in a case study to BERT. Negation is examined using a dataset of 72 simple sentences querying for category membership. A supplementary dataset of 16 sentences queries again for the "to be" relation only but including more natural sentence structure. Our work covers 51,329 negated statements covering a wide range of topics and relations. In the SQuAD based dataset we cover more natural language in terms of context and relation. In contrast to BIBREF22, we do not see a reliable preference of the true completions to false in the more natural negated statements. BIBREF23 test for comprehension of minimally modified statements in an adversarial setup while trying to keep the overall semantics the same. We try to maximize the change in semantics and invert meaning. Conclusion We show that pretrained language models have problems handling negation. Output predictions for the original LAMA query and the negated statement are highly correlated. Even though this elegant approach of querying a language model without fine-tuning allows for truly open domain question answering, promoting an answer no matter what is not always the better solution. Refraining from giving an answer can be more appropriate, making knowledge graphs currently a more reliable choice for question answering.
How did they extend LAMA evaluation framework to focus on negation?
78292bc57ee68fdb93ed45430d80acca25a9e916
How Ġdid Ġthey Ġextend ĠL AMA Ġevaluation Ġframework Ġto Ġfocus Ġon Ġneg ation ?
To Ġthis Ġend , Ġwe Ġintroduce Ġthe Ġneg ated ĠL AMA Ġdataset . ĠWe Ġconstruct Ġit Ġby Ġsimply Ġinserting Ġneg ation Ġelements Ġ( e . g ., ĠâĢ ľ not âĢ Ŀ ) Ġin ĠL AMA Ġclo ze Ġstatement
<s> How Ġdid Ġthey Ġextend ĠL AMA Ġevaluation Ġframework Ġto Ġfocus Ġon Ġneg ation ? </s> Introduction </s> P ret rained Ġlanguage Ġmodels Ġlike ĠTrans former - XL ĠB IB REF 1 , ĠEL Mo ĠB IB REF 2 Ġand ĠB ERT ĠB IB REF 3 Ġhave Ġemerged Ġas Ġuniversal Ġtools Ġthat Ġcapture Ġa Ġdiverse Ġrange Ġof Ġlinguistic Ġand Ġfactual Ġknowledge . </s> Recently , ĠB IB REF 0 Ġintroduced ĠL AMA Ġ( L Ang u age ĠModel ĠAnalysis ) Ġto Ġinvestigate Ġto Ġwhat Ġextent Ġpret rained Ġlanguage Ġmodels Ġhave Ġthe Ġcapacity Ġto Ġrecall Ġfactual Ġknowledge Ġwithout Ġthe Ġuse Ġof Ġfine - tun ing . ĠThe Ġtraining Ġobjective Ġof Ġpret rained Ġlanguage Ġmodels Ġis Ġto Ġpredict Ġmasked Ġtokens Ġin Ġa Ġsequence . ĠWith Ġthis ĠâĢ ľ fill - in - the - blank âĢ Ŀ Ġscheme , Ġquestion Ġanswering Ġtasks Ġcan Ġbe Ġreform ulated Ġas Ġclo ze Ġstatements . ĠFor Ġexample , ĠâĢ ľ Who Ġdeveloped Ġthe Ġtheory Ġof Ġrelativity ? âĢ Ŀ Ġis Ġreform ulated Ġas ĠâĢ ľ The Ġtheory Ġof Ġrelativity Ġwas Ġdeveloped Ġby Ġ[ MAS K ]. âĢ Ŀ . ĠThis Ġsetup Ġallows Ġfor Ġun super vised Ġopen Ġdomain Ġquestion Ġanswering . ĠB IB REF 0 Ġfind Ġthat , Ġon Ġthis Ġtask , Ġpret rained Ġlanguage Ġmodels Ġoutper form Ġsupervised Ġbas elines Ġusing Ġtraditional Ġknowledge Ġbases Ġwith Ġaccess Ġto Ġor acle Ġknowledge . </s> This Ġwork Ġanaly zes Ġthe Ġunderstanding Ġof Ġpret rained Ġlanguage Ġmodels Ġof Ġfactual Ġand Ġcommons ense Ġknowledge Ġstored Ġin Ġneg ated Ġstatements . ĠTo Ġthis Ġend , Ġwe Ġintroduce Ġthe Ġneg ated ĠL AMA Ġdataset . ĠWe Ġconstruct Ġit Ġby Ġsimply Ġinserting Ġneg ation Ġelements Ġ( e . g ., ĠâĢ ľ not âĢ Ŀ ) Ġin ĠL AMA Ġclo ze Ġstatement Ġ( e . g ., ĠâĢ ľ The Ġtheory Ġof Ġrelativity Ġwas Ġnot Ġdeveloped Ġby Ġ[ MAS K ]. âĢ Ŀ ). ĠIn Ġour Ġexperiments , Ġwe Ġquery Ġthe Ġpret rained Ġlanguage Ġmodels Ġwith Ġboth Ġoriginal ĠL AMA Ġand Ġneg ated ĠL AMA Ġstatements Ġand Ġcompare Ġtheir Ġpredictions Ġin Ġterms Ġof Ġrank Ġcorrelation Ġand Ġoverlap Ġof Ġtop Ġpredictions . ĠWe Ġfind Ġthat Ġthe Ġpredicted Ġfiller Ġwords Ġoften Ġhave Ġhigh Ġoverlap . ĠThus , Ġneg ating Ġa Ġclo ze Ġstatement Ġdoes Ġnot Ġchange Ġthe Ġpredictions Ġin Ġmany Ġcases ĠâĢĵ Ġbut Ġof Ġcourse Ġit Ġshould Ġas Ġour Ġexample ĠâĢ ľ birds Ġcan Ġfly âĢ Ŀ Ġvs . ĠâĢ ľ birds Ġcannot Ġfly âĢ Ŀ Ġshows . ĠWe Ġidentify Ġand Ġanalyze Ġa Ġsubset Ġof Ġclo ze Ġstatements Ġwhere Ġpredictions Ġare Ġdifferent . ĠWe Ġfind Ġthat ĠB ERT Ġhandles Ġneg ation Ġbest Ġamong Ġpret rained Ġlanguage Ġmodels , Ġbut Ġit Ġstill Ġfails Ġbadly Ġon Ġmost Ġneg ated Ġstatements . </s> Data </s> A Ġclo ze Ġstatement Ġis Ġgenerated Ġfrom Ġa Ġsubject - relation - object Ġtriple Ġfrom Ġa Ġknowledge Ġbase Ġand Ġfrom Ġa Ġtem pl atic Ġstatement Ġfor Ġthe Ġrelation Ġthat Ġcontains Ġvariables ĠX Ġand ĠY Ġfor Ġsubject Ġand Ġobject Ġ( e . g , ĠâĢ ľ X Ġwas Ġborn Ġin ĠY âĢ Ŀ ). ĠWe Ġthen Ġsubstitute Ġthe Ġsubject Ġfor ĠX Ġand ĠMAS K Ġfor ĠY . ĠThe Ġtri ples Ġare Ġchosen Ġsuch Ġthat ĠY Ġis Ġalways Ġa Ġsingle - token Ġanswer . </s> L AMA Ġcovers Ġdifferent Ġsources : ĠThe ĠGoogle - RE Ġset Ġcovers Ġthe Ġthree Ġrelations ĠâĢ ľ place Ġof Ġbirth âĢ Ŀ , ĠâĢ ľ date Ġof Ġbirth âĢ Ŀ Ġand ĠâĢ ľ place Ġof Ġdeath âĢ Ŀ . ĠT - RE x ĠB IB REF 4 Ġconsists Ġof Ġa Ġsubset Ġof ĠWik id ata Ġtri ples Ġcovering Ġ41 Ġrelations . ĠConcept Net ĠB IB REF 5 Ġcombines Ġ16 Ġcommons ense Ġrelationships Ġbetween Ġwords Ġand / or Ġphrases . ĠThe Ġunderlying ĠOpen ĠMind ĠCommon ĠSense Ġcorpus Ġprovides Ġmatching Ġstatements Ġto Ġquery Ġthe Ġlanguage Ġmodel . ĠS Qu AD ĠB IB REF 6 Ġis Ġa Ġstandard Ġquestion Ġanswering Ġdataset . ĠL AMA Ġcontains Ġa Ġsubset Ġof Ġ305 Ġcontext - ins ensitive Ġquestions Ġand Ġprovides Ġmanually Ġreform ulated Ġclo ze - style Ġstatements Ġto Ġquery Ġthe Ġmodel . </s> We Ġcreated Ġneg ated Ġversions Ġof ĠGoogle - RE , ĠT - RE x Ġand ĠS Qu AD Ġby Ġmanually Ġinserting Ġa Ġneg ation Ġelement Ġin Ġeach Ġtemplate Ġor Ġstatement . ĠWe Ġdid Ġthe Ġsame Ġfor Ġa Ġsubset Ġof ĠConcept Net Ġthat Ġis Ġeasy Ġto Ġnegate . ĠWe Ġselected Ġthis Ġsubset Ġby Ġfiltering Ġfor Ġsentence Ġlength Ġand Ġextracting Ġcommon Ġqueries . </s> Mod els </s> We Ġuse Ġthe Ġsource Ġcode Ġprovided Ġby ĠB IB REF 0 Ġand ĠB IB REF 7 Ġand Ġevaluate Ġusing ĠTrans former - XL Ġlarge Ġ( Tx l ), ĠEL Mo Ġoriginal Ġ( E b ), ĠEL Mo Ġ5 . 5 B Ġ( E 5 B ), ĠB ERT - base Ġ( B b ) Ġand ĠB ERT - large Ġ( Bl ). </s> Results </s> Table ĠT AB REF 1 Ġcompares Ġthe Ġpredictions Ġof Ġoriginal ĠL AMA Ġand Ġneg ated ĠL AMA . ĠAs Ġtrue Ġanswers Ġof Ġthe Ġneg ated Ġstatements Ġare Ġhighly Ġambiguous , Ġour Ġmeasures Ġare Ġspear man Ġrank Ġcorrelation Ġand Ġoverlap Ġin Ġrank Ġ1 Ġpredictions Ġbetween Ġthe Ġoriginal Ġand Ġneg ated Ġdataset . ĠTable ĠT AB REF 4 Ġgives Ġexamples Ġof ĠB ERT - large Ġpredictions . </s> We Ġobserve Ġrank Ġcorrelations Ġof Ġmore Ġthan Ġ0 . 85 Ġin Ġmost Ġcases Ġand Ġa Ġhigh Ġoverlap Ġin Ġfirst Ġranked Ġpredictions Ġlike : ĠâĢ ľ B irds Ġcan Ġfly . âĢ Ŀ Ġand ĠâĢ ľ B irds Ġcannot Ġfly . âĢ Ŀ . ĠB ERT Ġhas Ġslightly Ġbetter Ġresults Ġthan Ġthe Ġother Ġmodels . </s> Our Ġinterpretation Ġof Ġthe Ġresults Ġis Ġthat ĠB ERT Ġmostly Ġdid Ġnot Ġlearn Ġthe Ġmeaning Ġof Ġneg ation . ĠThe Ġimpressive Ġresults Ġin ĠQ A Ġsuggest Ġthat Ġpret rained Ġlanguage Ġmodels Ġare Ġable Ġto Ġmemor ize Ġaspects Ġof Ġspecific Ġfacts ; Ġbut , Ġapparently , Ġthey Ġignore Ġneg ation Ġmarkers Ġin Ġmany Ġcases Ġand Ġrely Ġon Ġthe Ġco - occ urrence Ġof Ġthe Ġsubject Ġwith Ġthe Ġoriginal Ġrelation Ġonly . ĠOne Ġreason Ġfor Ġthe Ġpoor Ġperformance Ġwe Ġobserve Ġprobably Ġis Ġthat Ġneg ated Ġstatements Ġoccur Ġmuch Ġless Ġin Ġtraining Ġcorpor a Ġthan Ġpositive Ġstatements . </s> A Ġkey Ġproblem Ġis Ġthat Ġthe ĠL AMA Ġsetup Ġdoes Ġnot Ġallow Ġto Ġrefrain Ġfrom Ġgiving Ġan Ġanswer . ĠGenerally , Ġprediction Ġprobabilities Ġdrop Ġin Ġthe Ġneg ated Ġstatements , Ġwhich Ġwould Ġsuggest Ġthe Ġexistence Ġof Ġa Ġthreshold Ġto Ġfilter Ġanswers . ĠBut Ġa Ġcloser Ġlook Ġat Ġthe Ġprobabilities Ġof Ġcorrect Ġand Ġincorrect Ġpredictions Ġshows Ġthat Ġthey Ġfall Ġinto Ġthe Ġsame Ġrange . ĠNo Ġcommon Ġthreshold Ġcan Ġbe Ġfound . </s> Given Ġthat Ġneg ation Ġhas Ġlittle Ġeffect Ġon Ġmost Ġqueries , Ġit Ġis Ġinteresting Ġto Ġlook Ġat Ġthe Ġsmall Ġnumber Ġof Ġqueries Ġwhere Ġpret rained Ġlanguage Ġmodels Ġmake Ġcorrect Ġpredictions , Ġi . e ., Ġthey Ġsolve Ġthe Ġclo ze Ġtask Ġas Ġa Ġhuman Ġsubject Ġwould Ġdo . ĠWe Ġgive Ġtwo Ġexamples Ġof Ġsuch Ġpatterns . ĠThe Ġpattern ĠâĢ ľ X Ġdid Ġnot Ġdie Ġin ĠY âĢ Ŀ Ġalways Ġresults Ġin Ġthe Ġgeneric Ġtop Ġranked Ġpredictions : ĠâĢ ľ battle âĢ Ŀ , ĠâĢ ľ office âĢ Ŀ , ĠâĢ ľ prison âĢ Ŀ Ġwhereas Ġthe Ġoriginal Ġpattern Ġis Ġlikely Ġto Ġrank Ġcities Ġfirst . ĠThis Ġseems Ġappropriate Ġsince Ġa Ġstatement Ġof Ġthe Ġform , Ġsay , ĠâĢ ľ X Ġdid Ġnot Ġdie Ġin ĠNew ĠYork âĢ Ŀ Ġis Ġrare Ġin Ġtext Ġcorpor a , Ġbut Ġstatements Ġcharacter izing Ġthe Ġsituation Ġin Ġwhich Ġthe Ġdeath Ġoccurred Ġ( âĢ ľ he Ġdid Ġnot Ġdie Ġin Ġprison âĢ Ŀ ) Ġsound Ġmore Ġnatural . ĠFor Ġthe Ġtemplate ĠâĢ ľ X Ġwas Ġborn Ġin ĠY âĢ Ŀ , Ġcities Ġare Ġpredicted . ĠIn Ġcontrast , Ġfor ĠâĢ ľ X Ġwas Ġnot Ġborn Ġin ĠY âĢ Ŀ , Ġcountries Ġare Ġpredicted . ĠBoth Ġtimes Ġit Ġrefers Ġto Ġa Ġmore Ġspecific Ġstatement , Ġmore Ġlikely Ġto Ġoccur Ġin Ġthe Ġtraining Ġcorpus . ĠPeople Ġwould Ġrefer Ġmore Ġoften Ġto Ġa Ġperson Ġbeing Ġborn Ġin Ġa Ġcity Ġand Ġnot Ġborn Ġin Ġa Ġcountry , Ġgiving Ġyou Ġin Ġboth Ġcases Ġmore Ġprecise Ġinformation . </s> Related ĠWork </s> P ret rained Ġembed d ings Ġhave Ġpushed Ġbas elines Ġon Ġa Ġvariety Ġof Ġquestion Ġanswering Ġdatasets ĠB IB REF 8 , ĠB IB REF 9 . ĠGenerally , Ġthe Ġpret rained Ġmodels Ġare Ġfine - tun ed Ġto Ġthe Ġspecific Ġtask ĠB IB REF 10 , ĠB IB REF 3 Ġbut Ġrecent Ġwork Ġhas Ġapplied Ġthe Ġmodels Ġwithout Ġthe Ġfine - tun ing Ġstep ĠB IB REF 11 , ĠB IB REF 0 . </s> There Ġis Ġa Ġwide Ġrange Ġof Ġliterature Ġanalyzing Ġlinguistic Ġknowledge Ġstored Ġin Ġpret rained Ġembed d ings ĠB IB REF 12 , ĠB IB REF 13 , ĠB IB REF 14 , ĠB IB REF 15 , ĠB IB REF 16 , ĠB IB REF 17 , ĠB IB REF 18 , ĠB IB REF 19 . </s> Con cerning Ġneg ation Ġthe Ġfollowing Ġpapers Ġare Ġof Ġinterest : ĠB IB REF 20 Ġanalyze Ġthe Ġgram matical Ġknowledge Ġcaptured Ġby ĠB ERT . ĠIn Ġa Ġcase Ġstudy , Ġthey Ġtest Ġfor Ġcorrect Ġlicensing Ġenvironments Ġfor Ġnegative Ġpol arity Ġitems . ĠThey Ġstudy Ġa Ġset Ġof Ġclass ifiers Ġdistinguishing Ġbetween Ġgram matically Ġcorrect Ġand Ġincorrect Ġsentences . ĠWe Ġtake Ġa Ġdifferent Ġapproach Ġby Ġfocusing Ġon Ġfactual Ġknowledge Ġstored Ġin Ġneg ated Ġstatements . ĠGram matically Ġcorrect Ġstatements Ġcan Ġstill Ġbe Ġfact ually Ġfalse Ġ( e . g ., ĠâĢ ľ General Ġrelativity , ĠNewton Ġdevelop âĢ Ŀ ). </s> BI B REF 21 Ġinvestigate Ġthe Ġunderstanding Ġof Ġfunction Ġwords ĠâĢĵ Ġamong Ġthem Ġneg ation Ġparticles ĠâĢĵ Ġusing Ġan Ġentail ment - Ġand Ġclassification - based Ġapproach . ĠThey Ġanalyze Ġthe Ġability Ġof Ġdifferent Ġmodel Ġarchitectures Ġand Ġtraining Ġobjectives Ġto Ġcapture Ġknowledge Ġof Ġsingle Ġsentences . ĠThe Ġmodels Ġare Ġfine - tun ed Ġto Ġthe Ġtask Ġof Ġinterest . ĠWe Ġon Ġthe Ġother Ġhand Ġquestion Ġto Ġwhat Ġextend Ġfactual Ġknowledge Ġpresent Ġin Ġneg ated Ġstatements Ġis Ġindirectly Ġacquired Ġduring Ġpret raining . </s> BI B REF 22 Ġdefines Ġthree Ġpsych ol ingu istic Ġdiagn ostics Ġfor Ġlanguage Ġmodels Ġand Ġapplies Ġthem Ġin Ġa Ġcase Ġstudy Ġto ĠB ERT . ĠNeg ation Ġis Ġexamined Ġusing Ġa Ġdataset Ġof Ġ72 Ġsimple Ġsentences Ġquer ying Ġfor Ġcategory Ġmembership . ĠA Ġsupplementary Ġdataset Ġof Ġ16 Ġsentences Ġqueries Ġagain Ġfor Ġthe Ġ" to Ġbe " Ġrelation Ġonly Ġbut Ġincluding Ġmore Ġnatural Ġsentence Ġstructure . ĠOur Ġwork Ġcovers Ġ51 , 329 Ġneg ated Ġstatements Ġcovering Ġa Ġwide Ġrange Ġof Ġtopics Ġand Ġrelations . ĠIn Ġthe ĠS Qu AD Ġbased Ġdataset Ġwe Ġcover Ġmore Ġnatural Ġlanguage Ġin Ġterms Ġof Ġcontext Ġand Ġrelation . ĠIn Ġcontrast Ġto ĠB IB REF 22 , Ġwe Ġdo Ġnot Ġsee Ġa Ġreliable Ġpreference Ġof Ġthe Ġtrue Ġcomple tions Ġto Ġfalse Ġin Ġthe Ġmore Ġnatural Ġneg ated Ġstatements . </s> BI B REF 23 Ġtest Ġfor Ġcomprehension Ġof Ġminim ally Ġmodified Ġstatements Ġin Ġan Ġadvers arial Ġsetup Ġwhile Ġtrying Ġto Ġkeep Ġthe Ġoverall Ġsemantics Ġthe Ġsame . ĠWe Ġtry Ġto Ġmaximize Ġthe Ġchange Ġin Ġsemantics Ġand Ġin vert Ġmeaning . </s> Conclusion </s> We Ġshow Ġthat Ġpret rained Ġlanguage Ġmodels Ġhave Ġproblems Ġhandling Ġneg ation . ĠOutput Ġpredictions Ġfor Ġthe Ġoriginal ĠL AMA Ġquery Ġand Ġthe Ġneg ated Ġstatement Ġare Ġhighly Ġcorrelated . </s> Even Ġthough Ġthis Ġelegant Ġapproach Ġof Ġquer ying Ġa Ġlanguage Ġmodel Ġwithout Ġfine - tun ing Ġallows Ġfor Ġtruly Ġopen Ġdomain Ġquestion Ġanswering , Ġpromoting Ġan Ġanswer Ġno Ġmatter Ġwhat Ġis Ġnot Ġalways Ġthe Ġbetter Ġsolution . ĠRef raining Ġfrom Ġgiving Ġan Ġanswer Ġcan Ġbe Ġmore Ġappropriate , Ġmaking Ġknowledge Ġgraphs Ġcurrently Ġa Ġmore Ġreliable Ġchoice Ġfor Ġquestion Ġanswering .