Unanswerable
NONE
1906.01040
Introduction </s> A Ġgrowing Ġbody Ġof Ġwork Ġon Ġadvers arial Ġexamples Ġhas Ġidentified Ġthat Ġfor Ġmachine - learning Ġ( ML ) Ġsystems Ġthat Ġoperate Ġon Ġhigh - dimensional Ġdata , Ġfor Ġnearly Ġevery Ġnatural Ġinput Ġthere Ġexists Ġa Ġsmall Ġpert urb ation Ġof Ġthe Ġpoint Ġthat Ġwill Ġbe Ġmis classified Ġby Ġthe Ġsystem , Ġposing Ġa Ġthreat Ġto Ġits Ġdeployment Ġin Ġcertain Ġcritical Ġsettings ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ. ĠMore Ġbroadly , Ġthe Ġsusceptibility Ġof ĠML Ġsystems Ġto Ġadvers arial Ġexamples Ġhas Ġprompted Ġa Ġre - examination Ġof Ġwhether Ġcurrent ĠML Ġsystems Ġare Ġtruly Ġlearning Ġor Ġif Ġthey Ġare Ġassemb l ages Ġof Ġtricks Ġthat Ġare Ġeffective Ġyet Ġbrittle Ġand Ġeasily Ġfooled ĠB IB REF 9 Ġ. ĠImpl icit Ġin Ġthis Ġline Ġof Ġreasoning Ġis Ġthe Ġassumption Ġthat Ġinstances Ġof ĠâĢ Ŀ real " Ġlearning , Ġsuch Ġas Ġhuman Ġcognition , Ġyield Ġextremely Ġrobust Ġsystems . ĠIndeed , Ġat Ġleast Ġin Ġcomputer Ġvision , Ġhuman Ġperception Ġis Ġregarded Ġas Ġthe Ġgold - standard Ġfor Ġrobust ness Ġto Ġadvers arial Ġexamples . </s> Ev idently , Ġhumans Ġcan Ġbe Ġfooled Ġby Ġa Ġvariety Ġof Ġillusions , Ġwhether Ġthey Ġbe Ġoptical , Ġauditory , Ġor Ġother ; Ġand Ġthere Ġis Ġa Ġlong Ġline Ġof Ġresearch Ġfrom Ġthe Ġcognitive Ġscience Ġand Ġpsychology Ġcommunities Ġinvestigating Ġthese ĠB IB REF 10 Ġ. ĠIn Ġgeneral , Ġhowever , Ġthese Ġillusions Ġare Ġviewed Ġas Ġisolated Ġexamples Ġthat Ġdo Ġnot Ġarise Ġfrequently , Ġand Ġwhich Ġare Ġfar Ġfrom Ġthe Ġinstances Ġencountered Ġin Ġeveryday Ġlife . </s> In Ġthis Ġwork , Ġwe Ġattempt Ġto Ġunderstand Ġhow Ġsusceptible Ġhumans ' Ġperceptual Ġsystems Ġfor Ġnatural Ġspeech Ġare Ġto Ġcarefully Ġdesigned ĠâĢ ľ ad vers arial Ġattacks . âĢ Ŀ ĠWe Ġinvestigate Ġthe Ġdensity Ġof Ġcertain Ġclasses Ġof Ġillusion , Ġthat Ġis , Ġthe Ġfraction Ġof Ġnatural Ġlanguage Ġutter ances Ġwhose Ġcomprehension Ġcan Ġbe Ġaffected Ġby Ġthe Ġillusion . ĠOur Ġstudy Ġcenters Ġaround Ġthe ĠMcG ur k Ġeffect , Ġwhich Ġis Ġthe Ġwell - stud ied Ġphenomenon Ġby Ġwhich Ġthe Ġperception Ġof Ġwhat Ġwe Ġhear Ġcan Ġbe Ġinfluenced Ġby Ġwhat Ġwe Ġsee ĠB IB REF 0 Ġ. ĠA Ġprototyp ical Ġexample Ġis Ġthat Ġthe Ġaudio Ġof Ġthe Ġphon eme ĠâĢ ľ ba a , âĢ Ŀ Ġaccompanied Ġby Ġa Ġvideo Ġof Ġsomeone Ġm ou thing ĠâĢ ľ va a âĢ Ŀ , Ġcan Ġbe Ġperceived Ġas ĠâĢ ľ va a âĢ Ŀ Ġor ĠâĢ ľ ga a âĢ Ŀ Ġ( Figure Ġ1 Ġ). ĠThis Ġeffect Ġpersists Ġeven Ġwhen Ġthe Ġsubject Ġis Ġaware Ġof Ġthe Ġsetup , Ġthough Ġthe Ġstrength Ġof Ġthe Ġeffect Ġvaries Ġsignificantly Ġacross Ġpeople Ġand Ġlanguages Ġand Ġwith Ġfactors Ġsuch Ġas Ġage , Ġgender , Ġand Ġdisorders ĠB IB REF 11 Ġ, ĠB IB REF 12 Ġ, ĠB IB REF 13 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ, ĠB IB REF 16 Ġ, ĠB IB REF 17 Ġ, ĠB IB REF 18 Ġ, ĠB IB REF 19 Ġ. </s> A Ġsignificant Ġdensity Ġof Ġillusion able Ġinstances Ġfor Ġhumans Ġmight Ġpresent Ġsimilar Ġtypes Ġof Ġsecurity Ġrisks Ġas Ġadvers arial Ġexamples Ġdo Ġfor ĠML Ġsystems . ĠAud itory Ġsignals Ġsuch Ġas Ġpublic Ġservice Ġannouncements , Ġinstructions Ġsent Ġto Ġfirst Ġresponders , Ġetc ., Ġcould Ġbe Ġtargeted Ġby Ġa Ġmalicious Ġagent . ĠGiven Ġonly Ġaccess Ġto Ġa Ġscreen Ġwithin Ġeyes ight Ġof Ġthe Ġintended Ġvictims , Ġthe Ġagent Ġmight Ġbe Ġable Ġto Ġsignificantly Ġobfusc ate Ġor Ġalter Ġthe Ġmessage Ġperceived Ġby Ġthose Ġwho Ġsee Ġthe Ġscreen Ġ( even Ġperipher ally ). </s> Related Ġwork </s> Ill usion able Ġinstances Ġfor Ġhumans Ġare Ġsimilar Ġto Ġadvers arial Ġexamples Ġfor ĠML Ġsystems . ĠSt rict ly Ġspeaking , Ġhowever , Ġour Ġinvestigation Ġof Ġthe Ġdensity Ġof Ġnatural Ġlanguage Ġfor Ġwhich ĠMcG ur k Ġillusions Ġcan Ġbe Ġcreated , Ġis Ġnot Ġthe Ġhuman Ġanalog Ġof Ġadvers arial Ġexamples . ĠThe Ġadvers arial Ġexamples Ġfor ĠML Ġsystems Ġare Ġdat ap oint s Ġthat Ġare Ġmis classified , Ġdespite Ġbeing Ġextremely Ġsimilar Ġto Ġa Ġtypical Ġdat ap oint Ġ( that Ġis Ġcorrectly Ġclassified ). ĠOur Ġillusions Ġof Ġmis d ub bed Ġaudio Ġare Ġnot Ġextremely Ġclose Ġto Ġany Ġtypically Ġencountered Ġinput , Ġsince Ġour ĠMcG ur k Ġsamples Ġhave Ġauditory Ġsignals Ġcorresponding Ġto Ġone Ġphon eme / word Ġand Ġvisual Ġsignals Ġcorresponding Ġto Ġanother . ĠAlso , Ġthere Ġis Ġa Ġcompelling Ġargument Ġfor Ġwhy Ġthe ĠMcG ur k Ġconfusion Ġoccurs , Ġnamely Ġthat Ġhuman Ġspeech Ġperception Ġis Ġb im od al Ġ( audio - visual ) Ġin Ġnature Ġwhen Ġlip Ġreading Ġis Ġavailable ĠB IB REF 20 Ġ, ĠB IB REF 21 Ġ. ĠTo Ġthe Ġbest Ġof Ġour Ġknowledge , Ġprior Ġto Ġour Ġwork , Ġthere Ġhas Ġbeen Ġlittle Ġsystematic Ġinvestigation Ġof Ġthe Ġextent Ġto Ġwhich Ġthe ĠMcG ur k Ġeffect , Ġor Ġother Ġtypes Ġof Ġillusions , Ġcan Ġbe Ġmade Ġdense Ġin Ġthe Ġset Ġof Ġinstances Ġencountered Ġin Ġeveryday Ġlife . ĠThe Ġclosest Ġwork Ġis ĠB IB REF 22 Ġ, Ġwhere Ġthe Ġauthors Ġdemonstrate Ġthat Ġsome Ġadvers arial Ġexamples Ġfor Ġcomputer Ġvision Ġsystems Ġalso Ġfool Ġhumans Ġwhen Ġhumans Ġwere Ġgiven Ġless Ġthan Ġa Ġtenth Ġof Ġsecond Ġto Ġview Ġthe Ġimage . ĠHowever , Ġsome Ġof Ġthese Ġexamples Ġseem Ġless Ġsatisfying Ġas Ġthe Ġpert urb ation Ġacts Ġas Ġa Ġpixel - space Ġinterpol ation Ġbetween Ġthe Ġoriginal Ġimage Ġand Ġthe ĠâĢ ľ inc orrect âĢ Ŀ Ġclass . ĠThis Ġresults Ġin Ġimages Ġthat Ġare Ġvisually Ġborderline Ġbetween Ġtwo Ġclasses , Ġand Ġas Ġsuch , Ġdo Ġnot Ġprovide Ġa Ġsense Ġof Ġillusion Ġto Ġthe Ġviewer . ĠIn Ġgeneral , Ġresearchers Ġhave Ġnot Ġprob ed Ġthe Ġrobust ness Ġof Ġhuman Ġperception Ġwith Ġthe Ġsame Ġtools , Ġintent , Ġor Ġperspective , Ġwith Ġwhich Ġthe Ġsecurity Ġcommunity Ġis Ġcurrently Ġinterrog ating Ġthe Ġrobust ness Ġof ĠML Ġsystems . </s> Problem Ġsetup </s> For Ġthe ĠMcG ur k Ġeffect , Ġwe Ġattempt Ġan Ġillusion Ġfor Ġa Ġlanguage Ġtoken Ġ( e . g . Ġphon eme , Ġword , Ġsentence ) Ġ$ x $ Ġby Ġcreating Ġa Ġvideo Ġwhere Ġan Ġaudio Ġstream Ġof Ġ$ x $ Ġis Ġvisually Ġdubbed Ġover Ġby Ġa Ġperson Ġsaying Ġ$ x ^ {\ prime Ġ} \ ne Ġx $ Ġ. ĠWe Ġstress Ġthat Ġthe Ġaudio Ġportion Ġof Ġthe Ġillusion Ġis Ġnot Ġmodified Ġand Ġcorresponds Ġto Ġa Ġperson Ġsaying Ġ$ x $ Ġ. ĠThe Ġillusion Ġ$ f ( x ^ {\ prime Ġ}, x ) $ Ġaffects Ġa Ġlistener Ġif Ġthey Ġperceive Ġwhat Ġis Ġbeing Ġsaid Ġto Ġbe Ġ$ y \ ne Ġx $ Ġif Ġthey Ġwatched Ġthe Ġill us ory Ġvideo Ġwhereas Ġthey Ġperceive Ġ$ x $ Ġif Ġthey Ġhad Ġeither Ġlistened Ġto Ġthe Ġaudio Ġstream Ġwithout Ġwatching Ġthe Ġvideo Ġor Ġhad Ġwatched Ġthe Ġoriginal Ġun al tered Ġvideo , Ġdepending Ġon Ġspecification . ĠWe Ġcall Ġa Ġtoken Ġillusion able Ġif Ġan Ġillusion Ġcan Ġbe Ġmade Ġfor Ġthe Ġtoken Ġthat Ġaffects Ġthe Ġperception Ġof Ġa Ġsignificant Ġfraction Ġof Ġpeople . </s> In ĠSection Ġ" Ph on eme - level Ġexperiments " Ġ, Ġwe Ġanalyze Ġthe Ġextent Ġto Ġwhich Ġthe ĠMcG ur k Ġeffect Ġcan Ġbe Ġused Ġto Ġcreate Ġillusions Ġfor Ġphon em es , Ġwords , Ġand Ġsentences , Ġand Ġanalyze Ġthe Ġfraction Ġof Ġnatural Ġlanguage Ġthat Ġis Ġsusceptible Ġto Ġsuch Ġillusion ability . ĠWe Ġthereby Ġobtain Ġa Ġlower Ġbound Ġon Ġthe Ġdensity Ġof Ġillusion able Ġnatural Ġspeech . </s> We Ġfind Ġthat Ġ1 ) Ġa Ġsignificant Ġfraction Ġof Ġwords Ġthat Ġoccur Ġin Ġeveryday Ġspeech Ġcan Ġbe Ġturned Ġinto ĠMcG ur k - style Ġillusions , Ġ2 ) Ġsuch Ġillusions Ġpersist Ġwhen Ġembedded Ġwithin Ġthe Ġcontext Ġof Ġnatural Ġsentences , Ġand Ġin Ġfact Ġaffect Ġa Ġsignificant Ġfraction Ġof Ġnatural Ġsentences , Ġand Ġ3 ) Ġthe Ġillusion ability Ġof Ġwords Ġand Ġsentences Ġcan Ġbe Ġpredicted Ġusing Ġfeatures Ġfrom Ġnatural Ġlanguage Ġmodeling . </s> Ph on eme - level Ġexperiments </s> We Ġbegan Ġby Ġdetermining Ġwhich Ġphon eme Ġsounds Ġcan Ġbe Ġpaired Ġwith Ġvideo Ġdub s Ġof Ġother Ġphon em es Ġto Ġeffect Ġa Ġperceived Ġphon eme Ġthat Ġis Ġdifferent Ġfrom Ġthe Ġactual Ġsound . ĠWe Ġcreated ĠMcG ur k Ġvideos Ġfor Ġall Ġvowel Ġpairs Ġpreceded Ġwith Ġthe Ġconson ant Ġ// Ġas Ġwell Ġas Ġfor Ġall Ġconson ant Ġpairs Ġfollowed Ġby Ġthe Ġvowel Ġ// Ġspoken Ġby Ġa Ġspeaker . ĠThere Ġare Ġ20 Ġvowel Ġphon em es Ġand Ġ24 Ġconson ant Ġphon em es Ġin ĠAmerican ĠEnglish Ġalthough Ġ/ Ê ¤ / Ġand Ġ/ Ê Ĵ / Ġare Ġredundant Ġfor Ġour Ġpurposes . ĠBased Ġon Ġlabels Ġprovided Ġby Ġ10 Ġindividuals Ġwe Ġfound Ġthat Ġalthough Ġvow els Ġwere Ġnot Ġeasily Ġconfused , Ġthere Ġare Ġa Ġnumber Ġof Ġillusion able Ġconson ants . ĠWe Ġnote Ġthat Ġthe Ġillusion able Ġphon eme Ġpairs Ġdepend Ġboth Ġon Ġthe Ġspeaker Ġand Ġlistener Ġidentities . ĠGiven ĠTable Ġ1 Ġof Ġillusion able Ġphon em es , Ġthe Ġgoal Ġwas Ġthen Ġto Ġunderstand Ġwhether Ġthese Ġcould Ġbe Ġlever aged Ġwithin Ġwords Ġor Ġsentences ; Ġand Ġif Ġso , Ġthe Ġfraction Ġof Ġnatural Ġspeech Ġthat Ġis Ġsusceptible . </s> Word - level Ġexperiments </s> We Ġsampled Ġ200 Ġunique Ġwords Ġ( listed Ġin ĠTable Ġ2 Ġ) Ġfrom Ġthe Ġ10 , 000 Ġmost Ġcommon Ġwords Ġin Ġthe ĠProject ĠGutenberg Ġnovels Ġin Ġproportion Ġto Ġtheir Ġfrequency Ġin Ġthe Ġcorpus . ĠThe Ġ10 k Ġwords Ġcollectively Ġhave Ġa Ġprevalence Ġof Ġ80 . 6 % Ġin Ġthe Ġcorpus . ĠOf Ġthe Ġ200 Ġsampled Ġwords , Ġ147 Ġ( 73 . 5 %) Ġcontained Ġphon em es Ġthat Ġour Ġpreliminary Ġphon eme Ġstudy Ġsuggested Ġmight Ġbe Ġillusion able . ĠFor Ġthese Ġ147 Ġwords , Ġwe Ġpaired Ġaudio Ġclips Ġspoken Ġby Ġthe Ġspeaker Ġwith Ġill us ory Ġvideo Ġdub s Ġof Ġthe Ġspeaker Ġsaying Ġthe Ġwords Ġwith Ġappropriately Ġswitched Ġout Ġphon em es . ĠWe Ġtested Ġthese Ġvideos Ġon Ġ20 Ġnaive Ġtest Ġsubjects Ġwho Ġdid Ġnot Ġparticipate Ġin Ġthe Ġpreliminary Ġstudy . ĠEach Ġsubject Ġwatched Ġhalf Ġof Ġthe Ġwords Ġand Ġlistened Ġwithout Ġvideo Ġto Ġthe Ġother Ġhalf Ġof Ġthe Ġwords , Ġand Ġwere Ġgiven Ġthe Ġinstructions : Ġ" Write Ġdown Ġwhat Ġyou Ġhear . ĠWhat Ġyou Ġhear Ġmay Ġor Ġmay Ġnot Ġbe Ġsens ical . ĠAlso Ġwrite Ġdown Ġif Ġa Ġclip Ġsounds Ġunclear Ġto Ġyou . ĠNot Ġthat Ġa Ġclip Ġmay Ġsound Ġnonsensical Ġbut Ġclear ." ĠSubjects Ġwere Ġallowed Ġup Ġto Ġthree Ġplays Ġof Ġeach Ġclip . </s> We Ġfound Ġthat Ġwatching Ġthe Ġill us ory Ġvideos Ġled Ġto Ġan Ġaverage Ġmis com pre hens ion Ġrate Ġof Ġ24 . 8 %, Ġa Ġrelative Ġ148 % Ġincrease Ġfrom Ġthe Ġbaseline Ġof Ġlistening Ġto Ġthe Ġaudio Ġalone Ġ( Table Ġ3 Ġ). ĠThe Ġill us ory Ġvideos Ġmade Ġpeople Ġless Ġconfident Ġabout Ġtheir Ġcorrect Ġanswers , Ġwith Ġan Ġadditional Ġ5 . 1 % Ġof Ġwords Ġbeing Ġheard Ġcorrectly Ġbut Ġunclear ly , Ġcompared Ġto Ġ2 . 1 % Ġfor Ġaudio Ġonly . ĠFor Ġ17 % Ġof Ġthe Ġ200 Ġwords , Ġthe Ġill us ory Ġvideos Ġincreased Ġthe Ġerror Ġrates Ġby Ġmore Ġthan Ġ30 % Ġabove Ġthe Ġaudio - only Ġbaseline . </s> To Ġcreate Ġa Ġpredictive Ġmodel Ġfor Ġword - level Ġillusion ability , Ġwe Ġused Ġillusion able Ġphon em es Ġenriched Ġwith Ġpositional Ġinformation Ġas Ġfeatures . ĠExplicit ly , Ġfor Ġeach Ġof Ġthe Ġ10 Ġillusion able Ġphon em es , Ġwe Ġcreated Ġthree Ġfeatures Ġfrom Ġthe Ġphon eme Ġbeing Ġin Ġan Ġinitial Ġposition Ġ( being Ġthe Ġfirst Ġphon eme Ġof Ġthe Ġword ), Ġa Ġmedial Ġposition Ġ( having Ġphon em es Ġcome Ġbefore Ġand Ġafter ), Ġor Ġa Ġfinal Ġposition Ġ( being Ġthe Ġlast Ġphon eme Ġof Ġthe Ġword ). ĠWe Ġthen Ġrepresented Ġeach Ġword Ġwith Ġa Ġbinary Ġbag - of - words Ġmodel ĠB IB REF 23 Ġ, Ġgiving Ġeach Ġof Ġthe Ġ30 Ġphon em es - with - ph on etic - context Ġfeatures Ġa Ġvalue Ġof Ġ1 Ġif Ġpresent Ġin Ġthe Ġword Ġand Ġ0 Ġotherwise . ĠWe Ġperformed Ġridge Ġregression Ġon Ġthese Ġfeatures Ġwith Ġa Ġconstant Ġterm . ĠWe Ġsearched Ġfor Ġthe Ġoptimal Ġ$ l 2 $ Ġregular ization Ġconstant Ġamong Ġthe Ġvalues Ġ[ 0 . 1 , Ġ1 , Ġ10 , Ġ100 ] Ġand Ġpicked Ġthe Ġoptimal Ġone Ġbased Ġon Ġtraining Ġset Ġperformance . ĠThe Ġtrain : test Ġsplit Ġwas Ġin Ġthe Ġproportion Ġ85 %: 15 % Ġand Ġwas Ġrandomly Ġchosen Ġfor Ġeach Ġtrial . ĠAcross Ġ10 k Ġrandomized Ġtrials , Ġwe Ġobtain Ġaverage Ġtraining Ġand Ġtest Ġset Ġcorrelations Ġof Ġ$ 91 . 1 \ pm Ġ0 . 6 \ % $ Ġand Ġ$ 44 . 6 \ pm Ġ28 . 9 \ % $ Ġrespectively . </s> Our Ġfinal Ġmodel Ġachieves Ġan Ġout - of - sample Ġcorrelation Ġof Ġ57 % Ġbetween Ġpredicted Ġand Ġobserved Ġillusion abil ites . ĠHere , Ġthe Ġobserved Ġillusion ability Ġof Ġthe Ġwords Ġis Ġcalculated Ġas Ġthe Ġdifference Ġbetween Ġthe Ġaccuracy Ġof Ġwat chers Ġof Ġthe Ġill us ory Ġvideos Ġand Ġthe Ġaccuracy Ġof Ġlisteners , Ġwhere ĠâĢ ľ acc uracy âĢ Ŀ Ġis Ġdefined Ġas Ġthe Ġfraction Ġof Ġrespondents Ġwho Ġwere Ġcorrect . ĠFor Ġeach Ġword , Ġthe Ġpredicted Ġillusion ability Ġis Ġcalculated Ġfrom Ġdoing Ġinference Ġon Ġthat Ġword Ġusing Ġthe Ġaveraged Ġregression Ġcoefficients Ġof Ġthe Ġregression Ġtrials Ġwhere Ġthe Ġword Ġis Ġnot Ġin Ġthe Ġtraining Ġset . </s> Our Ġpredicted Ġillusion ability Ġis Ġalso Ġcalibrated , Ġin Ġthe Ġsense Ġthat Ġfor Ġthe Ġwords Ġpredicted Ġto Ġhave Ġan Ġillusion ability Ġ< 0 . 1 , Ġthe Ġmean Ġempirical Ġillusion ability Ġis Ġ0 . 04 ; Ġfor Ġwords Ġwith Ġpredicted Ġillusion ability Ġin Ġthe Ġinterval Ġ[ 0 . 1 , Ġ0 . 2 ] Ġthe Ġmean Ġempirical Ġillusion ability Ġis Ġ0 . 14 ; Ġfor Ġpredicted Ġillusion ability Ġbetween Ġ[ 0 . 2 , Ġ0 . 3 ] Ġthe Ġmean Ġobserved Ġis Ġ0 . 27 ; Ġand Ġfor Ġpredicted Ġillusion ability Ġ> 0 . 3 , Ġthe Ġmean Ġobserved Ġis Ġ0 . 50 . ĠFigure Ġ2 Ġvisually Ġdepicts Ġthe Ġmatch Ġbetween Ġthe Ġobserved Ġand Ġpredicted Ġword Ġillusion abilities . </s> Sent ence - level Ġexperiments </s> We Ġset Ġup Ġthe Ġfollowing Ġexperiment Ġon Ġnaturally Ġoccurring Ġsentences . ĠWe Ġrandomly Ġsampled Ġ300 Ġsentences Ġof Ġlengths Ġ4 - 8 Ġwords Ġinclusive Ġfrom Ġthe Ġnovel ĠLittle ĠWomen ĠB IB REF 24 Ġfrom Ġthe ĠProject ĠGutenberg Ġcorpus . ĠFrom Ġthis Ġreduced Ġsample , Ġwe Ġselected Ġand Ġpert urbed Ġ32 Ġsentences Ġthat Ġwe Ġexpected Ġto Ġbe Ġillusion able Ġ( listed Ġin ĠTable Ġ4 Ġ). ĠWith Ġthe Ġspeaker , Ġwe Ġprepared Ġtwo Ġformats Ġof Ġeach Ġsentence : Ġoriginal Ġvideo Ġ( with Ġoriginal Ġaudio ), Ġand Ġill us ory Ġvideo Ġ( with Ġoriginal Ġaudio ). We Ġthen Ġevaluated Ġthe Ġperception Ġof Ġthese Ġon Ġ130 6 Ġnaive Ġtest Ġsubjects Ġon ĠAmazon ĠMechanical ĠTurk . ĠThe ĠTurk ers Ġwere Ġshown Ġvideos Ġfor Ġsix Ġrandomly Ġselected Ġsentences , Ġthree Ġill us ory Ġand Ġthree Ġoriginal , Ġand Ġwere Ġgiven Ġthe Ġprompt : Ġ" Press Ġany Ġkey Ġto Ġbegin Ġvideo Ġ[ index Ġ# ] Ġof Ġ6 . ĠWatch Ġthe Ġwhole Ġvideo , Ġand Ġthen Ġyou Ġwill Ġbe Ġprompted Ġto Ġwrite Ġdown Ġwhat Ġthe Ġspeaker Ġsaid ." ĠOnly Ġone Ġviewing Ġof Ġany Ġclip Ġwas Ġallowed , Ġto Ġsimulate Ġthe Ġnatural Ġsetting Ġof Ġobserving Ġa Ġlive Ġaudio / video Ġstream . ĠEach ĠTur ker Ġwas Ġlimited Ġto Ġsix Ġvideos Ġto Ġreduce Ġrespondent Ġfatigue . ĠTurk ers Ġwere Ġalso Ġasked Ġto Ġreport Ġtheir Ġlevel Ġof Ġconfidence Ġin Ġwhat Ġthey Ġheard Ġon Ġa Ġscale Ġfrom Ġno Ġuncertainty Ġ( 0 %) Ġto Ġcomplete Ġuncertainty Ġ( 100 %). ĠOne Ġhundred Ġand Ġtwelve ĠTurk ers Ġ( 8 . 6 %) Ġdid Ġnot Ġadhere Ġto Ġthe Ġprompt , Ġwriting Ġunrelated Ġresponses , Ġand Ġtheir Ġresults Ġwere Ġomitted Ġfrom Ġanalysis . ĠWe Ġfound Ġthat Ġwatching Ġthe Ġill us ory Ġvideos Ġled Ġto Ġan Ġaverage Ġmis com pre hens ion Ġrate Ġof Ġ32 . 8 %, Ġa Ġrelative Ġ145 % Ġincrease Ġfrom Ġthe Ġbaseline Ġof Ġlistening Ġto Ġthe Ġaudio Ġalone Ġ( Table Ġ5 Ġ). ĠThe Ġill us ory Ġvideos Ġmade Ġpeople Ġless Ġconfident Ġabout Ġtheir Ġcorrect Ġanswers . ĠTurk ers Ġwho Ġcorrectly Ġidentified Ġthe Ġaudio Ġmessage Ġin Ġan Ġill us ory Ġvideo Ġself - reported Ġan Ġaverage Ġuncertainty Ġof Ġ42 . 9 %, Ġwhich Ġis Ġa Ġrelative Ġ123 % Ġhigher Ġthan Ġthe Ġaverage Ġreported Ġby Ġthe ĠTurk ers Ġwho Ġcorrectly Ġunderstood Ġthe Ġoriginal Ġvideos . ĠExamples Ġof Ġmistakes Ġmade Ġby Ġlisteners Ġof Ġthe Ġill us ory Ġvideos Ġare Ġshown Ġin ĠTable Ġ6 Ġ. ĠOverall Ġwe Ġfound Ġthat Ġfor Ġ11 . 5 % Ġof Ġthe Ġ200 Ġsampled Ġsentences Ġ( 23 Ġout Ġof Ġthe Ġ30 Ġvideos Ġwe Ġcreated ), Ġthe Ġill us ory Ġvideos Ġincreased Ġthe Ġerror Ġrates Ġby Ġmore Ġthan Ġ10 % Ġabove Ġthe Ġaudio - only Ġbaseline . </s> We Ġobtained Ġa Ġsentence - level Ġillusion ability Ġprediction Ġmodel Ġwith Ġan Ġout - of - sample Ġcorrelation Ġof Ġ33 % Ġbetween Ġpredicted Ġand Ġobserved Ġillusion abilities . ĠHere , Ġthe Ġobserved Ġillusion ability Ġof Ġthe Ġsentences Ġwas Ġcalculated Ġas Ġthe Ġdifference Ġbetween Ġthe Ġaccuracy Ġof Ġwat chers Ġof Ġthe Ġill us ory Ġvideos Ġand Ġthe Ġaccuracy Ġof Ġwat chers Ġof Ġoriginal Ġvideos , Ġwhere ĠâĢ ľ acc uracy âĢ Ŀ Ġis Ġdefined Ġas Ġthe Ġfraction Ġof Ġrespondents Ġwho Ġwere Ġcorrect . ĠWe Ġobtained Ġpredicted Ġillusion abilities Ġby Ġsimply Ġusing Ġthe Ġmaximum Ġword Ġillusion ability Ġprediction Ġamongst Ġthe Ġwords Ġin Ġeach Ġsentence , Ġwith Ġword Ġpredictions Ġobtained Ġfrom Ġthe Ġword - level Ġmodel . ĠWe Ġattempted Ġto Ġimprove Ġour Ġsentence - level Ġpredictive Ġmodel Ġby Ġincorporating Ġhow Ġlikely Ġthe Ġwords Ġappear Ġunder Ġa Ġnatural Ġlanguage Ġdistribution , Ġconsidering Ġthree Ġclasses Ġof Ġwords : Ġwords Ġin Ġthe Ġsentence Ġfor Ġwhich Ġno Ġillusion Ġwas Ġattempted , Ġthe Ġwords Ġfor Ġwhich Ġan Ġillusion Ġwas Ġattempted , Ġand Ġthe Ġpotentially Ġperceived Ġwords Ġfor Ġwords Ġfor Ġwhich Ġan Ġillusion Ġwas Ġattempted . ĠWe Ġused Ġlog Ġword Ġfrequencies Ġobtained Ġfrom Ġthe Ġthe Ġtop Ġ36 . 7 k Ġmost Ġcommon Ġwords Ġfrom Ġthe ĠProject ĠGutenberg Ġcorpus . ĠThis Ġapproach Ġcould Ġnot Ġattain Ġbetter Ġout - of - sample Ġcorrelations Ġthan Ġthe Ġnaive Ġmethod . ĠThis Ġimplies Ġthat Ġcontext Ġis Ġimportant Ġfor Ġsentence - level Ġillusion ability , Ġand Ġmore Ġcomplex Ġlanguage Ġmodels Ġshould Ġbe Ġused . </s> Finally , Ġcomparing Ġword - level Ġand Ġsentence - level ĠMcG ur k Ġillusion abilities Ġin Ġnatural Ġspeech , Ġwe Ġobserve Ġthat Ġthat Ġthe Ġformer Ġis Ġsignificantly Ġhigher . ĠA Ġgreater ĠMcG ur k Ġeffect Ġat Ġthe Ġword Ġlevel Ġis Ġto Ġbe Ġexpected âĢĵ sent ences Ġprovide Ġcontext Ġwith Ġwhich Ġthe Ġviewer Ġcould Ġfill Ġin Ġconf usions Ġand Ġmisunderstand ings . ĠFurthermore , Ġwhen Ġwatching Ġa Ġsentence Ġvideo Ġcompared Ġto Ġa Ġshort Ġword Ġvideo , Ġthe Ġviewer 's Ġattention Ġis Ġmore Ġlikely Ġto Ġstray , Ġboth Ġfrom Ġthe Ġvisual Ġcomponent Ġof Ġthe Ġvideo , Ġwhich Ġevidently Ġreduces Ġthe ĠMcG ur k Ġeffect , Ġas Ġwell Ġas Ġthe Ġfrom Ġthe Ġaudio Ġcomponent , Ġwhich Ġlikely Ġprompts Ġthe Ġviewer Ġto Ġrely Ġeven Ġmore Ġheavily Ġon Ġcontext . ĠNevertheless , Ġthere Ġremains Ġa Ġsignificant Ġamount Ġof Ġillusion ability Ġat Ġthe Ġsentence - level . </s> Future ĠDirections </s> This Ġwork Ġis Ġan Ġinitial Ġstep Ġtowards Ġexploring Ġthe Ġdensity Ġof Ġillusion able Ġphenomena Ġfor Ġhumans . ĠThere Ġare Ġmany Ġnatural Ġdirections Ġfor Ġfuture Ġwork . ĠIn Ġthe Ġvein Ġof Ġfurther Ġunderstanding ĠMcG ur k - style Ġillusions , Ġit Ġseems Ġworth Ġbuilding Ġmore Ġaccurate Ġpredictive Ġmodels Ġfor Ġsentence - level Ġeffects , Ġand Ġfurther Ġinvestigating Ġthe Ġsecurity Ġrisks Ġposed Ġby ĠMcG ur k Ġillusions . ĠFor Ġexample , Ġone Ġconcrete Ġnext Ġstep Ġin Ġunderstanding ĠMcG ur k - style Ġillusions Ġwould Ġbe Ġto Ġactually Ġimplement Ġa Ġsystem Ġwhich Ġtakes Ġan Ġaudio Ġinput , Ġand Ġoutputs Ġa Ġvideo Ġdub Ġresulting Ġin Ġsignificant Ġmisunderstanding . ĠSuch Ġa Ġsystem Ġwould Ġneed Ġto Ġcombine Ġa Ġhigh - quality Ġspeech - to - video - sy nt hesis Ġsystem ĠB IB REF 25 Ġ, ĠB IB REF 26 Ġ, Ġwith Ġa Ġfl es hed - out Ġlanguage Ġmodel Ġand ĠMcG ur k Ġprediction Ġmodel . ĠThere Ġis Ġalso Ġthe Ġquestion Ġof Ġhow Ġto Ġguard Ġagainst ĠâĢ ľ attacks âĢ Ŀ Ġon Ġhuman Ġperception . ĠFor Ġexample , Ġin Ġthe Ġcase Ġof Ġthe ĠMcG ur k Ġeffect , Ġhow Ġcan Ġone Ġre phrase Ġa Ġpassage Ġof Ġtext Ġin Ġsuch Ġa Ġway Ġthat Ġthe Ġmeaning Ġis Ġunchanged , Ġbut Ġthe Ġre ph r ased Ġtext Ġis Ġsignificantly Ġmore Ġrobust Ġto ĠMcG ur k Ġstyle Ġmanip ulations ? ĠThe Ġcentral Ġquestion Ġin Ġthis Ġdirection Ġis Ġwhat Ġfraction Ġof Ġnatural Ġlanguage Ġcan Ġbe Ġmade Ġrobust Ġwithout Ġsignificantly Ġchanging Ġthe Ġsemantics . </s> A Ġbetter Ġunderstanding Ġof Ġwhen Ġand Ġwhy Ġcertain Ġhuman Ġperception Ġsystems Ġare Ġnon rob ust Ġcan Ġalso Ġbe Ġapplied Ġto Ġmake ĠML Ġsystems Ġmore Ġrobust . ĠIn Ġparticular , Ġneural Ġnetworks Ġhave Ġbeen Ġfound Ġto Ġbe Ġsusceptible Ġto Ġadvers arial Ġexamples Ġin Ġautomatic Ġspeech Ġrecognition ĠB IB REF 27 Ġ, ĠB IB REF 28 Ġand Ġto Ġthe ĠMcG ur k Ġeffect ĠB IB REF 29 Ġ, Ġand Ġa Ġrudimentary Ġapproach Ġto Ġmaking Ġlanguage Ġrobust Ġto Ġthe Ġlatter Ġproblem Ġwould Ġbe Ġto Ġuse Ġa Ġreduced Ġvocabulary Ġthat Ġavoids Ġwords Ġthat Ġscore Ġhighly Ġin Ġour Ġword - level Ġillusion ability Ġprediction Ġmodel . ĠRelated ly , Ġat Ġthe Ġinterface Ġof Ġcognitive Ġscience Ġand Ġadvers arial Ġexamples , Ġthere Ġhas Ġbeen Ġwork Ġsuggesting Ġthat Ġhumans Ġcan Ġanticipate Ġwhen Ġor Ġhow Ġmachines Ġwill Ġmis class ify , Ġincluding Ġfor Ġadvers arial Ġexamples ĠB IB REF 30 Ġ, ĠB IB REF 31 Ġ, ĠB IB REF 32 Ġ. </s> More Ġbroadly , Ġas Ġthe Ġtools Ġfor Ġprobing Ġthe Ġweaknesses Ġof ĠML Ġsystems Ġdevelop Ġfurther , Ġit Ġseems Ġlike Ġa Ġnatural Ġtime Ġto Ġre ex amine Ġthe Ġsupposed Ġrobust ness Ġof Ġhuman Ġperception . ĠWe Ġanticipate Ġunexpected Ġfindings . ĠTo Ġprovide Ġone Ġexample , Ġwe Ġsummarize Ġsome Ġpreliminary Ġresults Ġon Ġaudio - only Ġillusions . </s> Aud itory ĠIll usions </s> An Ġaudio Ġclip Ġof Ġthe Ġword ĠâĢ ľ La ure l " Ġgained Ġwidespread Ġattention Ġin Ġ2018 , Ġwith Ġcoverage Ġby Ġnotable Ġnews Ġoutlets Ġsuch Ġas ĠThe ĠNew ĠYork ĠTimes Ġand ĠTime . ĠRough ly Ġhalf Ġof Ġlisteners Ġperceive ĠâĢ ľ La ure l âĢ Ŀ Ġand Ġthe Ġother Ġhalf Ġperceive ĠâĢ ľ Y anny âĢ Ŀ Ġor Ġsimilar - sounding Ġwords , Ġwith Ġhigh Ġconfidence Ġon Ġboth Ġsides ĠB IB REF 1 Ġ. ĠOne Ġof Ġthe Ġreasons Ġthe Ġpublic Ġwas Ġintrigued Ġis Ġbecause Ġexamples Ġof Ġsuch Ġphenomena Ġare Ġviewed Ġas Ġrare , Ġisolated Ġinstances . ĠIn Ġa Ġpreliminary Ġattempt Ġto Ġinvestigate Ġthe Ġdensity Ġof Ġsuch Ġphenomena , Ġwe Ġidentified Ġfive Ġadditional Ġdistinct Ġexamples Ġ( Table Ġ7 Ġ). ĠThe Ġsupplementary Ġfiles Ġinclude Ġ10 Ġversions Ġof Ġone Ġof Ġthese Ġexamples , Ġwhere Ġlisteners Ġtend Ġto Ġperceive Ġeither ĠâĢ ľ world s âĢ Ŀ Ġor ĠâĢ ľ y ikes . âĢ Ŀ ĠAcross Ġthe Ġdifferent Ġaudio Ġclips , Ġone Ġshould Ġbe Ġable Ġto Ġhear Ġboth Ġinterpretations . ĠThe Ġthreshold Ġfor Ġswitching Ġfrom Ġone Ġinterpretation Ġto Ġanother Ġdiffers Ġfrom Ġperson Ġto Ġperson . </s> These Ġexamples Ġwere Ġgenerated Ġby Ġexamining Ġ5000 Ġwords , Ġand Ġselecting Ġthe Ġ50 Ġwhose Ġspect rog rams Ġcontain Ġa Ġbalance Ġof Ġhigh Ġand Ġlow Ġfrequency Ġcomponents Ġthat Ġmost Ġclosely Ġmatched Ġthose Ġfor Ġthe Ġword ĠâĢ ľ La ure l ". ĠEach Ġaudio Ġfile Ġcorrespond ed Ġto Ġthe ĠGoogle ĠCloud ĠText - to - Spe ech ĠAPI Ġsynthesis Ġof Ġa Ġword , Ġafter Ġlow Ġfrequencies Ġwere Ġd amped Ġand Ġthe Ġaudio Ġwas Ġslowed Ġ$ 1 . 3 $ Ġ- Ġ$ 1 . 9 $ Ġx . ĠAfter Ġlistening Ġto Ġthese Ġtop Ġ50 Ġcandidates , Ġwe Ġevaluated Ġthe Ġmost Ġpromising Ġfive Ġon Ġa Ġset Ġof Ġ15 Ġindividuals Ġ( 3 Ġfemale , Ġ12 Ġmale , Ġage Ġrange Ġ22 - 33 ). ĠWe Ġfound Ġmultiple Ġdistribution al Ġmodes Ġof Ġperceptions Ġfor Ġall Ġfive Ġaudio Ġclips . ĠFor Ġexample , Ġa Ġclip Ġof ĠâĢ ľ world s âĢ Ŀ Ġwith Ġthe Ġhigh Ġfrequencies Ġd amped Ġand Ġslowed Ġdown Ġ1 . 5 x Ġwas Ġperceived Ġby Ġfive Ġlisteners Ġas ĠâĢ ľ world s âĢ Ŀ , Ġfour Ġas ĠâĢ ľ y ikes / y ites âĢ Ŀ Ġand Ġsix Ġas ĠâĢ ľ n ights / lights âĢ Ŀ . ĠWhile Ġthese Ġexperiments Ġdo Ġnot Ġdemonstrate Ġa Ġdensity Ġof Ġsuch Ġexamples Ġwith Ġrespect Ġto Ġthe Ġset Ġof Ġall Ġwords âĢĶ and Ġit Ġis Ġunlikely Ġthat Ġill us ory Ġaudio Ġtracks Ġin Ġthis Ġstyle Ġcan Ġbe Ġcreated Ġfor Ġthe Ġmajority Ġof Ġwords âĢĶ they Ġillustrate Ġthat Ġeven Ġthe Ġsurprising ĠY anny Ġor ĠLaurel Ġphenomenon Ġis Ġnot Ġan Ġisolated Ġoccurrence . ĠIt Ġremains Ġto Ġbe Ġseen Ġhow Ġdense Ġsuch Ġphenomena Ġcan Ġbe , Ġgiven Ġthe Ġright Ġsort Ġof Ġsubtle Ġaudio Ġmanipulation . Ġ` </s> Conclusion </s> Our Ġwork Ġsuggests Ġthat Ġfor Ġa Ġsignificant Ġfraction Ġof Ġnatural Ġspeech , Ġhuman Ġperception Ġcan Ġbe Ġaltered Ġby Ġusing Ġsubtle , Ġlearn able Ġpert urb ations . ĠThis Ġis Ġan Ġinitial Ġstep Ġtowards Ġexploring Ġthe Ġdensity Ġof Ġillusion able Ġphenomenon Ġfor Ġhumans , Ġand Ġexamining Ġthe Ġextent Ġto Ġwhich Ġhuman Ġperception Ġmay Ġbe Ġvulnerable Ġto Ġsecurity Ġrisks Ġlike Ġthose Ġthat Ġadvers arial Ġexamples Ġpresent Ġfor ĠML Ġsystems . </s> We Ġhope Ġour Ġwork Ġinspires Ġfuture Ġinvestigations Ġinto Ġthe Ġdiscovery , Ġgeneration , Ġand Ġquant ification Ġof Ġmultim od al Ġand Ġunim od al Ġaud iov is ual Ġand Ġauditory Ġillusions Ġfor Ġhumans . ĠThere Ġexist Ġmany Ġopen Ġresearch Ġquestions Ġon Ġwhen Ġand Ġwhy Ġhumans Ġare Ġsusceptible Ġto Ġvarious Ġtypes Ġof Ġillusions , Ġhow Ġto Ġmodel Ġthe Ġillusion ability Ġof Ġnatural Ġlanguage , Ġand Ġhow Ġnatural Ġlanguage Ġcan Ġbe Ġmade Ġmore Ġrobust Ġto Ġill us ory Ġpert urb ations . ĠAdditionally , Ġwe Ġhope Ġsuch Ġinvestigations Ġinform Ġour Ġinterpretations Ġof Ġthe Ġstrengths Ġand Ġweaknesses Ġof Ġcurrent ĠML Ġsystems . ĠFinally , Ġthere Ġis Ġthe Ġpossibility Ġthat Ġsome Ġvulnerability Ġto Ġcarefully Ġcrafted Ġadvers arial Ġexamples Ġmay Ġbe Ġinherent Ġto Ġall Ġcomplex Ġlearning Ġsystems Ġthat Ġinteract Ġwith Ġhigh - dimensional Ġinputs Ġin Ġan Ġenvironment Ġwith Ġlimited Ġdata ; Ġany Ġthorough Ġinvestigation Ġof Ġthis Ġquestion Ġmust Ġalso Ġprobe Ġthe Ġhuman Ġcognitive Ġsystem . </s> Acknowled gements </s> This Ġresearch Ġwas Ġsupported Ġin Ġpart Ġby ĠNS F Ġaward ĠAF : 18 130 49 , Ġan ĠON R ĠYoung ĠInvestigator Ġaward Ġ( N 0001 4 - 18 - 1 - 22 95 ), Ġand Ġa Ġgrant Ġfrom ĠStanford 's ĠInstitute Ġfor ĠHuman - Cent ered ĠAI . ĠThe Ġauthors Ġwould Ġlike Ġto Ġthank ĠJean ĠBetter ton , ĠShiv am ĠGarg , ĠNoah ĠGoodman , ĠKelvin ĠGu u , ĠMichelle ĠLee , ĠPercy ĠLiang , ĠAle ks ander ĠMake lov , ĠJacob ĠPl ach ta , ĠJacob ĠStein hardt , ĠJim ĠTerry , Ġand ĠAlexander ĠWhat ley Ġfor Ġuseful Ġfeedback Ġon Ġthe Ġwork . ĠThe Ġresearch Ġwas Ġdone Ġunder ĠStanford ĠIR B ĠProtocol Ġ4 64 30 .
Introduction A growing body of work on adversarial examples has identified that for machine-learning (ML) systems that operate on high-dimensional data, for nearly every natural input there exists a small perturbation of the point that will be misclassified by the system, posing a threat to its deployment in certain critical settings BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 . More broadly, the susceptibility of ML systems to adversarial examples has prompted a re-examination of whether current ML systems are truly learning or if they are assemblages of tricks that are effective yet brittle and easily fooled BIBREF9 . Implicit in this line of reasoning is the assumption that instances of ”real" learning, such as human cognition, yield extremely robust systems. Indeed, at least in computer vision, human perception is regarded as the gold-standard for robustness to adversarial examples. Evidently, humans can be fooled by a variety of illusions, whether they be optical, auditory, or other; and there is a long line of research from the cognitive science and psychology communities investigating these BIBREF10 . In general, however, these illusions are viewed as isolated examples that do not arise frequently, and which are far from the instances encountered in everyday life. In this work, we attempt to understand how susceptible humans' perceptual systems for natural speech are to carefully designed “adversarial attacks.” We investigate the density of certain classes of illusion, that is, the fraction of natural language utterances whose comprehension can be affected by the illusion. Our study centers around the McGurk effect, which is the well-studied phenomenon by which the perception of what we hear can be influenced by what we see BIBREF0 . A prototypical example is that the audio of the phoneme “baa,” accompanied by a video of someone mouthing “vaa”, can be perceived as “vaa” or “gaa” (Figure 1 ). This effect persists even when the subject is aware of the setup, though the strength of the effect varies significantly across people and languages and with factors such as age, gender, and disorders BIBREF11 , BIBREF12 , BIBREF13 , BIBREF14 , BIBREF15 , BIBREF16 , BIBREF17 , BIBREF18 , BIBREF19 . A significant density of illusionable instances for humans might present similar types of security risks as adversarial examples do for ML systems. Auditory signals such as public service announcements, instructions sent to first responders, etc., could be targeted by a malicious agent. Given only access to a screen within eyesight of the intended victims, the agent might be able to significantly obfuscate or alter the message perceived by those who see the screen (even peripherally). Related work Illusionable instances for humans are similar to adversarial examples for ML systems. Strictly speaking, however, our investigation of the density of natural language for which McGurk illusions can be created, is not the human analog of adversarial examples. The adversarial examples for ML systems are datapoints that are misclassified, despite being extremely similar to a typical datapoint (that is correctly classified). Our illusions of misdubbed audio are not extremely close to any typically encountered input, since our McGurk samples have auditory signals corresponding to one phoneme/word and visual signals corresponding to another. Also, there is a compelling argument for why the McGurk confusion occurs, namely that human speech perception is bimodal (audio-visual) in nature when lip reading is available BIBREF20 , BIBREF21 . To the best of our knowledge, prior to our work, there has been little systematic investigation of the extent to which the McGurk effect, or other types of illusions, can be made dense in the set of instances encountered in everyday life. The closest work is BIBREF22 , where the authors demonstrate that some adversarial examples for computer vision systems also fool humans when humans were given less than a tenth of second to view the image. However, some of these examples seem less satisfying as the perturbation acts as a pixel-space interpolation between the original image and the “incorrect” class. This results in images that are visually borderline between two classes, and as such, do not provide a sense of illusion to the viewer. In general, researchers have not probed the robustness of human perception with the same tools, intent, or perspective, with which the security community is currently interrogating the robustness of ML systems. Problem setup For the McGurk effect, we attempt an illusion for a language token (e.g. phoneme, word, sentence) $x$ by creating a video where an audio stream of $x$ is visually dubbed over by a person saying $x^{\prime }\ne x$ . We stress that the audio portion of the illusion is not modified and corresponds to a person saying $x$ . The illusion $f(x^{\prime },x)$ affects a listener if they perceive what is being said to be $y\ne x$ if they watched the illusory video whereas they perceive $x$ if they had either listened to the audio stream without watching the video or had watched the original unaltered video, depending on specification. We call a token illusionable if an illusion can be made for the token that affects the perception of a significant fraction of people. In Section "Phoneme-level experiments" , we analyze the extent to which the McGurk effect can be used to create illusions for phonemes, words, and sentences, and analyze the fraction of natural language that is susceptible to such illusionability. We thereby obtain a lower bound on the density of illusionable natural speech. We find that 1) a significant fraction of words that occur in everyday speech can be turned into McGurk-style illusions, 2) such illusions persist when embedded within the context of natural sentences, and in fact affect a significant fraction of natural sentences, and 3) the illusionability of words and sentences can be predicted using features from natural language modeling. Phoneme-level experiments We began by determining which phoneme sounds can be paired with video dubs of other phonemes to effect a perceived phoneme that is different from the actual sound. We created McGurk videos for all vowel pairs preceded with the consonant // as well as for all consonant pairs followed by the vowel // spoken by a speaker. There are 20 vowel phonemes and 24 consonant phonemes in American English although /ʤ/ and /ʒ/ are redundant for our purposes. Based on labels provided by 10 individuals we found that although vowels were not easily confused, there are a number of illusionable consonants. We note that the illusionable phoneme pairs depend both on the speaker and listener identities. Given Table 1 of illusionable phonemes, the goal was then to understand whether these could be leveraged within words or sentences; and if so, the fraction of natural speech that is susceptible. Word-level experiments We sampled 200 unique words (listed in Table 2 ) from the 10,000 most common words in the Project Gutenberg novels in proportion to their frequency in the corpus. The 10k words collectively have a prevalence of 80.6% in the corpus. Of the 200 sampled words, 147 (73.5%) contained phonemes that our preliminary phoneme study suggested might be illusionable. For these 147 words, we paired audio clips spoken by the speaker with illusory video dubs of the speaker saying the words with appropriately switched out phonemes. We tested these videos on 20 naive test subjects who did not participate in the preliminary study. Each subject watched half of the words and listened without video to the other half of the words, and were given the instructions: "Write down what you hear. What you hear may or may not be sensical. Also write down if a clip sounds unclear to you. Not that a clip may sound nonsensical but clear." Subjects were allowed up to three plays of each clip. We found that watching the illusory videos led to an average miscomprehension rate of 24.8%, a relative 148% increase from the baseline of listening to the audio alone (Table 3 ). The illusory videos made people less confident about their correct answers, with an additional 5.1% of words being heard correctly but unclearly, compared to 2.1% for audio only. For 17% of the 200 words, the illusory videos increased the error rates by more than 30% above the audio-only baseline. To create a predictive model for word-level illusionability, we used illusionable phonemes enriched with positional information as features. Explicitly, for each of the 10 illusionable phonemes, we created three features from the phoneme being in an initial position (being the first phoneme of the word), a medial position (having phonemes come before and after), or a final position (being the last phoneme of the word). We then represented each word with a binary bag-of-words model BIBREF23 , giving each of the 30 phonemes-with-phonetic-context features a value of 1 if present in the word and 0 otherwise. We performed ridge regression on these features with a constant term. We searched for the optimal $l2$ regularization constant among the values [0.1, 1, 10, 100] and picked the optimal one based on training set performance. The train:test split was in the proportion 85%:15% and was randomly chosen for each trial. Across 10k randomized trials, we obtain average training and test set correlations of $91.1\pm 0.6\%$ and $44.6\pm 28.9\%$ respectively. Our final model achieves an out-of-sample correlation of 57% between predicted and observed illusionabilites. Here, the observed illusionability of the words is calculated as the difference between the accuracy of watchers of the illusory videos and the accuracy of listeners, where “accuracy” is defined as the fraction of respondents who were correct. For each word, the predicted illusionability is calculated from doing inference on that word using the averaged regression coefficients of the regression trials where the word is not in the training set. Our predicted illusionability is also calibrated, in the sense that for the words predicted to have an illusionability <0.1, the mean empirical illusionability is 0.04; for words with predicted illusionability in the interval [0.1, 0.2] the mean empirical illusionability is 0.14; for predicted illusionability between [0.2, 0.3] the mean observed is 0.27; and for predicted illusionability >0.3, the mean observed is 0.50. Figure 2 visually depicts the match between the observed and predicted word illusionabilities. Sentence-level experiments We set up the following experiment on naturally occurring sentences. We randomly sampled 300 sentences of lengths 4-8 words inclusive from the novel Little Women BIBREF24 from the Project Gutenberg corpus. From this reduced sample, we selected and perturbed 32 sentences that we expected to be illusionable (listed in Table 4 ). With the speaker, we prepared two formats of each sentence: original video (with original audio), and illusory video (with original audio).We then evaluated the perception of these on 1306 naive test subjects on Amazon Mechanical Turk. The Turkers were shown videos for six randomly selected sentences, three illusory and three original, and were given the prompt: "Press any key to begin video [index #] of 6. Watch the whole video, and then you will be prompted to write down what the speaker said." Only one viewing of any clip was allowed, to simulate the natural setting of observing a live audio/video stream. Each Turker was limited to six videos to reduce respondent fatigue. Turkers were also asked to report their level of confidence in what they heard on a scale from no uncertainty (0%) to complete uncertainty (100%). One hundred and twelve Turkers (8.6%) did not adhere to the prompt, writing unrelated responses, and their results were omitted from analysis. We found that watching the illusory videos led to an average miscomprehension rate of 32.8%, a relative 145% increase from the baseline of listening to the audio alone (Table 5 ). The illusory videos made people less confident about their correct answers. Turkers who correctly identified the audio message in an illusory video self-reported an average uncertainty of 42.9%, which is a relative 123% higher than the average reported by the Turkers who correctly understood the original videos. Examples of mistakes made by listeners of the illusory videos are shown in Table 6 . Overall we found that for 11.5% of the 200 sampled sentences (23 out of the 30 videos we created), the illusory videos increased the error rates by more than 10% above the audio-only baseline. We obtained a sentence-level illusionability prediction model with an out-of-sample correlation of 33% between predicted and observed illusionabilities. Here, the observed illusionability of the sentences was calculated as the difference between the accuracy of watchers of the illusory videos and the accuracy of watchers of original videos, where “accuracy” is defined as the fraction of respondents who were correct. We obtained predicted illusionabilities by simply using the maximum word illusionability prediction amongst the words in each sentence, with word predictions obtained from the word-level model. We attempted to improve our sentence-level predictive model by incorporating how likely the words appear under a natural language distribution, considering three classes of words: words in the sentence for which no illusion was attempted, the words for which an illusion was attempted, and the potentially perceived words for words for which an illusion was attempted. We used log word frequencies obtained from the the top 36.7k most common words from the Project Gutenberg corpus. This approach could not attain better out-of-sample correlations than the naive method. This implies that context is important for sentence-level illusionability, and more complex language models should be used. Finally, comparing word-level and sentence-level McGurk illusionabilities in natural speech, we observe that that the former is significantly higher. A greater McGurk effect at the word level is to be expected–sentences provide context with which the viewer could fill in confusions and misunderstandings. Furthermore, when watching a sentence video compared to a short word video, the viewer's attention is more likely to stray, both from the visual component of the video, which evidently reduces the McGurk effect, as well as the from the audio component, which likely prompts the viewer to rely even more heavily on context. Nevertheless, there remains a significant amount of illusionability at the sentence-level. Future Directions This work is an initial step towards exploring the density of illusionable phenomena for humans. There are many natural directions for future work. In the vein of further understanding McGurk-style illusions, it seems worth building more accurate predictive models for sentence-level effects, and further investigating the security risks posed by McGurk illusions. For example, one concrete next step in understanding McGurk-style illusions would be to actually implement a system which takes an audio input, and outputs a video dub resulting in significant misunderstanding. Such a system would need to combine a high-quality speech-to-video-synthesis system BIBREF25 , BIBREF26 , with a fleshed-out language model and McGurk prediction model. There is also the question of how to guard against “attacks” on human perception. For example, in the case of the McGurk effect, how can one rephrase a passage of text in such a way that the meaning is unchanged, but the rephrased text is significantly more robust to McGurk style manipulations? The central question in this direction is what fraction of natural language can be made robust without significantly changing the semantics. A better understanding of when and why certain human perception systems are nonrobust can also be applied to make ML systems more robust. In particular, neural networks have been found to be susceptible to adversarial examples in automatic speech recognition BIBREF27 , BIBREF28 and to the McGurk effect BIBREF29 , and a rudimentary approach to making language robust to the latter problem would be to use a reduced vocabulary that avoids words that score highly in our word-level illusionability prediction model. Relatedly, at the interface of cognitive science and adversarial examples, there has been work suggesting that humans can anticipate when or how machines will misclassify, including for adversarial examples BIBREF30 , BIBREF31 , BIBREF32 . More broadly, as the tools for probing the weaknesses of ML systems develop further, it seems like a natural time to reexamine the supposed robustness of human perception. We anticipate unexpected findings. To provide one example, we summarize some preliminary results on audio-only illusions. Auditory Illusions An audio clip of the word “Laurel" gained widespread attention in 2018, with coverage by notable news outlets such as The New York Times and Time. Roughly half of listeners perceive “Laurel” and the other half perceive “Yanny” or similar-sounding words, with high confidence on both sides BIBREF1 . One of the reasons the public was intrigued is because examples of such phenomena are viewed as rare, isolated instances. In a preliminary attempt to investigate the density of such phenomena, we identified five additional distinct examples (Table 7 ). The supplementary files include 10 versions of one of these examples, where listeners tend to perceive either “worlds” or “yikes.” Across the different audio clips, one should be able to hear both interpretations. The threshold for switching from one interpretation to another differs from person to person. These examples were generated by examining 5000 words, and selecting the 50 whose spectrograms contain a balance of high and low frequency components that most closely matched those for the word “Laurel". Each audio file corresponded to the Google Cloud Text-to-Speech API synthesis of a word, after low frequencies were damped and the audio was slowed $1.3$ - $1.9$ x. After listening to these top 50 candidates, we evaluated the most promising five on a set of 15 individuals (3 female, 12 male, age range 22-33). We found multiple distributional modes of perceptions for all five audio clips. For example, a clip of “worlds” with the high frequencies damped and slowed down 1.5x was perceived by five listeners as “worlds”, four as “yikes/yites” and six as “nights/lights”. While these experiments do not demonstrate a density of such examples with respect to the set of all words—and it is unlikely that illusory audio tracks in this style can be created for the majority of words—they illustrate that even the surprising Yanny or Laurel phenomenon is not an isolated occurrence. It remains to be seen how dense such phenomena can be, given the right sort of subtle audio manipulation. ` Conclusion Our work suggests that for a significant fraction of natural speech, human perception can be altered by using subtle, learnable perturbations. This is an initial step towards exploring the density of illusionable phenomenon for humans, and examining the extent to which human perception may be vulnerable to security risks like those that adversarial examples present for ML systems. We hope our work inspires future investigations into the discovery, generation, and quantification of multimodal and unimodal audiovisual and auditory illusions for humans. There exist many open research questions on when and why humans are susceptible to various types of illusions, how to model the illusionability of natural language, and how natural language can be made more robust to illusory perturbations. Additionally, we hope such investigations inform our interpretations of the strengths and weaknesses of current ML systems. Finally, there is the possibility that some vulnerability to carefully crafted adversarial examples may be inherent to all complex learning systems that interact with high-dimensional inputs in an environment with limited data; any thorough investigation of this question must also probe the human cognitive system. Acknowledgements This research was supported in part by NSF award AF:1813049, an ONR Young Investigator award (N00014-18-1-2295), and a grant from Stanford's Institute for Human-Centered AI. The authors would like to thank Jean Betterton, Shivam Garg, Noah Goodman, Kelvin Guu, Michelle Lee, Percy Liang, Aleksander Makelov, Jacob Plachta, Jacob Steinhardt, Jim Terry, and Alexander Whatley for useful feedback on the work. The research was done under Stanford IRB Protocol 46430.
Are humans and machine learning systems fooled by the same kinds of illusions?
27cf16bc9ef71761b9df6217f00f39f21130ce15
Are Ġhumans Ġand Ġmachine Ġlearning Ġsystems Ġfooled Ġby Ġthe Ġsame Ġkinds Ġof Ġillusions ?
Un answer able
<s> Are Ġhumans Ġand Ġmachine Ġlearning Ġsystems Ġfooled Ġby Ġthe Ġsame Ġkinds Ġof Ġillusions ? </s> Introduction </s> A Ġgrowing Ġbody Ġof Ġwork Ġon Ġadvers arial Ġexamples Ġhas Ġidentified Ġthat Ġfor Ġmachine - learning Ġ( ML ) Ġsystems Ġthat Ġoperate Ġon Ġhigh - dimensional Ġdata , Ġfor Ġnearly Ġevery Ġnatural Ġinput Ġthere Ġexists Ġa Ġsmall Ġpert urb ation Ġof Ġthe Ġpoint Ġthat Ġwill Ġbe Ġmis classified Ġby Ġthe Ġsystem , Ġposing Ġa Ġthreat Ġto Ġits Ġdeployment Ġin Ġcertain Ġcritical Ġsettings ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ. ĠMore Ġbroadly , Ġthe Ġsusceptibility Ġof ĠML Ġsystems Ġto Ġadvers arial Ġexamples Ġhas Ġprompted Ġa Ġre - examination Ġof Ġwhether Ġcurrent ĠML Ġsystems Ġare Ġtruly Ġlearning Ġor Ġif Ġthey Ġare Ġassemb l ages Ġof Ġtricks Ġthat Ġare Ġeffective Ġyet Ġbrittle Ġand Ġeasily Ġfooled ĠB IB REF 9 Ġ. ĠImpl icit Ġin Ġthis Ġline Ġof Ġreasoning Ġis Ġthe Ġassumption Ġthat Ġinstances Ġof ĠâĢ Ŀ real " Ġlearning , Ġsuch Ġas Ġhuman Ġcognition , Ġyield Ġextremely Ġrobust Ġsystems . ĠIndeed , Ġat Ġleast Ġin Ġcomputer Ġvision , Ġhuman Ġperception Ġis Ġregarded Ġas Ġthe Ġgold - standard Ġfor Ġrobust ness Ġto Ġadvers arial Ġexamples . </s> Ev idently , Ġhumans Ġcan Ġbe Ġfooled Ġby Ġa Ġvariety Ġof Ġillusions , Ġwhether Ġthey Ġbe Ġoptical , Ġauditory , Ġor Ġother ; Ġand Ġthere Ġis Ġa Ġlong Ġline Ġof Ġresearch Ġfrom Ġthe Ġcognitive Ġscience Ġand Ġpsychology Ġcommunities Ġinvestigating Ġthese ĠB IB REF 10 Ġ. ĠIn Ġgeneral , Ġhowever , Ġthese Ġillusions Ġare Ġviewed Ġas Ġisolated Ġexamples Ġthat Ġdo Ġnot Ġarise Ġfrequently , Ġand Ġwhich Ġare Ġfar Ġfrom Ġthe Ġinstances Ġencountered Ġin Ġeveryday Ġlife . </s> In Ġthis Ġwork , Ġwe Ġattempt Ġto Ġunderstand Ġhow Ġsusceptible Ġhumans ' Ġperceptual Ġsystems Ġfor Ġnatural Ġspeech Ġare Ġto Ġcarefully Ġdesigned ĠâĢ ľ ad vers arial Ġattacks . âĢ Ŀ ĠWe Ġinvestigate Ġthe Ġdensity Ġof Ġcertain Ġclasses Ġof Ġillusion , Ġthat Ġis , Ġthe Ġfraction Ġof Ġnatural Ġlanguage Ġutter ances Ġwhose Ġcomprehension Ġcan Ġbe Ġaffected Ġby Ġthe Ġillusion . ĠOur Ġstudy Ġcenters Ġaround Ġthe ĠMcG ur k Ġeffect , Ġwhich Ġis Ġthe Ġwell - stud ied Ġphenomenon Ġby Ġwhich Ġthe Ġperception Ġof Ġwhat Ġwe Ġhear Ġcan Ġbe Ġinfluenced Ġby Ġwhat Ġwe Ġsee ĠB IB REF 0 Ġ. ĠA Ġprototyp ical Ġexample Ġis Ġthat Ġthe Ġaudio Ġof Ġthe Ġphon eme ĠâĢ ľ ba a , âĢ Ŀ Ġaccompanied Ġby Ġa Ġvideo Ġof Ġsomeone Ġm ou thing ĠâĢ ľ va a âĢ Ŀ , Ġcan Ġbe Ġperceived Ġas ĠâĢ ľ va a âĢ Ŀ Ġor ĠâĢ ľ ga a âĢ Ŀ Ġ( Figure Ġ1 Ġ). ĠThis Ġeffect Ġpersists Ġeven Ġwhen Ġthe Ġsubject Ġis Ġaware Ġof Ġthe Ġsetup , Ġthough Ġthe Ġstrength Ġof Ġthe Ġeffect Ġvaries Ġsignificantly Ġacross Ġpeople Ġand Ġlanguages Ġand Ġwith Ġfactors Ġsuch Ġas Ġage , Ġgender , Ġand Ġdisorders ĠB IB REF 11 Ġ, ĠB IB REF 12 Ġ, ĠB IB REF 13 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ, ĠB IB REF 16 Ġ, ĠB IB REF 17 Ġ, ĠB IB REF 18 Ġ, ĠB IB REF 19 Ġ. </s> A Ġsignificant Ġdensity Ġof Ġillusion able Ġinstances Ġfor Ġhumans Ġmight Ġpresent Ġsimilar Ġtypes Ġof Ġsecurity Ġrisks Ġas Ġadvers arial Ġexamples Ġdo Ġfor ĠML Ġsystems . ĠAud itory Ġsignals Ġsuch Ġas Ġpublic Ġservice Ġannouncements , Ġinstructions Ġsent Ġto Ġfirst Ġresponders , Ġetc ., Ġcould Ġbe Ġtargeted Ġby Ġa Ġmalicious Ġagent . ĠGiven Ġonly Ġaccess Ġto Ġa Ġscreen Ġwithin Ġeyes ight Ġof Ġthe Ġintended Ġvictims , Ġthe Ġagent Ġmight Ġbe Ġable Ġto Ġsignificantly Ġobfusc ate Ġor Ġalter Ġthe Ġmessage Ġperceived Ġby Ġthose Ġwho Ġsee Ġthe Ġscreen Ġ( even Ġperipher ally ). </s> Related Ġwork </s> Ill usion able Ġinstances Ġfor Ġhumans Ġare Ġsimilar Ġto Ġadvers arial Ġexamples Ġfor ĠML Ġsystems . ĠSt rict ly Ġspeaking , Ġhowever , Ġour Ġinvestigation Ġof Ġthe Ġdensity Ġof Ġnatural Ġlanguage Ġfor Ġwhich ĠMcG ur k Ġillusions Ġcan Ġbe Ġcreated , Ġis Ġnot Ġthe Ġhuman Ġanalog Ġof Ġadvers arial Ġexamples . ĠThe Ġadvers arial Ġexamples Ġfor ĠML Ġsystems Ġare Ġdat ap oint s Ġthat Ġare Ġmis classified , Ġdespite Ġbeing Ġextremely Ġsimilar Ġto Ġa Ġtypical Ġdat ap oint Ġ( that Ġis Ġcorrectly Ġclassified ). ĠOur Ġillusions Ġof Ġmis d ub bed Ġaudio Ġare Ġnot Ġextremely Ġclose Ġto Ġany Ġtypically Ġencountered Ġinput , Ġsince Ġour ĠMcG ur k Ġsamples Ġhave Ġauditory Ġsignals Ġcorresponding Ġto Ġone Ġphon eme / word Ġand Ġvisual Ġsignals Ġcorresponding Ġto Ġanother . ĠAlso , Ġthere Ġis Ġa Ġcompelling Ġargument Ġfor Ġwhy Ġthe ĠMcG ur k Ġconfusion Ġoccurs , Ġnamely Ġthat Ġhuman Ġspeech Ġperception Ġis Ġb im od al Ġ( audio - visual ) Ġin Ġnature Ġwhen Ġlip Ġreading Ġis Ġavailable ĠB IB REF 20 Ġ, ĠB IB REF 21 Ġ. ĠTo Ġthe Ġbest Ġof Ġour Ġknowledge , Ġprior Ġto Ġour Ġwork , Ġthere Ġhas Ġbeen Ġlittle Ġsystematic Ġinvestigation Ġof Ġthe Ġextent Ġto Ġwhich Ġthe ĠMcG ur k Ġeffect , Ġor Ġother Ġtypes Ġof Ġillusions , Ġcan Ġbe Ġmade Ġdense Ġin Ġthe Ġset Ġof Ġinstances Ġencountered Ġin Ġeveryday Ġlife . ĠThe Ġclosest Ġwork Ġis ĠB IB REF 22 Ġ, Ġwhere Ġthe Ġauthors Ġdemonstrate Ġthat Ġsome Ġadvers arial Ġexamples Ġfor Ġcomputer Ġvision Ġsystems Ġalso Ġfool Ġhumans Ġwhen Ġhumans Ġwere Ġgiven Ġless Ġthan Ġa Ġtenth Ġof Ġsecond Ġto Ġview Ġthe Ġimage . ĠHowever , Ġsome Ġof Ġthese Ġexamples Ġseem Ġless Ġsatisfying Ġas Ġthe Ġpert urb ation Ġacts Ġas Ġa Ġpixel - space Ġinterpol ation Ġbetween Ġthe Ġoriginal Ġimage Ġand Ġthe ĠâĢ ľ inc orrect âĢ Ŀ Ġclass . ĠThis Ġresults Ġin Ġimages Ġthat Ġare Ġvisually Ġborderline Ġbetween Ġtwo Ġclasses , Ġand Ġas Ġsuch , Ġdo Ġnot Ġprovide Ġa Ġsense Ġof Ġillusion Ġto Ġthe Ġviewer . ĠIn Ġgeneral , Ġresearchers Ġhave Ġnot Ġprob ed Ġthe Ġrobust ness Ġof Ġhuman Ġperception Ġwith Ġthe Ġsame Ġtools , Ġintent , Ġor Ġperspective , Ġwith Ġwhich Ġthe Ġsecurity Ġcommunity Ġis Ġcurrently Ġinterrog ating Ġthe Ġrobust ness Ġof ĠML Ġsystems . </s> Problem Ġsetup </s> For Ġthe ĠMcG ur k Ġeffect , Ġwe Ġattempt Ġan Ġillusion Ġfor Ġa Ġlanguage Ġtoken Ġ( e . g . Ġphon eme , Ġword , Ġsentence ) Ġ$ x $ Ġby Ġcreating Ġa Ġvideo Ġwhere Ġan Ġaudio Ġstream Ġof Ġ$ x $ Ġis Ġvisually Ġdubbed Ġover Ġby Ġa Ġperson Ġsaying Ġ$ x ^ {\ prime Ġ} \ ne Ġx $ Ġ. ĠWe Ġstress Ġthat Ġthe Ġaudio Ġportion Ġof Ġthe Ġillusion Ġis Ġnot Ġmodified Ġand Ġcorresponds Ġto Ġa Ġperson Ġsaying Ġ$ x $ Ġ. ĠThe Ġillusion Ġ$ f ( x ^ {\ prime Ġ}, x ) $ Ġaffects Ġa Ġlistener Ġif Ġthey Ġperceive Ġwhat Ġis Ġbeing Ġsaid Ġto Ġbe Ġ$ y \ ne Ġx $ Ġif Ġthey Ġwatched Ġthe Ġill us ory Ġvideo Ġwhereas Ġthey Ġperceive Ġ$ x $ Ġif Ġthey Ġhad Ġeither Ġlistened Ġto Ġthe Ġaudio Ġstream Ġwithout Ġwatching Ġthe Ġvideo Ġor Ġhad Ġwatched Ġthe Ġoriginal Ġun al tered Ġvideo , Ġdepending Ġon Ġspecification . ĠWe Ġcall Ġa Ġtoken Ġillusion able Ġif Ġan Ġillusion Ġcan Ġbe Ġmade Ġfor Ġthe Ġtoken Ġthat Ġaffects Ġthe Ġperception Ġof Ġa Ġsignificant Ġfraction Ġof Ġpeople . </s> In ĠSection Ġ" Ph on eme - level Ġexperiments " Ġ, Ġwe Ġanalyze Ġthe Ġextent Ġto Ġwhich Ġthe ĠMcG ur k Ġeffect Ġcan Ġbe Ġused Ġto Ġcreate Ġillusions Ġfor Ġphon em es , Ġwords , Ġand Ġsentences , Ġand Ġanalyze Ġthe Ġfraction Ġof Ġnatural Ġlanguage Ġthat Ġis Ġsusceptible Ġto Ġsuch Ġillusion ability . ĠWe Ġthereby Ġobtain Ġa Ġlower Ġbound Ġon Ġthe Ġdensity Ġof Ġillusion able Ġnatural Ġspeech . </s> We Ġfind Ġthat Ġ1 ) Ġa Ġsignificant Ġfraction Ġof Ġwords Ġthat Ġoccur Ġin Ġeveryday Ġspeech Ġcan Ġbe Ġturned Ġinto ĠMcG ur k - style Ġillusions , Ġ2 ) Ġsuch Ġillusions Ġpersist Ġwhen Ġembedded Ġwithin Ġthe Ġcontext Ġof Ġnatural Ġsentences , Ġand Ġin Ġfact Ġaffect Ġa Ġsignificant Ġfraction Ġof Ġnatural Ġsentences , Ġand Ġ3 ) Ġthe Ġillusion ability Ġof Ġwords Ġand Ġsentences Ġcan Ġbe Ġpredicted Ġusing Ġfeatures Ġfrom Ġnatural Ġlanguage Ġmodeling . </s> Ph on eme - level Ġexperiments </s> We Ġbegan Ġby Ġdetermining Ġwhich Ġphon eme Ġsounds Ġcan Ġbe Ġpaired Ġwith Ġvideo Ġdub s Ġof Ġother Ġphon em es Ġto Ġeffect Ġa Ġperceived Ġphon eme Ġthat Ġis Ġdifferent Ġfrom Ġthe Ġactual Ġsound . ĠWe Ġcreated ĠMcG ur k Ġvideos Ġfor Ġall Ġvowel Ġpairs Ġpreceded Ġwith Ġthe Ġconson ant Ġ// Ġas Ġwell Ġas Ġfor Ġall Ġconson ant Ġpairs Ġfollowed Ġby Ġthe Ġvowel Ġ// Ġspoken Ġby Ġa Ġspeaker . ĠThere Ġare Ġ20 Ġvowel Ġphon em es Ġand Ġ24 Ġconson ant Ġphon em es Ġin ĠAmerican ĠEnglish Ġalthough Ġ/ Ê ¤ / Ġand Ġ/ Ê Ĵ / Ġare Ġredundant Ġfor Ġour Ġpurposes . ĠBased Ġon Ġlabels Ġprovided Ġby Ġ10 Ġindividuals Ġwe Ġfound Ġthat Ġalthough Ġvow els Ġwere Ġnot Ġeasily Ġconfused , Ġthere Ġare Ġa Ġnumber Ġof Ġillusion able Ġconson ants . ĠWe Ġnote Ġthat Ġthe Ġillusion able Ġphon eme Ġpairs Ġdepend Ġboth Ġon Ġthe Ġspeaker Ġand Ġlistener Ġidentities . ĠGiven ĠTable Ġ1 Ġof Ġillusion able Ġphon em es , Ġthe Ġgoal Ġwas Ġthen Ġto Ġunderstand Ġwhether Ġthese Ġcould Ġbe Ġlever aged Ġwithin Ġwords Ġor Ġsentences ; Ġand Ġif Ġso , Ġthe Ġfraction Ġof Ġnatural Ġspeech Ġthat Ġis Ġsusceptible . </s> Word - level Ġexperiments </s> We Ġsampled Ġ200 Ġunique Ġwords Ġ( listed Ġin ĠTable Ġ2 Ġ) Ġfrom Ġthe Ġ10 , 000 Ġmost Ġcommon Ġwords Ġin Ġthe ĠProject ĠGutenberg Ġnovels Ġin Ġproportion Ġto Ġtheir Ġfrequency Ġin Ġthe Ġcorpus . ĠThe Ġ10 k Ġwords Ġcollectively Ġhave Ġa Ġprevalence Ġof Ġ80 . 6 % Ġin Ġthe Ġcorpus . ĠOf Ġthe Ġ200 Ġsampled Ġwords , Ġ147 Ġ( 73 . 5 %) Ġcontained Ġphon em es Ġthat Ġour Ġpreliminary Ġphon eme Ġstudy Ġsuggested Ġmight Ġbe Ġillusion able . ĠFor Ġthese Ġ147 Ġwords , Ġwe Ġpaired Ġaudio Ġclips Ġspoken Ġby Ġthe Ġspeaker Ġwith Ġill us ory Ġvideo Ġdub s Ġof Ġthe Ġspeaker Ġsaying Ġthe Ġwords Ġwith Ġappropriately Ġswitched Ġout Ġphon em es . ĠWe Ġtested Ġthese Ġvideos Ġon Ġ20 Ġnaive Ġtest Ġsubjects Ġwho Ġdid Ġnot Ġparticipate Ġin Ġthe Ġpreliminary Ġstudy . ĠEach Ġsubject Ġwatched Ġhalf Ġof Ġthe Ġwords Ġand Ġlistened Ġwithout Ġvideo Ġto Ġthe Ġother Ġhalf Ġof Ġthe Ġwords , Ġand Ġwere Ġgiven Ġthe Ġinstructions : Ġ" Write Ġdown Ġwhat Ġyou Ġhear . ĠWhat Ġyou Ġhear Ġmay Ġor Ġmay Ġnot Ġbe Ġsens ical . ĠAlso Ġwrite Ġdown Ġif Ġa Ġclip Ġsounds Ġunclear Ġto Ġyou . ĠNot Ġthat Ġa Ġclip Ġmay Ġsound Ġnonsensical Ġbut Ġclear ." ĠSubjects Ġwere Ġallowed Ġup Ġto Ġthree Ġplays Ġof Ġeach Ġclip . </s> We Ġfound Ġthat Ġwatching Ġthe Ġill us ory Ġvideos Ġled Ġto Ġan Ġaverage Ġmis com pre hens ion Ġrate Ġof Ġ24 . 8 %, Ġa Ġrelative Ġ148 % Ġincrease Ġfrom Ġthe Ġbaseline Ġof Ġlistening Ġto Ġthe Ġaudio Ġalone Ġ( Table Ġ3 Ġ). ĠThe Ġill us ory Ġvideos Ġmade Ġpeople Ġless Ġconfident Ġabout Ġtheir Ġcorrect Ġanswers , Ġwith Ġan Ġadditional Ġ5 . 1 % Ġof Ġwords Ġbeing Ġheard Ġcorrectly Ġbut Ġunclear ly , Ġcompared Ġto Ġ2 . 1 % Ġfor Ġaudio Ġonly . ĠFor Ġ17 % Ġof Ġthe Ġ200 Ġwords , Ġthe Ġill us ory Ġvideos Ġincreased Ġthe Ġerror Ġrates Ġby Ġmore Ġthan Ġ30 % Ġabove Ġthe Ġaudio - only Ġbaseline . </s> To Ġcreate Ġa Ġpredictive Ġmodel Ġfor Ġword - level Ġillusion ability , Ġwe Ġused Ġillusion able Ġphon em es Ġenriched Ġwith Ġpositional Ġinformation Ġas Ġfeatures . ĠExplicit ly , Ġfor Ġeach Ġof Ġthe Ġ10 Ġillusion able Ġphon em es , Ġwe Ġcreated Ġthree Ġfeatures Ġfrom Ġthe Ġphon eme Ġbeing Ġin Ġan Ġinitial Ġposition Ġ( being Ġthe Ġfirst Ġphon eme Ġof Ġthe Ġword ), Ġa Ġmedial Ġposition Ġ( having Ġphon em es Ġcome Ġbefore Ġand Ġafter ), Ġor Ġa Ġfinal Ġposition Ġ( being Ġthe Ġlast Ġphon eme Ġof Ġthe Ġword ). ĠWe Ġthen Ġrepresented Ġeach Ġword Ġwith Ġa Ġbinary Ġbag - of - words Ġmodel ĠB IB REF 23 Ġ, Ġgiving Ġeach Ġof Ġthe Ġ30 Ġphon em es - with - ph on etic - context Ġfeatures Ġa Ġvalue Ġof Ġ1 Ġif Ġpresent Ġin Ġthe Ġword Ġand Ġ0 Ġotherwise . ĠWe Ġperformed Ġridge Ġregression Ġon Ġthese Ġfeatures Ġwith Ġa Ġconstant Ġterm . ĠWe Ġsearched Ġfor Ġthe Ġoptimal Ġ$ l 2 $ Ġregular ization Ġconstant Ġamong Ġthe Ġvalues Ġ[ 0 . 1 , Ġ1 , Ġ10 , Ġ100 ] Ġand Ġpicked Ġthe Ġoptimal Ġone Ġbased Ġon Ġtraining Ġset Ġperformance . ĠThe Ġtrain : test Ġsplit Ġwas Ġin Ġthe Ġproportion Ġ85 %: 15 % Ġand Ġwas Ġrandomly Ġchosen Ġfor Ġeach Ġtrial . ĠAcross Ġ10 k Ġrandomized Ġtrials , Ġwe Ġobtain Ġaverage Ġtraining Ġand Ġtest Ġset Ġcorrelations Ġof Ġ$ 91 . 1 \ pm Ġ0 . 6 \ % $ Ġand Ġ$ 44 . 6 \ pm Ġ28 . 9 \ % $ Ġrespectively . </s> Our Ġfinal Ġmodel Ġachieves Ġan Ġout - of - sample Ġcorrelation Ġof Ġ57 % Ġbetween Ġpredicted Ġand Ġobserved Ġillusion abil ites . ĠHere , Ġthe Ġobserved Ġillusion ability Ġof Ġthe Ġwords Ġis Ġcalculated Ġas Ġthe Ġdifference Ġbetween Ġthe Ġaccuracy Ġof Ġwat chers Ġof Ġthe Ġill us ory Ġvideos Ġand Ġthe Ġaccuracy Ġof Ġlisteners , Ġwhere ĠâĢ ľ acc uracy âĢ Ŀ Ġis Ġdefined Ġas Ġthe Ġfraction Ġof Ġrespondents Ġwho Ġwere Ġcorrect . ĠFor Ġeach Ġword , Ġthe Ġpredicted Ġillusion ability Ġis Ġcalculated Ġfrom Ġdoing Ġinference Ġon Ġthat Ġword Ġusing Ġthe Ġaveraged Ġregression Ġcoefficients Ġof Ġthe Ġregression Ġtrials Ġwhere Ġthe Ġword Ġis Ġnot Ġin Ġthe Ġtraining Ġset . </s> Our Ġpredicted Ġillusion ability Ġis Ġalso Ġcalibrated , Ġin Ġthe Ġsense Ġthat Ġfor Ġthe Ġwords Ġpredicted Ġto Ġhave Ġan Ġillusion ability Ġ< 0 . 1 , Ġthe Ġmean Ġempirical Ġillusion ability Ġis Ġ0 . 04 ; Ġfor Ġwords Ġwith Ġpredicted Ġillusion ability Ġin Ġthe Ġinterval Ġ[ 0 . 1 , Ġ0 . 2 ] Ġthe Ġmean Ġempirical Ġillusion ability Ġis Ġ0 . 14 ; Ġfor Ġpredicted Ġillusion ability Ġbetween Ġ[ 0 . 2 , Ġ0 . 3 ] Ġthe Ġmean Ġobserved Ġis Ġ0 . 27 ; Ġand Ġfor Ġpredicted Ġillusion ability Ġ> 0 . 3 , Ġthe Ġmean Ġobserved Ġis Ġ0 . 50 . ĠFigure Ġ2 Ġvisually Ġdepicts Ġthe Ġmatch Ġbetween Ġthe Ġobserved Ġand Ġpredicted Ġword Ġillusion abilities . </s> Sent ence - level Ġexperiments </s> We Ġset Ġup Ġthe Ġfollowing Ġexperiment Ġon Ġnaturally Ġoccurring Ġsentences . ĠWe Ġrandomly Ġsampled Ġ300 Ġsentences Ġof Ġlengths Ġ4 - 8 Ġwords Ġinclusive Ġfrom Ġthe Ġnovel ĠLittle ĠWomen ĠB IB REF 24 Ġfrom Ġthe ĠProject ĠGutenberg Ġcorpus . ĠFrom Ġthis Ġreduced Ġsample , Ġwe Ġselected Ġand Ġpert urbed Ġ32 Ġsentences Ġthat Ġwe Ġexpected Ġto Ġbe Ġillusion able Ġ( listed Ġin ĠTable Ġ4 Ġ). ĠWith Ġthe Ġspeaker , Ġwe Ġprepared Ġtwo Ġformats Ġof Ġeach Ġsentence : Ġoriginal Ġvideo Ġ( with Ġoriginal Ġaudio ), Ġand Ġill us ory Ġvideo Ġ( with Ġoriginal Ġaudio ). We Ġthen Ġevaluated Ġthe Ġperception Ġof Ġthese Ġon Ġ130 6 Ġnaive Ġtest Ġsubjects Ġon ĠAmazon ĠMechanical ĠTurk . ĠThe ĠTurk ers Ġwere Ġshown Ġvideos Ġfor Ġsix Ġrandomly Ġselected Ġsentences , Ġthree Ġill us ory Ġand Ġthree Ġoriginal , Ġand Ġwere Ġgiven Ġthe Ġprompt : Ġ" Press Ġany Ġkey Ġto Ġbegin Ġvideo Ġ[ index Ġ# ] Ġof Ġ6 . ĠWatch Ġthe Ġwhole Ġvideo , Ġand Ġthen Ġyou Ġwill Ġbe Ġprompted Ġto Ġwrite Ġdown Ġwhat Ġthe Ġspeaker Ġsaid ." ĠOnly Ġone Ġviewing Ġof Ġany Ġclip Ġwas Ġallowed , Ġto Ġsimulate Ġthe Ġnatural Ġsetting Ġof Ġobserving Ġa Ġlive Ġaudio / video Ġstream . ĠEach ĠTur ker Ġwas Ġlimited Ġto Ġsix Ġvideos Ġto Ġreduce Ġrespondent Ġfatigue . ĠTurk ers Ġwere Ġalso Ġasked Ġto Ġreport Ġtheir Ġlevel Ġof Ġconfidence Ġin Ġwhat Ġthey Ġheard Ġon Ġa Ġscale Ġfrom Ġno Ġuncertainty Ġ( 0 %) Ġto Ġcomplete Ġuncertainty Ġ( 100 %). ĠOne Ġhundred Ġand Ġtwelve ĠTurk ers Ġ( 8 . 6 %) Ġdid Ġnot Ġadhere Ġto Ġthe Ġprompt , Ġwriting Ġunrelated Ġresponses , Ġand Ġtheir Ġresults Ġwere Ġomitted Ġfrom Ġanalysis . ĠWe Ġfound Ġthat Ġwatching Ġthe Ġill us ory Ġvideos Ġled Ġto Ġan Ġaverage Ġmis com pre hens ion Ġrate Ġof Ġ32 . 8 %, Ġa Ġrelative Ġ145 % Ġincrease Ġfrom Ġthe Ġbaseline Ġof Ġlistening Ġto Ġthe Ġaudio Ġalone Ġ( Table Ġ5 Ġ). ĠThe Ġill us ory Ġvideos Ġmade Ġpeople Ġless Ġconfident Ġabout Ġtheir Ġcorrect Ġanswers . ĠTurk ers Ġwho Ġcorrectly Ġidentified Ġthe Ġaudio Ġmessage Ġin Ġan Ġill us ory Ġvideo Ġself - reported Ġan Ġaverage Ġuncertainty Ġof Ġ42 . 9 %, Ġwhich Ġis Ġa Ġrelative Ġ123 % Ġhigher Ġthan Ġthe Ġaverage Ġreported Ġby Ġthe ĠTurk ers Ġwho Ġcorrectly Ġunderstood Ġthe Ġoriginal Ġvideos . ĠExamples Ġof Ġmistakes Ġmade Ġby Ġlisteners Ġof Ġthe Ġill us ory Ġvideos Ġare Ġshown Ġin ĠTable Ġ6 Ġ. ĠOverall Ġwe Ġfound Ġthat Ġfor Ġ11 . 5 % Ġof Ġthe Ġ200 Ġsampled Ġsentences Ġ( 23 Ġout Ġof Ġthe Ġ30 Ġvideos Ġwe Ġcreated ), Ġthe Ġill us ory Ġvideos Ġincreased Ġthe Ġerror Ġrates Ġby Ġmore Ġthan Ġ10 % Ġabove Ġthe Ġaudio - only Ġbaseline . </s> We Ġobtained Ġa Ġsentence - level Ġillusion ability Ġprediction Ġmodel Ġwith Ġan Ġout - of - sample Ġcorrelation Ġof Ġ33 % Ġbetween Ġpredicted Ġand Ġobserved Ġillusion abilities . ĠHere , Ġthe Ġobserved Ġillusion ability Ġof Ġthe Ġsentences Ġwas Ġcalculated Ġas Ġthe Ġdifference Ġbetween Ġthe Ġaccuracy Ġof Ġwat chers Ġof Ġthe Ġill us ory Ġvideos Ġand Ġthe Ġaccuracy Ġof Ġwat chers Ġof Ġoriginal Ġvideos , Ġwhere ĠâĢ ľ acc uracy âĢ Ŀ Ġis Ġdefined Ġas Ġthe Ġfraction Ġof Ġrespondents Ġwho Ġwere Ġcorrect . ĠWe Ġobtained Ġpredicted Ġillusion abilities Ġby Ġsimply Ġusing Ġthe Ġmaximum Ġword Ġillusion ability Ġprediction Ġamongst Ġthe Ġwords Ġin Ġeach Ġsentence , Ġwith Ġword Ġpredictions Ġobtained Ġfrom Ġthe Ġword - level Ġmodel . ĠWe Ġattempted Ġto Ġimprove Ġour Ġsentence - level Ġpredictive Ġmodel Ġby Ġincorporating Ġhow Ġlikely Ġthe Ġwords Ġappear Ġunder Ġa Ġnatural Ġlanguage Ġdistribution , Ġconsidering Ġthree Ġclasses Ġof Ġwords : Ġwords Ġin Ġthe Ġsentence Ġfor Ġwhich Ġno Ġillusion Ġwas Ġattempted , Ġthe Ġwords Ġfor Ġwhich Ġan Ġillusion Ġwas Ġattempted , Ġand Ġthe Ġpotentially Ġperceived Ġwords Ġfor Ġwords Ġfor Ġwhich Ġan Ġillusion Ġwas Ġattempted . ĠWe Ġused Ġlog Ġword Ġfrequencies Ġobtained Ġfrom Ġthe Ġthe Ġtop Ġ36 . 7 k Ġmost Ġcommon Ġwords Ġfrom Ġthe ĠProject ĠGutenberg Ġcorpus . ĠThis Ġapproach Ġcould Ġnot Ġattain Ġbetter Ġout - of - sample Ġcorrelations Ġthan Ġthe Ġnaive Ġmethod . ĠThis Ġimplies Ġthat Ġcontext Ġis Ġimportant Ġfor Ġsentence - level Ġillusion ability , Ġand Ġmore Ġcomplex Ġlanguage Ġmodels Ġshould Ġbe Ġused . </s> Finally , Ġcomparing Ġword - level Ġand Ġsentence - level ĠMcG ur k Ġillusion abilities Ġin Ġnatural Ġspeech , Ġwe Ġobserve Ġthat Ġthat Ġthe Ġformer Ġis Ġsignificantly Ġhigher . ĠA Ġgreater ĠMcG ur k Ġeffect Ġat Ġthe Ġword Ġlevel Ġis Ġto Ġbe Ġexpected âĢĵ sent ences Ġprovide Ġcontext Ġwith Ġwhich Ġthe Ġviewer Ġcould Ġfill Ġin Ġconf usions Ġand Ġmisunderstand ings . ĠFurthermore , Ġwhen Ġwatching Ġa Ġsentence Ġvideo Ġcompared Ġto Ġa Ġshort Ġword Ġvideo , Ġthe Ġviewer 's Ġattention Ġis Ġmore Ġlikely Ġto Ġstray , Ġboth Ġfrom Ġthe Ġvisual Ġcomponent Ġof Ġthe Ġvideo , Ġwhich Ġevidently Ġreduces Ġthe ĠMcG ur k Ġeffect , Ġas Ġwell Ġas Ġthe Ġfrom Ġthe Ġaudio Ġcomponent , Ġwhich Ġlikely Ġprompts Ġthe Ġviewer Ġto Ġrely Ġeven Ġmore Ġheavily Ġon Ġcontext . ĠNevertheless , Ġthere Ġremains Ġa Ġsignificant Ġamount Ġof Ġillusion ability Ġat Ġthe Ġsentence - level . </s> Future ĠDirections </s> This Ġwork Ġis Ġan Ġinitial Ġstep Ġtowards Ġexploring Ġthe Ġdensity Ġof Ġillusion able Ġphenomena Ġfor Ġhumans . ĠThere Ġare Ġmany Ġnatural Ġdirections Ġfor Ġfuture Ġwork . ĠIn Ġthe Ġvein Ġof Ġfurther Ġunderstanding ĠMcG ur k - style Ġillusions , Ġit Ġseems Ġworth Ġbuilding Ġmore Ġaccurate Ġpredictive Ġmodels Ġfor Ġsentence - level Ġeffects , Ġand Ġfurther Ġinvestigating Ġthe Ġsecurity Ġrisks Ġposed Ġby ĠMcG ur k Ġillusions . ĠFor Ġexample , Ġone Ġconcrete Ġnext Ġstep Ġin Ġunderstanding ĠMcG ur k - style Ġillusions Ġwould Ġbe Ġto Ġactually Ġimplement Ġa Ġsystem Ġwhich Ġtakes Ġan Ġaudio Ġinput , Ġand Ġoutputs Ġa Ġvideo Ġdub Ġresulting Ġin Ġsignificant Ġmisunderstanding . ĠSuch Ġa Ġsystem Ġwould Ġneed Ġto Ġcombine Ġa Ġhigh - quality Ġspeech - to - video - sy nt hesis Ġsystem ĠB IB REF 25 Ġ, ĠB IB REF 26 Ġ, Ġwith Ġa Ġfl es hed - out Ġlanguage Ġmodel Ġand ĠMcG ur k Ġprediction Ġmodel . ĠThere Ġis Ġalso Ġthe Ġquestion Ġof Ġhow Ġto Ġguard Ġagainst ĠâĢ ľ attacks âĢ Ŀ Ġon Ġhuman Ġperception . ĠFor Ġexample , Ġin Ġthe Ġcase Ġof Ġthe ĠMcG ur k Ġeffect , Ġhow Ġcan Ġone Ġre phrase Ġa Ġpassage Ġof Ġtext Ġin Ġsuch Ġa Ġway Ġthat Ġthe Ġmeaning Ġis Ġunchanged , Ġbut Ġthe Ġre ph r ased Ġtext Ġis Ġsignificantly Ġmore Ġrobust Ġto ĠMcG ur k Ġstyle Ġmanip ulations ? ĠThe Ġcentral Ġquestion Ġin Ġthis Ġdirection Ġis Ġwhat Ġfraction Ġof Ġnatural Ġlanguage Ġcan Ġbe Ġmade Ġrobust Ġwithout Ġsignificantly Ġchanging Ġthe Ġsemantics . </s> A Ġbetter Ġunderstanding Ġof Ġwhen Ġand Ġwhy Ġcertain Ġhuman Ġperception Ġsystems Ġare Ġnon rob ust Ġcan Ġalso Ġbe Ġapplied Ġto Ġmake ĠML Ġsystems Ġmore Ġrobust . ĠIn Ġparticular , Ġneural Ġnetworks Ġhave Ġbeen Ġfound Ġto Ġbe Ġsusceptible Ġto Ġadvers arial Ġexamples Ġin Ġautomatic Ġspeech Ġrecognition ĠB IB REF 27 Ġ, ĠB IB REF 28 Ġand Ġto Ġthe ĠMcG ur k Ġeffect ĠB IB REF 29 Ġ, Ġand Ġa Ġrudimentary Ġapproach Ġto Ġmaking Ġlanguage Ġrobust Ġto Ġthe Ġlatter Ġproblem Ġwould Ġbe Ġto Ġuse Ġa Ġreduced Ġvocabulary Ġthat Ġavoids Ġwords Ġthat Ġscore Ġhighly Ġin Ġour Ġword - level Ġillusion ability Ġprediction Ġmodel . ĠRelated ly , Ġat Ġthe Ġinterface Ġof Ġcognitive Ġscience Ġand Ġadvers arial Ġexamples , Ġthere Ġhas Ġbeen Ġwork Ġsuggesting Ġthat Ġhumans Ġcan Ġanticipate Ġwhen Ġor Ġhow Ġmachines Ġwill Ġmis class ify , Ġincluding Ġfor Ġadvers arial Ġexamples ĠB IB REF 30 Ġ, ĠB IB REF 31 Ġ, ĠB IB REF 32 Ġ. </s> More Ġbroadly , Ġas Ġthe Ġtools Ġfor Ġprobing Ġthe Ġweaknesses Ġof ĠML Ġsystems Ġdevelop Ġfurther , Ġit Ġseems Ġlike Ġa Ġnatural Ġtime Ġto Ġre ex amine Ġthe Ġsupposed Ġrobust ness Ġof Ġhuman Ġperception . ĠWe Ġanticipate Ġunexpected Ġfindings . ĠTo Ġprovide Ġone Ġexample , Ġwe Ġsummarize Ġsome Ġpreliminary Ġresults Ġon Ġaudio - only Ġillusions . </s> Aud itory ĠIll usions </s> An Ġaudio Ġclip Ġof Ġthe Ġword ĠâĢ ľ La ure l " Ġgained Ġwidespread Ġattention Ġin Ġ2018 , Ġwith Ġcoverage Ġby Ġnotable Ġnews Ġoutlets Ġsuch Ġas ĠThe ĠNew ĠYork ĠTimes Ġand ĠTime . ĠRough ly Ġhalf Ġof Ġlisteners Ġperceive ĠâĢ ľ La ure l âĢ Ŀ Ġand Ġthe Ġother Ġhalf Ġperceive ĠâĢ ľ Y anny âĢ Ŀ Ġor Ġsimilar - sounding Ġwords , Ġwith Ġhigh Ġconfidence Ġon Ġboth Ġsides ĠB IB REF 1 Ġ. ĠOne Ġof Ġthe Ġreasons Ġthe Ġpublic Ġwas Ġintrigued Ġis Ġbecause Ġexamples Ġof Ġsuch Ġphenomena Ġare Ġviewed Ġas Ġrare , Ġisolated Ġinstances . ĠIn Ġa Ġpreliminary Ġattempt Ġto Ġinvestigate Ġthe Ġdensity Ġof Ġsuch Ġphenomena , Ġwe Ġidentified Ġfive Ġadditional Ġdistinct Ġexamples Ġ( Table Ġ7 Ġ). ĠThe Ġsupplementary Ġfiles Ġinclude Ġ10 Ġversions Ġof Ġone Ġof Ġthese Ġexamples , Ġwhere Ġlisteners Ġtend Ġto Ġperceive Ġeither ĠâĢ ľ world s âĢ Ŀ Ġor ĠâĢ ľ y ikes . âĢ Ŀ ĠAcross Ġthe Ġdifferent Ġaudio Ġclips , Ġone Ġshould Ġbe Ġable Ġto Ġhear Ġboth Ġinterpretations . ĠThe Ġthreshold Ġfor Ġswitching Ġfrom Ġone Ġinterpretation Ġto Ġanother Ġdiffers Ġfrom Ġperson Ġto Ġperson . </s> These Ġexamples Ġwere Ġgenerated Ġby Ġexamining Ġ5000 Ġwords , Ġand Ġselecting Ġthe Ġ50 Ġwhose Ġspect rog rams Ġcontain Ġa Ġbalance Ġof Ġhigh Ġand Ġlow Ġfrequency Ġcomponents Ġthat Ġmost Ġclosely Ġmatched Ġthose Ġfor Ġthe Ġword ĠâĢ ľ La ure l ". ĠEach Ġaudio Ġfile Ġcorrespond ed Ġto Ġthe ĠGoogle ĠCloud ĠText - to - Spe ech ĠAPI Ġsynthesis Ġof Ġa Ġword , Ġafter Ġlow Ġfrequencies Ġwere Ġd amped Ġand Ġthe Ġaudio Ġwas Ġslowed Ġ$ 1 . 3 $ Ġ- Ġ$ 1 . 9 $ Ġx . ĠAfter Ġlistening Ġto Ġthese Ġtop Ġ50 Ġcandidates , Ġwe Ġevaluated Ġthe Ġmost Ġpromising Ġfive Ġon Ġa Ġset Ġof Ġ15 Ġindividuals Ġ( 3 Ġfemale , Ġ12 Ġmale , Ġage Ġrange Ġ22 - 33 ). ĠWe Ġfound Ġmultiple Ġdistribution al Ġmodes Ġof Ġperceptions Ġfor Ġall Ġfive Ġaudio Ġclips . ĠFor Ġexample , Ġa Ġclip Ġof ĠâĢ ľ world s âĢ Ŀ Ġwith Ġthe Ġhigh Ġfrequencies Ġd amped Ġand Ġslowed Ġdown Ġ1 . 5 x Ġwas Ġperceived Ġby Ġfive Ġlisteners Ġas ĠâĢ ľ world s âĢ Ŀ , Ġfour Ġas ĠâĢ ľ y ikes / y ites âĢ Ŀ Ġand Ġsix Ġas ĠâĢ ľ n ights / lights âĢ Ŀ . ĠWhile Ġthese Ġexperiments Ġdo Ġnot Ġdemonstrate Ġa Ġdensity Ġof Ġsuch Ġexamples Ġwith Ġrespect Ġto Ġthe Ġset Ġof Ġall Ġwords âĢĶ and Ġit Ġis Ġunlikely Ġthat Ġill us ory Ġaudio Ġtracks Ġin Ġthis Ġstyle Ġcan Ġbe Ġcreated Ġfor Ġthe Ġmajority Ġof Ġwords âĢĶ they Ġillustrate Ġthat Ġeven Ġthe Ġsurprising ĠY anny Ġor ĠLaurel Ġphenomenon Ġis Ġnot Ġan Ġisolated Ġoccurrence . ĠIt Ġremains Ġto Ġbe Ġseen Ġhow Ġdense Ġsuch Ġphenomena Ġcan Ġbe , Ġgiven Ġthe Ġright Ġsort Ġof Ġsubtle Ġaudio Ġmanipulation . Ġ` </s> Conclusion </s> Our Ġwork Ġsuggests Ġthat Ġfor Ġa Ġsignificant Ġfraction Ġof Ġnatural Ġspeech , Ġhuman Ġperception Ġcan Ġbe Ġaltered Ġby Ġusing Ġsubtle , Ġlearn able Ġpert urb ations . ĠThis Ġis Ġan Ġinitial Ġstep Ġtowards Ġexploring Ġthe Ġdensity Ġof Ġillusion able Ġphenomenon Ġfor Ġhumans , Ġand Ġexamining Ġthe Ġextent Ġto Ġwhich Ġhuman Ġperception Ġmay Ġbe Ġvulnerable Ġto Ġsecurity Ġrisks Ġlike Ġthose Ġthat Ġadvers arial Ġexamples Ġpresent Ġfor ĠML Ġsystems . </s> We Ġhope Ġour Ġwork Ġinspires Ġfuture Ġinvestigations Ġinto Ġthe Ġdiscovery , Ġgeneration , Ġand Ġquant ification Ġof Ġmultim od al Ġand Ġunim od al Ġaud iov is ual Ġand Ġauditory Ġillusions Ġfor Ġhumans . ĠThere Ġexist Ġmany Ġopen Ġresearch Ġquestions Ġon Ġwhen Ġand Ġwhy Ġhumans Ġare Ġsusceptible Ġto Ġvarious Ġtypes Ġof Ġillusions , Ġhow Ġto Ġmodel Ġthe Ġillusion ability Ġof Ġnatural Ġlanguage , Ġand Ġhow Ġnatural Ġlanguage Ġcan Ġbe Ġmade Ġmore Ġrobust Ġto Ġill us ory Ġpert urb ations . ĠAdditionally , Ġwe Ġhope Ġsuch Ġinvestigations Ġinform Ġour Ġinterpretations Ġof Ġthe Ġstrengths Ġand Ġweaknesses Ġof Ġcurrent ĠML Ġsystems . ĠFinally , Ġthere Ġis Ġthe Ġpossibility Ġthat Ġsome Ġvulnerability Ġto Ġcarefully Ġcrafted Ġadvers arial Ġexamples Ġmay Ġbe Ġinherent Ġto Ġall Ġcomplex Ġlearning Ġsystems Ġthat Ġinteract Ġwith Ġhigh - dimensional Ġinputs Ġin Ġan Ġenvironment Ġwith Ġlimited Ġdata ; Ġany Ġthorough Ġinvestigation Ġof Ġthis Ġquestion Ġmust Ġalso Ġprobe Ġthe Ġhuman Ġcognitive Ġsystem . </s> Acknowled gements </s> This Ġresearch Ġwas Ġsupported Ġin Ġpart Ġby ĠNS F Ġaward ĠAF : 18 130 49 , Ġan ĠON R ĠYoung ĠInvestigator Ġaward Ġ( N 0001 4 - 18 - 1 - 22 95 ), Ġand Ġa Ġgrant Ġfrom ĠStanford 's ĠInstitute Ġfor ĠHuman - Cent ered ĠAI . ĠThe Ġauthors Ġwould Ġlike Ġto Ġthank ĠJean ĠBetter ton , ĠShiv am ĠGarg , ĠNoah ĠGoodman , ĠKelvin ĠGu u , ĠMichelle ĠLee , ĠPercy ĠLiang , ĠAle ks ander ĠMake lov , ĠJacob ĠPl ach ta , ĠJacob ĠStein hardt , ĠJim ĠTerry , Ġand ĠAlexander ĠWhat ley Ġfor Ġuseful Ġfeedback Ġon Ġthe Ġwork . ĠThe Ġresearch Ġwas Ġdone Ġunder ĠStanford ĠIR B ĠProtocol Ġ4 64 30 .