Unanswerable
NONE
Unanswerable
NONE
1605.06083
Introduction </s> The ĠFlickr 30 K Ġdataset ĠB IB REF 0 Ġis Ġa Ġcollection Ġof Ġover Ġ30 , 000 Ġimages Ġwith Ġ5 Ġcrowds ourced Ġdescriptions Ġeach . ĠIt Ġis Ġcommonly Ġused Ġto Ġtrain Ġand Ġevaluate Ġneural Ġnetwork Ġmodels Ġthat Ġgenerate Ġimage Ġdescriptions Ġ( e . g . ĠB IB REF 2 Ġ). ĠAn Ġunt ested Ġassumption Ġbehind Ġthe Ġdataset Ġis Ġthat Ġthe Ġdescriptions Ġare Ġbased Ġon Ġthe Ġimages , Ġand Ġnothing Ġelse . ĠHere Ġare Ġthe Ġauthors Ġ( about Ġthe ĠFlickr 8 K Ġdataset , Ġa Ġsubset Ġof ĠFlickr 30 K ): </s> âĢ ľ By Ġasking Ġpeople Ġto Ġdescribe Ġthe Ġpeople , Ġobjects , Ġscenes Ġand Ġactivities Ġthat Ġare Ġshown Ġin Ġa Ġpicture Ġwithout Ġgiving Ġthem Ġany Ġfurther Ġinformation Ġabout Ġthe Ġcontext Ġin Ġwhich Ġthe Ġpicture Ġwas Ġtaken , Ġwe Ġwere Ġable Ġto Ġobtain Ġconceptual Ġdescriptions Ġthat Ġfocus Ġonly Ġon Ġthe Ġinformation Ġthat Ġcan Ġbe Ġobtained Ġfrom Ġthe Ġimage Ġalone . âĢ Ŀ ĠB IB REF 1 </s> What Ġthis Ġassumption Ġoverlook s Ġis Ġthe Ġamount Ġof Ġinterpretation Ġor Ġrecon text ual ization Ġcarried Ġout Ġby Ġthe Ġannot ators . ĠLet Ġus Ġtake Ġa Ġconcrete Ġexample . ĠFigure ĠFIG REF 1 Ġshows Ġan Ġimage Ġfrom Ġthe ĠFlickr 30 K Ġdataset . </s> This Ġimage Ġcomes Ġwith Ġthe Ġfive Ġdescriptions Ġbelow . ĠAll Ġbut Ġthe Ġfirst Ġone Ġcontain Ġinformation Ġthat Ġcannot Ġcome Ġfrom Ġthe Ġimage Ġalone . ĠRe levant Ġparts Ġare Ġhighlighted Ġin Ġbold : </s> We Ġneed Ġto Ġunderstand Ġthat Ġthe Ġdescriptions Ġin Ġthe ĠFlickr 30 K Ġdataset Ġare Ġsubjective Ġdescriptions Ġof Ġevents . ĠThis Ġcan Ġbe Ġa Ġgood Ġthing : Ġthe Ġdescriptions Ġtell Ġus Ġwhat Ġare Ġthe Ġsalient Ġparts Ġof Ġeach Ġimage Ġto Ġthe Ġaverage Ġhuman Ġannot ator . ĠSo Ġthe Ġtwo Ġhumans Ġin ĠFigure ĠFIG REF 1 Ġare Ġrelevant , Ġbut Ġthe Ġtwo Ġsoap Ġdispens ers Ġare Ġnot . ĠBut Ġsubject ivity Ġcan Ġalso Ġresult Ġin Ġstereotypical Ġdescriptions , Ġin Ġthis Ġcase Ġsuggesting Ġthat Ġthe Ġmale Ġis Ġmore Ġlikely Ġto Ġbe Ġthe Ġmanager , Ġand Ġthe Ġfemale Ġis Ġmore Ġlikely Ġto Ġbe Ġthe Ġsubordinate . Ġrash tch ian 2010 collect ing Ġdo Ġnote Ġthat Ġsome Ġdescriptions Ġare Ġspeculative Ġin Ġnature , Ġwhich Ġthey Ġsay Ġhurts Ġthe Ġaccuracy Ġand Ġthe Ġconsistency Ġof Ġthe Ġdescriptions . ĠBut Ġthe Ġproblem Ġis Ġnot Ġwith Ġthe Ġlack Ġof Ġconsistency Ġhere . ĠQuite Ġthe Ġcontrary : Ġthe Ġproblem Ġis Ġthat Ġstereotypes Ġmay Ġbe Ġpervasive Ġenough Ġfor Ġthe Ġdata Ġto Ġbe Ġconsistently Ġbiased . ĠAnd Ġso Ġlanguage Ġmodels Ġtrained Ġon Ġthis Ġdata Ġmay Ġpropagate Ġharmful Ġstereotypes , Ġsuch Ġas Ġthe Ġidea Ġthat Ġwomen Ġare Ġless Ġsuited Ġfor Ġleadership Ġpositions . </s> This Ġpaper Ġaims Ġto Ġgive Ġan Ġoverview Ġof Ġlinguistic Ġbias Ġand Ġunw arranted Ġinf erences Ġresulting Ġfrom Ġstereotypes Ġand Ġprejudices . ĠI Ġwill Ġbuild Ġon Ġearlier Ġwork Ġon Ġlinguistic Ġbias Ġin Ġgeneral ĠB IB REF 3 Ġ, Ġproviding Ġexamples Ġfrom Ġthe ĠFlickr 30 K Ġdata , Ġand Ġpresent Ġa Ġtax onomy Ġof Ġunw arranted Ġinf erences . ĠFinally , ĠI Ġwill Ġdiscuss Ġseveral Ġmethods Ġto Ġanalyze Ġthe Ġdata Ġin Ġorder Ġto Ġdetect Ġbiases . </s> St ere otype - driven Ġdescriptions </s> St ere otypes Ġare Ġideas Ġabout Ġhow Ġother Ġ( groups Ġof ) Ġpeople Ġcommonly Ġbehave Ġand Ġwhat Ġthey Ġare Ġlikely Ġto Ġdo . ĠThese Ġideas Ġguide Ġthe Ġway Ġwe Ġtalk Ġabout Ġthe Ġworld . ĠI Ġdistinguish Ġtwo Ġkinds Ġof Ġverbal Ġbehavior Ġthat Ġresult Ġfrom Ġstereotypes : Ġ( i ) Ġlinguistic Ġbias , Ġand Ġ( ii ) Ġunw arranted Ġinf erences . ĠThe Ġformer Ġis Ġdiscussed Ġin Ġmore Ġdetail Ġby Ġbe uke bo om 2014 me chan isms , Ġwho Ġdefines Ġlinguistic Ġbias Ġas ĠâĢ ľ a Ġsystematic Ġasymm etry Ġin Ġword Ġchoice Ġas Ġa Ġfunction Ġof Ġthe Ġsocial Ġcategory Ġto Ġwhich Ġthe Ġtarget Ġbelongs . âĢ Ŀ ĠSo Ġthis Ġbias Ġbecomes Ġvisible Ġthrough Ġthe Ġdistribution Ġof Ġterms Ġused Ġto Ġdescribe Ġentities Ġin Ġa Ġparticular Ġcategory . ĠUn w arranted Ġinf erences Ġare Ġthe Ġresult Ġof Ġspeculation Ġabout Ġthe Ġimage ; Ġhere , Ġthe Ġannot ator Ġgoes Ġbeyond Ġwhat Ġcan Ġbe Ġglanced Ġfrom Ġthe Ġimage Ġand Ġmakes Ġuse Ġof Ġtheir Ġknowledge Ġand Ġexpectations Ġabout Ġthe Ġworld Ġto Ġprovide Ġan Ġoverly Ġspecific Ġdescription . ĠSuch Ġdescriptions Ġare Ġdirectly Ġidentifiable Ġas Ġsuch , Ġand Ġin Ġfact Ġwe Ġhave Ġalready Ġseen Ġfour Ġof Ġthem Ġ( desc ript ions Ġ2 âĢĵ 5 ) Ġdiscussed Ġearlier . </s> L ingu istic Ġbias </s> Generally Ġspeaking , Ġpeople Ġtend Ġto Ġuse Ġmore Ġconcrete Ġor Ġspecific Ġlanguage Ġwhen Ġthey Ġhave Ġto Ġdescribe Ġa Ġperson Ġthat Ġdoes Ġnot Ġmeet Ġtheir Ġexpectations . Ġbe uke bo om 2014 me chan isms Ġlists Ġseveral Ġlinguistic Ġ` tools ' Ġthat Ġpeople Ġuse Ġto Ġmark Ġindividuals Ġwho Ġdev iate Ġfrom Ġthe Ġnorm . ĠI Ġwill Ġmention Ġtwo Ġof Ġthem . </s> [ left margin = 0 cm ] </s> One Ġwell - stud ied Ġexample ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġis Ġsexist Ġlanguage , Ġwhere Ġthe Ġsex Ġof Ġa Ġperson Ġtends Ġto Ġbe Ġmentioned Ġmore Ġfrequently Ġif Ġtheir Ġrole Ġor Ġoccupation Ġis Ġinconsistent Ġwith Ġ` traditional ' Ġgender Ġroles Ġ( e . g . Ġfemale Ġsurgeon , Ġmale Ġnurse ). ĠBe uke bo om Ġalso Ġnotes Ġthat Ġadject ives Ġare Ġused Ġto Ġcreate ĠâĢ ľ more Ġnarrow Ġlabels Ġ[ or Ġsub types ] Ġfor Ġindividuals Ġwho Ġdo Ġnot Ġfit Ġwith Ġgeneral Ġsocial Ġcategory Ġexpectations âĢ Ŀ Ġ( p . Ġ3 ). ĠE . g . Ġtough Ġwoman Ġmakes Ġan Ġexception Ġto Ġthe Ġ` rule ' Ġthat Ġwomen Ġaren 't Ġconsidered Ġto Ġbe Ġtough . </s> can Ġbe Ġused Ġwhen Ġprior Ġbeliefs Ġabout Ġa Ġparticular Ġsocial Ġcategory Ġare Ġviolated , Ġe . g . ĠThe Ġgarbage Ġman Ġwas Ġnot Ġstupid . ĠSee Ġalso ĠB IB REF 6 Ġ. </s> These Ġexamples Ġare Ġsimilar Ġin Ġthat Ġthe Ġspeaker Ġhas Ġto Ġput Ġin Ġadditional Ġeffort Ġto Ġmark Ġthe Ġsubject Ġfor Ġbeing Ġunusual . ĠBut Ġthey Ġdiffer Ġin Ġwhat Ġwe Ġcan Ġconclude Ġabout Ġthe Ġspeaker , Ġespecially Ġin Ġthe Ġcontext Ġof Ġthe ĠFlickr 30 K Ġdata . ĠNeg ations Ġare Ġmuch Ġmore Ġovertly Ġdisplaying Ġthe Ġannot ator 's Ġprior Ġbeliefs . ĠWhen Ġone Ġannot ator Ġwrites Ġthat ĠA Ġlittle Ġboy Ġis Ġeating Ġpie Ġwithout Ġut ens ils Ġ( image Ġ26 59 04 67 89 ), Ġthis Ġimmediately Ġreveals Ġthe Ġannot ator 's Ġnormative Ġbeliefs Ġabout Ġthe Ġworld : Ġpie Ġshould Ġbe Ġeaten Ġwith Ġut ens ils . ĠBut Ġwhen Ġanother Ġannot ator Ġtalks Ġabout Ġa Ġgirls Ġbasketball Ġgame Ġ( image Ġ8 245 36 60 95 ), Ġthis Ġcannot Ġbe Ġtaken Ġas Ġan Ġindication Ġthat Ġthe Ġannot ator Ġis Ġbiased Ġabout Ġthe Ġgender Ġof Ġbasketball Ġplayers ; Ġthey Ġmight Ġjust Ġbe Ġhelpful Ġby Ġproviding Ġa Ġdetailed Ġdescription . ĠIn Ġsection Ġ3 ĠI Ġwill Ġdiscuss Ġhow Ġto Ġestablish Ġwhether Ġor Ġnot Ġthere Ġis Ġany Ġbias Ġin Ġthe Ġdata Ġregarding Ġthe Ġuse Ġof Ġadject ives . </s> Un w arranted Ġinf erences </s> Un w arranted Ġinf erences Ġare Ġstatements Ġabout Ġthe Ġsubject ( s ) Ġof Ġan Ġimage Ġthat Ġgo Ġbeyond Ġwhat Ġthe Ġvisual Ġdata Ġalone Ġcan Ġtell Ġus . ĠThey Ġare Ġbased Ġon Ġadditional Ġassumptions Ġabout Ġthe Ġworld . ĠAfter Ġinspecting Ġa Ġsubset Ġof Ġthe ĠFlickr 30 K Ġdata , ĠI Ġhave Ġgrouped Ġthese Ġinf erences Ġinto Ġsix Ġcategories Ġ( image Ġexamples Ġbetween Ġparentheses ): </s> [ left margin = 0 cm ] </s> We 've Ġseen Ġan Ġexample Ġof Ġthis Ġin Ġthe Ġintroduction , Ġwhere Ġthe Ġ` manager ' Ġwas Ġsaid Ġto Ġbe Ġtalking Ġabout Ġjob Ġperformance Ġand Ġsc olding Ġ[ a Ġworker ] Ġin Ġa Ġstern Ġlecture Ġ( 80 63 007 ). </s> Many Ġdark - skinned Ġindividuals Ġare Ġcalled ĠAfrican - American Ġregardless Ġof Ġwhether Ġthe Ġpicture Ġhas Ġbeen Ġtaken Ġin Ġthe ĠUSA Ġor Ġnot Ġ( 42 80 272 ). ĠAnd Ġpeople Ġwho Ġlook ĠAsian Ġare Ġcalled ĠChinese Ġ( 14 34 15 17 32 ) Ġor ĠJapanese Ġ( 48 346 64 666 ). </s> In Ġimage Ġ4 183 120 Ġ( Figure ĠFIG REF 16 Ġ), Ġpeople Ġsitting Ġat Ġa Ġgym Ġare Ġsaid Ġto Ġbe Ġwatching Ġa Ġgame , Ġeven Ġthough Ġthere Ġcould Ġbe Ġany Ġsort Ġof Ġevent Ġgoing Ġon . ĠBut Ġsince Ġthe Ġlocation Ġis Ġso Ġstrongly Ġassociated Ġwith Ġsports , Ġcrowd workers Ġreadily Ġmake Ġthe Ġassumption . </s> Quite Ġa Ġfew Ġannotations Ġfocus Ġon Ġexplaining Ġthe Ġwhy Ġof Ġthe Ġsituation . ĠFor Ġexample , Ġin Ġimage Ġ396 30 38 375 Ġa Ġman Ġis Ġfast ening Ġhis Ġclimbing Ġharness Ġin Ġorder Ġto Ġhave Ġsome Ġfun . ĠAnd Ġin Ġan Ġextreme Ġcase , Ġone Ġannot ator Ġwrites Ġabout Ġa Ġpicture Ġof Ġa Ġdancing Ġwoman Ġthat Ġthe Ġschool Ġis Ġhaving Ġa Ġspecial Ġevent Ġin Ġorder Ġto Ġshow Ġthe Ġameric an Ġculture Ġon Ġhow Ġother Ġcultures Ġare Ġdealt Ġwith Ġin Ġparties Ġ( 36 363 29 461 ). ĠThis Ġis Ġreminiscent Ġof Ġthe ĠSt ere otyp ic ĠExplan atory ĠB ias ĠB IB REF 7 Ġ, Ġwhich Ġrefers Ġto ĠâĢ ľ the Ġtendency Ġto Ġprovide Ġrelatively Ġmore Ġexplanations Ġin Ġdescriptions Ġof Ġstereotype Ġinconsistent , Ġcompared Ġto Ġconsistent Ġbehavior âĢ Ŀ ĠB IB REF 6 Ġ. ĠSo Ġin Ġtheory , Ġodd Ġor Ġsurprising Ġsituations Ġshould Ġreceive Ġmore Ġexplanations , Ġsince Ġa Ġdescription Ġalone Ġmay Ġnot Ġmake Ġenough Ġsense Ġin Ġthose Ġcases , Ġbut Ġit Ġis Ġbeyond Ġthe Ġscope Ġof Ġthis Ġpaper Ġto Ġtest Ġwhether Ġor Ġnot Ġthe ĠFlickr 30 K Ġdata Ġsuffers Ġfrom Ġthe ĠSE B . </s> O lder Ġpeople Ġwith Ġchildren Ġaround Ġthem Ġare Ġcommonly Ġseen Ġas Ġparents Ġ( 5 287 405 ), Ġsmall Ġchildren Ġas Ġsiblings Ġ( 20 58 42 ), Ġmen Ġand Ġwomen Ġas Ġlovers Ġ( 44 296 60 ), Ġgroups Ġof Ġyoung Ġpeople Ġas Ġfriends Ġ( 369 79 ). </s> An not ators Ġwill Ġoften Ġguess Ġthe Ġstatus Ġor Ġoccupation Ġof Ġpeople Ġin Ġan Ġimage . ĠSometimes Ġthese Ġguesses Ġare Ġrelatively Ġgeneral Ġ( e . g . Ġcollege - aged Ġpeople Ġbeing Ġcalled Ġstudents Ġin Ġimage Ġ369 79 ), Ġbut Ġother Ġtimes Ġthese Ġare Ġvery Ġspecific Ġ( e . g . Ġa Ġman Ġin Ġa Ġworkshop Ġbeing Ġcalled Ġa Ġgraphics Ġdesigner , Ġ58 67 606 ). </s> Detect ing Ġstereotype - driven Ġdescriptions </s> In Ġorder Ġto Ġget Ġan Ġidea Ġof Ġthe Ġkinds Ġof Ġstereotype - driven Ġdescriptions Ġthat Ġare Ġin Ġthe ĠFlickr 30 K Ġdataset , ĠI Ġmade Ġa Ġbrowser - based Ġannotation Ġtool Ġthat Ġshows Ġboth Ġthe Ġimages Ġand Ġtheir Ġassociated Ġdescriptions . ĠYou Ġcan Ġsimply Ġleaf Ġthrough Ġthe Ġimages Ġby Ġclicking Ġ` Next ' Ġor Ġ` Random ' Ġuntil Ġyou Ġfind Ġan Ġinteresting Ġpattern . </s> Eth nic ity / race </s> One Ġinteresting Ġpattern Ġis Ġthat Ġthe Ġethnicity / race Ġof Ġbabies Ġdoesn 't Ġseem Ġto Ġbe Ġmentioned Ġunless Ġthe Ġbaby Ġis Ġblack Ġor Ġas ian . ĠIn Ġother Ġwords : Ġwhite Ġseems Ġto Ġbe Ġthe Ġdefault , Ġand Ġothers Ġseem Ġto Ġbe Ġmarked . ĠHow Ġcan Ġwe Ġtell Ġwhether Ġor Ġnot Ġthe Ġdata Ġis Ġactually Ġbiased ? </s> We Ġdon 't Ġknow Ġwhether Ġor Ġnot Ġan Ġentity Ġbelongs Ġto Ġa Ġparticular Ġsocial Ġclass Ġ( in Ġthis Ġcase : Ġethnic Ġgroup ) Ġuntil Ġit Ġis Ġmarked Ġas Ġsuch . ĠBut Ġwe Ġcan Ġapproximate Ġthe Ġproportion Ġby Ġlooking Ġat Ġall Ġthe Ġimages Ġwhere Ġthe Ġannot ators Ġhave Ġused Ġa Ġmarker Ġ( in Ġthis Ġcase : Ġadject ives Ġlike Ġblack , Ġwhite , Ġas ian ), Ġand Ġfor Ġthose Ġimages Ġcount Ġhow Ġmany Ġdescriptions Ġ( out Ġof Ġfive ) Ġcontain Ġa Ġmarker . ĠThis Ġgives Ġus Ġan Ġupper Ġbound Ġthat Ġtells Ġus Ġhow Ġoften Ġethnicity Ġis Ġindicated Ġby Ġthe Ġannot ators . ĠNote Ġthat Ġthis Ġupper Ġbound Ġlies Ġsomewhere Ġbetween Ġ20 % Ġ( one Ġdescription ) Ġand Ġ100 % Ġ( 5 Ġdescriptions ). ĠFigure ĠT AB REF 22 Ġpresents Ġcount Ġdata Ġfor Ġthe Ġethnic Ġmarking Ġof Ġbabies . ĠIt Ġincludes Ġtwo Ġfalse Ġpositives Ġ( talking Ġabout Ġa Ġwhite Ġbaby Ġst roller Ġrather Ġthan Ġa Ġwhite Ġbaby ). ĠIn Ġthe ĠAsian Ġgroup Ġthere Ġis Ġan Ġadditional Ġcomplication : Ġsometimes Ġthe Ġmother Ġgets Ġmarked Ġrather Ġthan Ġthe Ġbaby . ĠE . g . ĠAn ĠAsian Ġwoman Ġholds Ġa Ġbaby Ġgirl . ĠI Ġhave Ġcounted Ġthese Ġoccurrences Ġas Ġwell . </s> The Ġnumbers Ġin ĠTable ĠT AB REF 22 Ġare Ġstriking : Ġthere Ġseems Ġto Ġbe Ġa Ġreal , Ġsystematic Ġdifference Ġin Ġethnicity Ġmarking Ġbetween Ġthe Ġgroups . ĠWe Ġcan Ġtake Ġone Ġstep Ġfurther Ġand Ġlook Ġat Ġall Ġthe Ġ6 97 Ġpictures Ġwith Ġthe Ġword Ġ` baby ' Ġin Ġit . ĠIf Ġthere Ġturn Ġout Ġto Ġbe Ġdisproportionately Ġmany Ġwhite Ġbabies , Ġthis Ġstrengthens Ġthe Ġconclusion Ġthat Ġthe Ġdataset Ġis Ġbiased . </s> I Ġhave Ġmanually Ġcategorized Ġeach Ġof Ġthe Ġbaby Ġimages . ĠThere Ġare Ġ504 Ġwhite , Ġ66 Ġas ian , Ġand Ġ36 Ġblack Ġbabies . Ġ73 Ġimages Ġdo Ġnot Ġcontain Ġa Ġbaby , Ġand Ġ18 Ġimages Ġdo Ġnot Ġfall Ġinto Ġany Ġof Ġthe Ġother Ġcategories . ĠWhile Ġthis Ġdoes Ġbring Ġdown Ġthe Ġaverage Ġnumber Ġof Ġtimes Ġeach Ġcategory Ġwas Ġmarked , Ġit Ġalso Ġincreases Ġthe Ġcontrast Ġbetween Ġwhite Ġbabies Ġ( who Ġget Ġmarked Ġin Ġless Ġthan Ġ1 % Ġof Ġthe Ġimages ) Ġand Ġas ian / black Ġbabies Ġ( who Ġget Ġmarked Ġmuch Ġmore Ġoften ). ĠA Ġnext Ġstep Ġwould Ġbe Ġto Ġsee Ġwhether Ġthese Ġobservations Ġalso Ġhold Ġfor Ġother Ġage Ġgroups , Ġi . e . Ġchildren Ġand Ġadults . ĠIN LINE FORM 0 </s> Other Ġmethods </s> It Ġmay Ġbe Ġdifficult Ġto Ġspot Ġpatterns Ġby Ġjust Ġlooking Ġat Ġa Ġcollection Ġof Ġimages . ĠAnother Ġmethod Ġis Ġto Ġtag Ġall Ġdescriptions Ġwith Ġpart - of - speech Ġinformation , Ġso Ġthat Ġit Ġbecomes Ġpossible Ġto Ġsee Ġe . g . Ġwhich Ġadject ives Ġare Ġmost Ġcommonly Ġused Ġfor Ġparticular Ġnoun s . ĠOne Ġmethod Ġreaders Ġmay Ġfind Ġparticularly Ġuseful Ġis Ġto Ġleverage Ġthe Ġstructure Ġof ĠFlickr 30 K ĠEnt ities ĠB IB REF 8 Ġ. ĠThis Ġdataset Ġen ric hes ĠFlickr 30 K Ġby Ġadding Ġcore ference Ġannotations , Ġi . e . Ġwhich Ġphrase Ġin Ġeach Ġdescription Ġrefers Ġto Ġthe Ġsame Ġentity Ġin Ġthe Ġcorresponding Ġimage . ĠI Ġhave Ġused Ġthis Ġdata Ġto Ġcreate Ġa Ġcore ference Ġgraph Ġby Ġlinking Ġall Ġphrases Ġthat Ġrefer Ġto Ġthe Ġsame Ġentity . ĠFollowing Ġthis , ĠI Ġapplied ĠLou v ain Ġclust ering ĠB IB REF 9 Ġto Ġthe Ġcore ference Ġgraph , Ġresulting Ġin Ġclusters Ġof Ġexpressions Ġthat Ġrefer Ġto Ġsimilar Ġentities . ĠLooking Ġat Ġthose Ġclusters Ġhelps Ġto Ġget Ġa Ġsense Ġof Ġthe Ġenormous Ġvariation Ġin Ġreferring Ġexpressions . ĠTo Ġget Ġan Ġidea Ġof Ġthe Ġrichness Ġof Ġthis Ġdata , Ġhere Ġis Ġa Ġsmall Ġsample Ġof Ġthe Ġphrases Ġused Ġto Ġdescribe Ġbe ards Ġ( cl uster Ġ268 ): Ġa Ġsc ruff y Ġbeard ; Ġa Ġthick Ġbeard ; Ġlarge Ġwhite Ġbeard ; Ġa Ġbubble Ġbeard ; Ġred Ġfacial Ġhair ; Ġa Ġbra ided Ġbeard ; Ġa Ġflaming Ġred Ġbeard . ĠIn Ġthis Ġcase , Ġ` red Ġfacial Ġhair ' Ġreally Ġstands Ġout Ġas Ġa Ġdescription ; Ġwhy Ġnot Ġchoose Ġthe Ġsimpler Ġ` beard ' Ġinstead ? </s> Discussion </s> In Ġthe Ġprevious Ġsection , ĠI Ġhave Ġoutlined Ġseveral Ġmethods Ġto Ġmanually Ġdetect Ġstereotypes , Ġbiases , Ġand Ġodd Ġphrases . ĠBecause Ġthere Ġare Ġmany Ġways Ġin Ġwhich Ġa Ġphrase Ġcan Ġbe Ġbiased , Ġit Ġis Ġdifficult Ġto Ġautomatically Ġdetect Ġbias Ġfrom Ġthe Ġdata . ĠSo Ġhow Ġshould Ġwe Ġdeal Ġwith Ġstereotype - driven Ġdescriptions ? </s> Conclusion </s> This Ġpaper Ġprovided Ġa Ġtax onomy Ġof Ġstereotype - driven Ġdescriptions Ġin Ġthe ĠFlickr 30 K Ġdataset . ĠI Ġhave Ġdivided Ġthese Ġdescriptions Ġinto Ġtwo Ġclasses : Ġlinguistic Ġbias Ġand Ġunw arranted Ġinf erences . ĠThe Ġformer Ġcorresponds Ġto Ġthe Ġannot ators ' Ġchoice Ġof Ġwords Ġwhen Ġconfronted Ġwith Ġan Ġimage Ġthat Ġmay Ġor Ġmay Ġnot Ġmatch Ġtheir Ġstereotypical Ġexpect ancies . ĠThe Ġlatter Ġcorresponds Ġto Ġthe Ġtendency Ġof Ġannot ators Ġto Ġgo Ġbeyond Ġwhat Ġthe Ġphysical Ġdata Ġcan Ġtell Ġus , Ġand Ġexpand Ġtheir Ġdescriptions Ġbased Ġon Ġtheir Ġpast Ġexperiences Ġand Ġknowledge Ġof Ġthe Ġworld . ĠA cknowled ging Ġthese Ġphenomena Ġis Ġimportant , Ġbecause Ġon Ġthe Ġone Ġhand Ġit Ġhelps Ġus Ġthink Ġabout Ġwhat Ġis Ġlearn able Ġfrom Ġthe Ġdata , Ġand Ġon Ġthe Ġother Ġhand Ġit Ġserves Ġas Ġa Ġwarning : Ġif Ġwe Ġtrain Ġand Ġevaluate Ġlanguage Ġmodels Ġon Ġthis Ġdata , Ġwe Ġare Ġeffectively Ġteaching Ġthem Ġto Ġbe Ġbiased . </s> I Ġhave Ġalso Ġlooked Ġat Ġmethods Ġto Ġdetect Ġstereotype - driven Ġdescriptions , Ġbut Ġdue Ġto Ġthe Ġrichness Ġof Ġlanguage Ġit Ġis Ġdifficult Ġto Ġfind Ġan Ġautomated Ġmeasure . ĠDepending Ġon Ġwhether Ġyour Ġgoal Ġis Ġproduction Ġor Ġinterpretation , Ġit Ġmay Ġeither Ġbe Ġuseful Ġto Ġsuppress Ġor Ġto Ġemphasize Ġbiases Ġin Ġhuman Ġlanguage . ĠFinally , ĠI Ġhave Ġdiscussed Ġstereotyp ing Ġbehavior Ġas Ġthe Ġaddition Ġof Ġa Ġcontextual Ġlayer Ġon Ġtop Ġof Ġa Ġmore Ġbasic Ġdescription . ĠThis Ġraises Ġthe Ġquestion Ġwhat Ġkind Ġof Ġdescriptions Ġwe Ġwould Ġlike Ġour Ġmodels Ġto Ġproduce . </s> Acknowled gments </s> Thanks Ġto ĠPie k ĠV oss en Ġand ĠAnt s ke ĠF ok kens Ġfor Ġdiscussion , Ġand Ġto ĠDesmond ĠElliott Ġand Ġan Ġanonymous Ġreviewer Ġfor Ġcomments Ġon Ġan Ġearlier Ġversion Ġof Ġthis Ġpaper . ĠThis Ġresearch Ġwas Ġsupported Ġby Ġthe ĠNetherlands ĠOrganization Ġfor ĠScientific ĠResearch Ġ( NW O ) Ġvia Ġthe ĠSp ino za - pri ze Ġawarded Ġto ĠPie k ĠV oss en Ġ( SP I Ġ30 - 673 , Ġ2014 - 2019 ).
Introduction The Flickr30K dataset BIBREF0 is a collection of over 30,000 images with 5 crowdsourced descriptions each. It is commonly used to train and evaluate neural network models that generate image descriptions (e.g. BIBREF2 ). An untested assumption behind the dataset is that the descriptions are based on the images, and nothing else. Here are the authors (about the Flickr8K dataset, a subset of Flickr30K): “By asking people to describe the people, objects, scenes and activities that are shown in a picture without giving them any further information about the context in which the picture was taken, we were able to obtain conceptual descriptions that focus only on the information that can be obtained from the image alone.” BIBREF1 What this assumption overlooks is the amount of interpretation or recontextualization carried out by the annotators. Let us take a concrete example. Figure FIGREF1 shows an image from the Flickr30K dataset. This image comes with the five descriptions below. All but the first one contain information that cannot come from the image alone. Relevant parts are highlighted in bold: We need to understand that the descriptions in the Flickr30K dataset are subjective descriptions of events. This can be a good thing: the descriptions tell us what are the salient parts of each image to the average human annotator. So the two humans in Figure FIGREF1 are relevant, but the two soap dispensers are not. But subjectivity can also result in stereotypical descriptions, in this case suggesting that the male is more likely to be the manager, and the female is more likely to be the subordinate. rashtchian2010collecting do note that some descriptions are speculative in nature, which they say hurts the accuracy and the consistency of the descriptions. But the problem is not with the lack of consistency here. Quite the contrary: the problem is that stereotypes may be pervasive enough for the data to be consistently biased. And so language models trained on this data may propagate harmful stereotypes, such as the idea that women are less suited for leadership positions. This paper aims to give an overview of linguistic bias and unwarranted inferences resulting from stereotypes and prejudices. I will build on earlier work on linguistic bias in general BIBREF3 , providing examples from the Flickr30K data, and present a taxonomy of unwarranted inferences. Finally, I will discuss several methods to analyze the data in order to detect biases. Stereotype-driven descriptions Stereotypes are ideas about how other (groups of) people commonly behave and what they are likely to do. These ideas guide the way we talk about the world. I distinguish two kinds of verbal behavior that result from stereotypes: (i) linguistic bias, and (ii) unwarranted inferences. The former is discussed in more detail by beukeboom2014mechanisms, who defines linguistic bias as “a systematic asymmetry in word choice as a function of the social category to which the target belongs.” So this bias becomes visible through the distribution of terms used to describe entities in a particular category. Unwarranted inferences are the result of speculation about the image; here, the annotator goes beyond what can be glanced from the image and makes use of their knowledge and expectations about the world to provide an overly specific description. Such descriptions are directly identifiable as such, and in fact we have already seen four of them (descriptions 2–5) discussed earlier. Linguistic bias Generally speaking, people tend to use more concrete or specific language when they have to describe a person that does not meet their expectations. beukeboom2014mechanisms lists several linguistic `tools' that people use to mark individuals who deviate from the norm. I will mention two of them. [leftmargin=0cm] One well-studied example BIBREF4 , BIBREF5 is sexist language, where the sex of a person tends to be mentioned more frequently if their role or occupation is inconsistent with `traditional' gender roles (e.g. female surgeon, male nurse). Beukeboom also notes that adjectives are used to create “more narrow labels [or subtypes] for individuals who do not fit with general social category expectations” (p. 3). E.g. tough woman makes an exception to the `rule' that women aren't considered to be tough. can be used when prior beliefs about a particular social category are violated, e.g. The garbage man was not stupid. See also BIBREF6 . These examples are similar in that the speaker has to put in additional effort to mark the subject for being unusual. But they differ in what we can conclude about the speaker, especially in the context of the Flickr30K data. Negations are much more overtly displaying the annotator's prior beliefs. When one annotator writes that A little boy is eating pie without utensils (image 2659046789), this immediately reveals the annotator's normative beliefs about the world: pie should be eaten with utensils. But when another annotator talks about a girls basketball game (image 8245366095), this cannot be taken as an indication that the annotator is biased about the gender of basketball players; they might just be helpful by providing a detailed description. In section 3 I will discuss how to establish whether or not there is any bias in the data regarding the use of adjectives. Unwarranted inferences Unwarranted inferences are statements about the subject(s) of an image that go beyond what the visual data alone can tell us. They are based on additional assumptions about the world. After inspecting a subset of the Flickr30K data, I have grouped these inferences into six categories (image examples between parentheses): [leftmargin=0cm] We've seen an example of this in the introduction, where the `manager' was said to be talking about job performance and scolding [a worker] in a stern lecture (8063007). Many dark-skinned individuals are called African-American regardless of whether the picture has been taken in the USA or not (4280272). And people who look Asian are called Chinese (1434151732) or Japanese (4834664666). In image 4183120 (Figure FIGREF16 ), people sitting at a gym are said to be watching a game, even though there could be any sort of event going on. But since the location is so strongly associated with sports, crowdworkers readily make the assumption. Quite a few annotations focus on explaining the why of the situation. For example, in image 3963038375 a man is fastening his climbing harness in order to have some fun. And in an extreme case, one annotator writes about a picture of a dancing woman that the school is having a special event in order to show the american culture on how other cultures are dealt with in parties (3636329461). This is reminiscent of the Stereotypic Explanatory Bias BIBREF7 , which refers to “the tendency to provide relatively more explanations in descriptions of stereotype inconsistent, compared to consistent behavior” BIBREF6 . So in theory, odd or surprising situations should receive more explanations, since a description alone may not make enough sense in those cases, but it is beyond the scope of this paper to test whether or not the Flickr30K data suffers from the SEB. Older people with children around them are commonly seen as parents (5287405), small children as siblings (205842), men and women as lovers (4429660), groups of young people as friends (36979). Annotators will often guess the status or occupation of people in an image. Sometimes these guesses are relatively general (e.g. college-aged people being called students in image 36979), but other times these are very specific (e.g. a man in a workshop being called a graphics designer, 5867606). Detecting stereotype-driven descriptions In order to get an idea of the kinds of stereotype-driven descriptions that are in the Flickr30K dataset, I made a browser-based annotation tool that shows both the images and their associated descriptions. You can simply leaf through the images by clicking `Next' or `Random' until you find an interesting pattern. Ethnicity/race One interesting pattern is that the ethnicity/race of babies doesn't seem to be mentioned unless the baby is black or asian. In other words: white seems to be the default, and others seem to be marked. How can we tell whether or not the data is actually biased? We don't know whether or not an entity belongs to a particular social class (in this case: ethnic group) until it is marked as such. But we can approximate the proportion by looking at all the images where the annotators have used a marker (in this case: adjectives like black, white, asian), and for those images count how many descriptions (out of five) contain a marker. This gives us an upper bound that tells us how often ethnicity is indicated by the annotators. Note that this upper bound lies somewhere between 20% (one description) and 100% (5 descriptions). Figure TABREF22 presents count data for the ethnic marking of babies. It includes two false positives (talking about a white baby stroller rather than a white baby). In the Asian group there is an additional complication: sometimes the mother gets marked rather than the baby. E.g. An Asian woman holds a baby girl. I have counted these occurrences as well. The numbers in Table TABREF22 are striking: there seems to be a real, systematic difference in ethnicity marking between the groups. We can take one step further and look at all the 697 pictures with the word `baby' in it. If there turn out to be disproportionately many white babies, this strengthens the conclusion that the dataset is biased. I have manually categorized each of the baby images. There are 504 white, 66 asian, and 36 black babies. 73 images do not contain a baby, and 18 images do not fall into any of the other categories. While this does bring down the average number of times each category was marked, it also increases the contrast between white babies (who get marked in less than 1% of the images) and asian/black babies (who get marked much more often). A next step would be to see whether these observations also hold for other age groups, i.e. children and adults. INLINEFORM0 Other methods It may be difficult to spot patterns by just looking at a collection of images. Another method is to tag all descriptions with part-of-speech information, so that it becomes possible to see e.g. which adjectives are most commonly used for particular nouns. One method readers may find particularly useful is to leverage the structure of Flickr30K Entities BIBREF8 . This dataset enriches Flickr30K by adding coreference annotations, i.e. which phrase in each description refers to the same entity in the corresponding image. I have used this data to create a coreference graph by linking all phrases that refer to the same entity. Following this, I applied Louvain clustering BIBREF9 to the coreference graph, resulting in clusters of expressions that refer to similar entities. Looking at those clusters helps to get a sense of the enormous variation in referring expressions. To get an idea of the richness of this data, here is a small sample of the phrases used to describe beards (cluster 268): a scruffy beard; a thick beard; large white beard; a bubble beard; red facial hair; a braided beard; a flaming red beard. In this case, `red facial hair' really stands out as a description; why not choose the simpler `beard' instead? Discussion In the previous section, I have outlined several methods to manually detect stereotypes, biases, and odd phrases. Because there are many ways in which a phrase can be biased, it is difficult to automatically detect bias from the data. So how should we deal with stereotype-driven descriptions? Conclusion This paper provided a taxonomy of stereotype-driven descriptions in the Flickr30K dataset. I have divided these descriptions into two classes: linguistic bias and unwarranted inferences. The former corresponds to the annotators' choice of words when confronted with an image that may or may not match their stereotypical expectancies. The latter corresponds to the tendency of annotators to go beyond what the physical data can tell us, and expand their descriptions based on their past experiences and knowledge of the world. Acknowledging these phenomena is important, because on the one hand it helps us think about what is learnable from the data, and on the other hand it serves as a warning: if we train and evaluate language models on this data, we are effectively teaching them to be biased. I have also looked at methods to detect stereotype-driven descriptions, but due to the richness of language it is difficult to find an automated measure. Depending on whether your goal is production or interpretation, it may either be useful to suppress or to emphasize biases in human language. Finally, I have discussed stereotyping behavior as the addition of a contextual layer on top of a more basic description. This raises the question what kind of descriptions we would like our models to produce. Acknowledgments Thanks to Piek Vossen and Antske Fokkens for discussion, and to Desmond Elliott and an anonymous reviewer for comments on an earlier version of this paper. This research was supported by the Netherlands Organization for Scientific Research (NWO) via the Spinoza-prize awarded to Piek Vossen (SPI 30-673, 2014-2019).
What evaluations methods do they take?
71e4ba4e87e6596aeca187127c0d088df6570c57
What Ġevaluations Ġmethods Ġdo Ġthey Ġtake ?
Un answer able
<s> What Ġevaluations Ġmethods Ġdo Ġthey Ġtake ? </s> Introduction </s> The ĠFlickr 30 K Ġdataset ĠB IB REF 0 Ġis Ġa Ġcollection Ġof Ġover Ġ30 , 000 Ġimages Ġwith Ġ5 Ġcrowds ourced Ġdescriptions Ġeach . ĠIt Ġis Ġcommonly Ġused Ġto Ġtrain Ġand Ġevaluate Ġneural Ġnetwork Ġmodels Ġthat Ġgenerate Ġimage Ġdescriptions Ġ( e . g . ĠB IB REF 2 Ġ). ĠAn Ġunt ested Ġassumption Ġbehind Ġthe Ġdataset Ġis Ġthat Ġthe Ġdescriptions Ġare Ġbased Ġon Ġthe Ġimages , Ġand Ġnothing Ġelse . ĠHere Ġare Ġthe Ġauthors Ġ( about Ġthe ĠFlickr 8 K Ġdataset , Ġa Ġsubset Ġof ĠFlickr 30 K ): </s> âĢ ľ By Ġasking Ġpeople Ġto Ġdescribe Ġthe Ġpeople , Ġobjects , Ġscenes Ġand Ġactivities Ġthat Ġare Ġshown Ġin Ġa Ġpicture Ġwithout Ġgiving Ġthem Ġany Ġfurther Ġinformation Ġabout Ġthe Ġcontext Ġin Ġwhich Ġthe Ġpicture Ġwas Ġtaken , Ġwe Ġwere Ġable Ġto Ġobtain Ġconceptual Ġdescriptions Ġthat Ġfocus Ġonly Ġon Ġthe Ġinformation Ġthat Ġcan Ġbe Ġobtained Ġfrom Ġthe Ġimage Ġalone . âĢ Ŀ ĠB IB REF 1 </s> What Ġthis Ġassumption Ġoverlook s Ġis Ġthe Ġamount Ġof Ġinterpretation Ġor Ġrecon text ual ization Ġcarried Ġout Ġby Ġthe Ġannot ators . ĠLet Ġus Ġtake Ġa Ġconcrete Ġexample . ĠFigure ĠFIG REF 1 Ġshows Ġan Ġimage Ġfrom Ġthe ĠFlickr 30 K Ġdataset . </s> This Ġimage Ġcomes Ġwith Ġthe Ġfive Ġdescriptions Ġbelow . ĠAll Ġbut Ġthe Ġfirst Ġone Ġcontain Ġinformation Ġthat Ġcannot Ġcome Ġfrom Ġthe Ġimage Ġalone . ĠRe levant Ġparts Ġare Ġhighlighted Ġin Ġbold : </s> We Ġneed Ġto Ġunderstand Ġthat Ġthe Ġdescriptions Ġin Ġthe ĠFlickr 30 K Ġdataset Ġare Ġsubjective Ġdescriptions Ġof Ġevents . ĠThis Ġcan Ġbe Ġa Ġgood Ġthing : Ġthe Ġdescriptions Ġtell Ġus Ġwhat Ġare Ġthe Ġsalient Ġparts Ġof Ġeach Ġimage Ġto Ġthe Ġaverage Ġhuman Ġannot ator . ĠSo Ġthe Ġtwo Ġhumans Ġin ĠFigure ĠFIG REF 1 Ġare Ġrelevant , Ġbut Ġthe Ġtwo Ġsoap Ġdispens ers Ġare Ġnot . ĠBut Ġsubject ivity Ġcan Ġalso Ġresult Ġin Ġstereotypical Ġdescriptions , Ġin Ġthis Ġcase Ġsuggesting Ġthat Ġthe Ġmale Ġis Ġmore Ġlikely Ġto Ġbe Ġthe Ġmanager , Ġand Ġthe Ġfemale Ġis Ġmore Ġlikely Ġto Ġbe Ġthe Ġsubordinate . Ġrash tch ian 2010 collect ing Ġdo Ġnote Ġthat Ġsome Ġdescriptions Ġare Ġspeculative Ġin Ġnature , Ġwhich Ġthey Ġsay Ġhurts Ġthe Ġaccuracy Ġand Ġthe Ġconsistency Ġof Ġthe Ġdescriptions . ĠBut Ġthe Ġproblem Ġis Ġnot Ġwith Ġthe Ġlack Ġof Ġconsistency Ġhere . ĠQuite Ġthe Ġcontrary : Ġthe Ġproblem Ġis Ġthat Ġstereotypes Ġmay Ġbe Ġpervasive Ġenough Ġfor Ġthe Ġdata Ġto Ġbe Ġconsistently Ġbiased . ĠAnd Ġso Ġlanguage Ġmodels Ġtrained Ġon Ġthis Ġdata Ġmay Ġpropagate Ġharmful Ġstereotypes , Ġsuch Ġas Ġthe Ġidea Ġthat Ġwomen Ġare Ġless Ġsuited Ġfor Ġleadership Ġpositions . </s> This Ġpaper Ġaims Ġto Ġgive Ġan Ġoverview Ġof Ġlinguistic Ġbias Ġand Ġunw arranted Ġinf erences Ġresulting Ġfrom Ġstereotypes Ġand Ġprejudices . ĠI Ġwill Ġbuild Ġon Ġearlier Ġwork Ġon Ġlinguistic Ġbias Ġin Ġgeneral ĠB IB REF 3 Ġ, Ġproviding Ġexamples Ġfrom Ġthe ĠFlickr 30 K Ġdata , Ġand Ġpresent Ġa Ġtax onomy Ġof Ġunw arranted Ġinf erences . ĠFinally , ĠI Ġwill Ġdiscuss Ġseveral Ġmethods Ġto Ġanalyze Ġthe Ġdata Ġin Ġorder Ġto Ġdetect Ġbiases . </s> St ere otype - driven Ġdescriptions </s> St ere otypes Ġare Ġideas Ġabout Ġhow Ġother Ġ( groups Ġof ) Ġpeople Ġcommonly Ġbehave Ġand Ġwhat Ġthey Ġare Ġlikely Ġto Ġdo . ĠThese Ġideas Ġguide Ġthe Ġway Ġwe Ġtalk Ġabout Ġthe Ġworld . ĠI Ġdistinguish Ġtwo Ġkinds Ġof Ġverbal Ġbehavior Ġthat Ġresult Ġfrom Ġstereotypes : Ġ( i ) Ġlinguistic Ġbias , Ġand Ġ( ii ) Ġunw arranted Ġinf erences . ĠThe Ġformer Ġis Ġdiscussed Ġin Ġmore Ġdetail Ġby Ġbe uke bo om 2014 me chan isms , Ġwho Ġdefines Ġlinguistic Ġbias Ġas ĠâĢ ľ a Ġsystematic Ġasymm etry Ġin Ġword Ġchoice Ġas Ġa Ġfunction Ġof Ġthe Ġsocial Ġcategory Ġto Ġwhich Ġthe Ġtarget Ġbelongs . âĢ Ŀ ĠSo Ġthis Ġbias Ġbecomes Ġvisible Ġthrough Ġthe Ġdistribution Ġof Ġterms Ġused Ġto Ġdescribe Ġentities Ġin Ġa Ġparticular Ġcategory . ĠUn w arranted Ġinf erences Ġare Ġthe Ġresult Ġof Ġspeculation Ġabout Ġthe Ġimage ; Ġhere , Ġthe Ġannot ator Ġgoes Ġbeyond Ġwhat Ġcan Ġbe Ġglanced Ġfrom Ġthe Ġimage Ġand Ġmakes Ġuse Ġof Ġtheir Ġknowledge Ġand Ġexpectations Ġabout Ġthe Ġworld Ġto Ġprovide Ġan Ġoverly Ġspecific Ġdescription . ĠSuch Ġdescriptions Ġare Ġdirectly Ġidentifiable Ġas Ġsuch , Ġand Ġin Ġfact Ġwe Ġhave Ġalready Ġseen Ġfour Ġof Ġthem Ġ( desc ript ions Ġ2 âĢĵ 5 ) Ġdiscussed Ġearlier . </s> L ingu istic Ġbias </s> Generally Ġspeaking , Ġpeople Ġtend Ġto Ġuse Ġmore Ġconcrete Ġor Ġspecific Ġlanguage Ġwhen Ġthey Ġhave Ġto Ġdescribe Ġa Ġperson Ġthat Ġdoes Ġnot Ġmeet Ġtheir Ġexpectations . Ġbe uke bo om 2014 me chan isms Ġlists Ġseveral Ġlinguistic Ġ` tools ' Ġthat Ġpeople Ġuse Ġto Ġmark Ġindividuals Ġwho Ġdev iate Ġfrom Ġthe Ġnorm . ĠI Ġwill Ġmention Ġtwo Ġof Ġthem . </s> [ left margin = 0 cm ] </s> One Ġwell - stud ied Ġexample ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġis Ġsexist Ġlanguage , Ġwhere Ġthe Ġsex Ġof Ġa Ġperson Ġtends Ġto Ġbe Ġmentioned Ġmore Ġfrequently Ġif Ġtheir Ġrole Ġor Ġoccupation Ġis Ġinconsistent Ġwith Ġ` traditional ' Ġgender Ġroles Ġ( e . g . Ġfemale Ġsurgeon , Ġmale Ġnurse ). ĠBe uke bo om Ġalso Ġnotes Ġthat Ġadject ives Ġare Ġused Ġto Ġcreate ĠâĢ ľ more Ġnarrow Ġlabels Ġ[ or Ġsub types ] Ġfor Ġindividuals Ġwho Ġdo Ġnot Ġfit Ġwith Ġgeneral Ġsocial Ġcategory Ġexpectations âĢ Ŀ Ġ( p . Ġ3 ). ĠE . g . Ġtough Ġwoman Ġmakes Ġan Ġexception Ġto Ġthe Ġ` rule ' Ġthat Ġwomen Ġaren 't Ġconsidered Ġto Ġbe Ġtough . </s> can Ġbe Ġused Ġwhen Ġprior Ġbeliefs Ġabout Ġa Ġparticular Ġsocial Ġcategory Ġare Ġviolated , Ġe . g . ĠThe Ġgarbage Ġman Ġwas Ġnot Ġstupid . ĠSee Ġalso ĠB IB REF 6 Ġ. </s> These Ġexamples Ġare Ġsimilar Ġin Ġthat Ġthe Ġspeaker Ġhas Ġto Ġput Ġin Ġadditional Ġeffort Ġto Ġmark Ġthe Ġsubject Ġfor Ġbeing Ġunusual . ĠBut Ġthey Ġdiffer Ġin Ġwhat Ġwe Ġcan Ġconclude Ġabout Ġthe Ġspeaker , Ġespecially Ġin Ġthe Ġcontext Ġof Ġthe ĠFlickr 30 K Ġdata . ĠNeg ations Ġare Ġmuch Ġmore Ġovertly Ġdisplaying Ġthe Ġannot ator 's Ġprior Ġbeliefs . ĠWhen Ġone Ġannot ator Ġwrites Ġthat ĠA Ġlittle Ġboy Ġis Ġeating Ġpie Ġwithout Ġut ens ils Ġ( image Ġ26 59 04 67 89 ), Ġthis Ġimmediately Ġreveals Ġthe Ġannot ator 's Ġnormative Ġbeliefs Ġabout Ġthe Ġworld : Ġpie Ġshould Ġbe Ġeaten Ġwith Ġut ens ils . ĠBut Ġwhen Ġanother Ġannot ator Ġtalks Ġabout Ġa Ġgirls Ġbasketball Ġgame Ġ( image Ġ8 245 36 60 95 ), Ġthis Ġcannot Ġbe Ġtaken Ġas Ġan Ġindication Ġthat Ġthe Ġannot ator Ġis Ġbiased Ġabout Ġthe Ġgender Ġof Ġbasketball Ġplayers ; Ġthey Ġmight Ġjust Ġbe Ġhelpful Ġby Ġproviding Ġa Ġdetailed Ġdescription . ĠIn Ġsection Ġ3 ĠI Ġwill Ġdiscuss Ġhow Ġto Ġestablish Ġwhether Ġor Ġnot Ġthere Ġis Ġany Ġbias Ġin Ġthe Ġdata Ġregarding Ġthe Ġuse Ġof Ġadject ives . </s> Un w arranted Ġinf erences </s> Un w arranted Ġinf erences Ġare Ġstatements Ġabout Ġthe Ġsubject ( s ) Ġof Ġan Ġimage Ġthat Ġgo Ġbeyond Ġwhat Ġthe Ġvisual Ġdata Ġalone Ġcan Ġtell Ġus . ĠThey Ġare Ġbased Ġon Ġadditional Ġassumptions Ġabout Ġthe Ġworld . ĠAfter Ġinspecting Ġa Ġsubset Ġof Ġthe ĠFlickr 30 K Ġdata , ĠI Ġhave Ġgrouped Ġthese Ġinf erences Ġinto Ġsix Ġcategories Ġ( image Ġexamples Ġbetween Ġparentheses ): </s> [ left margin = 0 cm ] </s> We 've Ġseen Ġan Ġexample Ġof Ġthis Ġin Ġthe Ġintroduction , Ġwhere Ġthe Ġ` manager ' Ġwas Ġsaid Ġto Ġbe Ġtalking Ġabout Ġjob Ġperformance Ġand Ġsc olding Ġ[ a Ġworker ] Ġin Ġa Ġstern Ġlecture Ġ( 80 63 007 ). </s> Many Ġdark - skinned Ġindividuals Ġare Ġcalled ĠAfrican - American Ġregardless Ġof Ġwhether Ġthe Ġpicture Ġhas Ġbeen Ġtaken Ġin Ġthe ĠUSA Ġor Ġnot Ġ( 42 80 272 ). ĠAnd Ġpeople Ġwho Ġlook ĠAsian Ġare Ġcalled ĠChinese Ġ( 14 34 15 17 32 ) Ġor ĠJapanese Ġ( 48 346 64 666 ). </s> In Ġimage Ġ4 183 120 Ġ( Figure ĠFIG REF 16 Ġ), Ġpeople Ġsitting Ġat Ġa Ġgym Ġare Ġsaid Ġto Ġbe Ġwatching Ġa Ġgame , Ġeven Ġthough Ġthere Ġcould Ġbe Ġany Ġsort Ġof Ġevent Ġgoing Ġon . ĠBut Ġsince Ġthe Ġlocation Ġis Ġso Ġstrongly Ġassociated Ġwith Ġsports , Ġcrowd workers Ġreadily Ġmake Ġthe Ġassumption . </s> Quite Ġa Ġfew Ġannotations Ġfocus Ġon Ġexplaining Ġthe Ġwhy Ġof Ġthe Ġsituation . ĠFor Ġexample , Ġin Ġimage Ġ396 30 38 375 Ġa Ġman Ġis Ġfast ening Ġhis Ġclimbing Ġharness Ġin Ġorder Ġto Ġhave Ġsome Ġfun . ĠAnd Ġin Ġan Ġextreme Ġcase , Ġone Ġannot ator Ġwrites Ġabout Ġa Ġpicture Ġof Ġa Ġdancing Ġwoman Ġthat Ġthe Ġschool Ġis Ġhaving Ġa Ġspecial Ġevent Ġin Ġorder Ġto Ġshow Ġthe Ġameric an Ġculture Ġon Ġhow Ġother Ġcultures Ġare Ġdealt Ġwith Ġin Ġparties Ġ( 36 363 29 461 ). ĠThis Ġis Ġreminiscent Ġof Ġthe ĠSt ere otyp ic ĠExplan atory ĠB ias ĠB IB REF 7 Ġ, Ġwhich Ġrefers Ġto ĠâĢ ľ the Ġtendency Ġto Ġprovide Ġrelatively Ġmore Ġexplanations Ġin Ġdescriptions Ġof Ġstereotype Ġinconsistent , Ġcompared Ġto Ġconsistent Ġbehavior âĢ Ŀ ĠB IB REF 6 Ġ. ĠSo Ġin Ġtheory , Ġodd Ġor Ġsurprising Ġsituations Ġshould Ġreceive Ġmore Ġexplanations , Ġsince Ġa Ġdescription Ġalone Ġmay Ġnot Ġmake Ġenough Ġsense Ġin Ġthose Ġcases , Ġbut Ġit Ġis Ġbeyond Ġthe Ġscope Ġof Ġthis Ġpaper Ġto Ġtest Ġwhether Ġor Ġnot Ġthe ĠFlickr 30 K Ġdata Ġsuffers Ġfrom Ġthe ĠSE B . </s> O lder Ġpeople Ġwith Ġchildren Ġaround Ġthem Ġare Ġcommonly Ġseen Ġas Ġparents Ġ( 5 287 405 ), Ġsmall Ġchildren Ġas Ġsiblings Ġ( 20 58 42 ), Ġmen Ġand Ġwomen Ġas Ġlovers Ġ( 44 296 60 ), Ġgroups Ġof Ġyoung Ġpeople Ġas Ġfriends Ġ( 369 79 ). </s> An not ators Ġwill Ġoften Ġguess Ġthe Ġstatus Ġor Ġoccupation Ġof Ġpeople Ġin Ġan Ġimage . ĠSometimes Ġthese Ġguesses Ġare Ġrelatively Ġgeneral Ġ( e . g . Ġcollege - aged Ġpeople Ġbeing Ġcalled Ġstudents Ġin Ġimage Ġ369 79 ), Ġbut Ġother Ġtimes Ġthese Ġare Ġvery Ġspecific Ġ( e . g . Ġa Ġman Ġin Ġa Ġworkshop Ġbeing Ġcalled Ġa Ġgraphics Ġdesigner , Ġ58 67 606 ). </s> Detect ing Ġstereotype - driven Ġdescriptions </s> In Ġorder Ġto Ġget Ġan Ġidea Ġof Ġthe Ġkinds Ġof Ġstereotype - driven Ġdescriptions Ġthat Ġare Ġin Ġthe ĠFlickr 30 K Ġdataset , ĠI Ġmade Ġa Ġbrowser - based Ġannotation Ġtool Ġthat Ġshows Ġboth Ġthe Ġimages Ġand Ġtheir Ġassociated Ġdescriptions . ĠYou Ġcan Ġsimply Ġleaf Ġthrough Ġthe Ġimages Ġby Ġclicking Ġ` Next ' Ġor Ġ` Random ' Ġuntil Ġyou Ġfind Ġan Ġinteresting Ġpattern . </s> Eth nic ity / race </s> One Ġinteresting Ġpattern Ġis Ġthat Ġthe Ġethnicity / race Ġof Ġbabies Ġdoesn 't Ġseem Ġto Ġbe Ġmentioned Ġunless Ġthe Ġbaby Ġis Ġblack Ġor Ġas ian . ĠIn Ġother Ġwords : Ġwhite Ġseems Ġto Ġbe Ġthe Ġdefault , Ġand Ġothers Ġseem Ġto Ġbe Ġmarked . ĠHow Ġcan Ġwe Ġtell Ġwhether Ġor Ġnot Ġthe Ġdata Ġis Ġactually Ġbiased ? </s> We Ġdon 't Ġknow Ġwhether Ġor Ġnot Ġan Ġentity Ġbelongs Ġto Ġa Ġparticular Ġsocial Ġclass Ġ( in Ġthis Ġcase : Ġethnic Ġgroup ) Ġuntil Ġit Ġis Ġmarked Ġas Ġsuch . ĠBut Ġwe Ġcan Ġapproximate Ġthe Ġproportion Ġby Ġlooking Ġat Ġall Ġthe Ġimages Ġwhere Ġthe Ġannot ators Ġhave Ġused Ġa Ġmarker Ġ( in Ġthis Ġcase : Ġadject ives Ġlike Ġblack , Ġwhite , Ġas ian ), Ġand Ġfor Ġthose Ġimages Ġcount Ġhow Ġmany Ġdescriptions Ġ( out Ġof Ġfive ) Ġcontain Ġa Ġmarker . ĠThis Ġgives Ġus Ġan Ġupper Ġbound Ġthat Ġtells Ġus Ġhow Ġoften Ġethnicity Ġis Ġindicated Ġby Ġthe Ġannot ators . ĠNote Ġthat Ġthis Ġupper Ġbound Ġlies Ġsomewhere Ġbetween Ġ20 % Ġ( one Ġdescription ) Ġand Ġ100 % Ġ( 5 Ġdescriptions ). ĠFigure ĠT AB REF 22 Ġpresents Ġcount Ġdata Ġfor Ġthe Ġethnic Ġmarking Ġof Ġbabies . ĠIt Ġincludes Ġtwo Ġfalse Ġpositives Ġ( talking Ġabout Ġa Ġwhite Ġbaby Ġst roller Ġrather Ġthan Ġa Ġwhite Ġbaby ). ĠIn Ġthe ĠAsian Ġgroup Ġthere Ġis Ġan Ġadditional Ġcomplication : Ġsometimes Ġthe Ġmother Ġgets Ġmarked Ġrather Ġthan Ġthe Ġbaby . ĠE . g . ĠAn ĠAsian Ġwoman Ġholds Ġa Ġbaby Ġgirl . ĠI Ġhave Ġcounted Ġthese Ġoccurrences Ġas Ġwell . </s> The Ġnumbers Ġin ĠTable ĠT AB REF 22 Ġare Ġstriking : Ġthere Ġseems Ġto Ġbe Ġa Ġreal , Ġsystematic Ġdifference Ġin Ġethnicity Ġmarking Ġbetween Ġthe Ġgroups . ĠWe Ġcan Ġtake Ġone Ġstep Ġfurther Ġand Ġlook Ġat Ġall Ġthe Ġ6 97 Ġpictures Ġwith Ġthe Ġword Ġ` baby ' Ġin Ġit . ĠIf Ġthere Ġturn Ġout Ġto Ġbe Ġdisproportionately Ġmany Ġwhite Ġbabies , Ġthis Ġstrengthens Ġthe Ġconclusion Ġthat Ġthe Ġdataset Ġis Ġbiased . </s> I Ġhave Ġmanually Ġcategorized Ġeach Ġof Ġthe Ġbaby Ġimages . ĠThere Ġare Ġ504 Ġwhite , Ġ66 Ġas ian , Ġand Ġ36 Ġblack Ġbabies . Ġ73 Ġimages Ġdo Ġnot Ġcontain Ġa Ġbaby , Ġand Ġ18 Ġimages Ġdo Ġnot Ġfall Ġinto Ġany Ġof Ġthe Ġother Ġcategories . ĠWhile Ġthis Ġdoes Ġbring Ġdown Ġthe Ġaverage Ġnumber Ġof Ġtimes Ġeach Ġcategory Ġwas Ġmarked , Ġit Ġalso Ġincreases Ġthe Ġcontrast Ġbetween Ġwhite Ġbabies Ġ( who Ġget Ġmarked Ġin Ġless Ġthan Ġ1 % Ġof Ġthe Ġimages ) Ġand Ġas ian / black Ġbabies Ġ( who Ġget Ġmarked Ġmuch Ġmore Ġoften ). ĠA Ġnext Ġstep Ġwould Ġbe Ġto Ġsee Ġwhether Ġthese Ġobservations Ġalso Ġhold Ġfor Ġother Ġage Ġgroups , Ġi . e . Ġchildren Ġand Ġadults . ĠIN LINE FORM 0 </s> Other Ġmethods </s> It Ġmay Ġbe Ġdifficult Ġto Ġspot Ġpatterns Ġby Ġjust Ġlooking Ġat Ġa Ġcollection Ġof Ġimages . ĠAnother Ġmethod Ġis Ġto Ġtag Ġall Ġdescriptions Ġwith Ġpart - of - speech Ġinformation , Ġso Ġthat Ġit Ġbecomes Ġpossible Ġto Ġsee Ġe . g . Ġwhich Ġadject ives Ġare Ġmost Ġcommonly Ġused Ġfor Ġparticular Ġnoun s . ĠOne Ġmethod Ġreaders Ġmay Ġfind Ġparticularly Ġuseful Ġis Ġto Ġleverage Ġthe Ġstructure Ġof ĠFlickr 30 K ĠEnt ities ĠB IB REF 8 Ġ. ĠThis Ġdataset Ġen ric hes ĠFlickr 30 K Ġby Ġadding Ġcore ference Ġannotations , Ġi . e . Ġwhich Ġphrase Ġin Ġeach Ġdescription Ġrefers Ġto Ġthe Ġsame Ġentity Ġin Ġthe Ġcorresponding Ġimage . ĠI Ġhave Ġused Ġthis Ġdata Ġto Ġcreate Ġa Ġcore ference Ġgraph Ġby Ġlinking Ġall Ġphrases Ġthat Ġrefer Ġto Ġthe Ġsame Ġentity . ĠFollowing Ġthis , ĠI Ġapplied ĠLou v ain Ġclust ering ĠB IB REF 9 Ġto Ġthe Ġcore ference Ġgraph , Ġresulting Ġin Ġclusters Ġof Ġexpressions Ġthat Ġrefer Ġto Ġsimilar Ġentities . ĠLooking Ġat Ġthose Ġclusters Ġhelps Ġto Ġget Ġa Ġsense Ġof Ġthe Ġenormous Ġvariation Ġin Ġreferring Ġexpressions . ĠTo Ġget Ġan Ġidea Ġof Ġthe Ġrichness Ġof Ġthis Ġdata , Ġhere Ġis Ġa Ġsmall Ġsample Ġof Ġthe Ġphrases Ġused Ġto Ġdescribe Ġbe ards Ġ( cl uster Ġ268 ): Ġa Ġsc ruff y Ġbeard ; Ġa Ġthick Ġbeard ; Ġlarge Ġwhite Ġbeard ; Ġa Ġbubble Ġbeard ; Ġred Ġfacial Ġhair ; Ġa Ġbra ided Ġbeard ; Ġa Ġflaming Ġred Ġbeard . ĠIn Ġthis Ġcase , Ġ` red Ġfacial Ġhair ' Ġreally Ġstands Ġout Ġas Ġa Ġdescription ; Ġwhy Ġnot Ġchoose Ġthe Ġsimpler Ġ` beard ' Ġinstead ? </s> Discussion </s> In Ġthe Ġprevious Ġsection , ĠI Ġhave Ġoutlined Ġseveral Ġmethods Ġto Ġmanually Ġdetect Ġstereotypes , Ġbiases , Ġand Ġodd Ġphrases . ĠBecause Ġthere Ġare Ġmany Ġways Ġin Ġwhich Ġa Ġphrase Ġcan Ġbe Ġbiased , Ġit Ġis Ġdifficult Ġto Ġautomatically Ġdetect Ġbias Ġfrom Ġthe Ġdata . ĠSo Ġhow Ġshould Ġwe Ġdeal Ġwith Ġstereotype - driven Ġdescriptions ? </s> Conclusion </s> This Ġpaper Ġprovided Ġa Ġtax onomy Ġof Ġstereotype - driven Ġdescriptions Ġin Ġthe ĠFlickr 30 K Ġdataset . ĠI Ġhave Ġdivided Ġthese Ġdescriptions Ġinto Ġtwo Ġclasses : Ġlinguistic Ġbias Ġand Ġunw arranted Ġinf erences . ĠThe Ġformer Ġcorresponds Ġto Ġthe Ġannot ators ' Ġchoice Ġof Ġwords Ġwhen Ġconfronted Ġwith Ġan Ġimage Ġthat Ġmay Ġor Ġmay Ġnot Ġmatch Ġtheir Ġstereotypical Ġexpect ancies . ĠThe Ġlatter Ġcorresponds Ġto Ġthe Ġtendency Ġof Ġannot ators Ġto Ġgo Ġbeyond Ġwhat Ġthe Ġphysical Ġdata Ġcan Ġtell Ġus , Ġand Ġexpand Ġtheir Ġdescriptions Ġbased Ġon Ġtheir Ġpast Ġexperiences Ġand Ġknowledge Ġof Ġthe Ġworld . ĠA cknowled ging Ġthese Ġphenomena Ġis Ġimportant , Ġbecause Ġon Ġthe Ġone Ġhand Ġit Ġhelps Ġus Ġthink Ġabout Ġwhat Ġis Ġlearn able Ġfrom Ġthe Ġdata , Ġand Ġon Ġthe Ġother Ġhand Ġit Ġserves Ġas Ġa Ġwarning : Ġif Ġwe Ġtrain Ġand Ġevaluate Ġlanguage Ġmodels Ġon Ġthis Ġdata , Ġwe Ġare Ġeffectively Ġteaching Ġthem Ġto Ġbe Ġbiased . </s> I Ġhave Ġalso Ġlooked Ġat Ġmethods Ġto Ġdetect Ġstereotype - driven Ġdescriptions , Ġbut Ġdue Ġto Ġthe Ġrichness Ġof Ġlanguage Ġit Ġis Ġdifficult Ġto Ġfind Ġan Ġautomated Ġmeasure . ĠDepending Ġon Ġwhether Ġyour Ġgoal Ġis Ġproduction Ġor Ġinterpretation , Ġit Ġmay Ġeither Ġbe Ġuseful Ġto Ġsuppress Ġor Ġto Ġemphasize Ġbiases Ġin Ġhuman Ġlanguage . ĠFinally , ĠI Ġhave Ġdiscussed Ġstereotyp ing Ġbehavior Ġas Ġthe Ġaddition Ġof Ġa Ġcontextual Ġlayer Ġon Ġtop Ġof Ġa Ġmore Ġbasic Ġdescription . ĠThis Ġraises Ġthe Ġquestion Ġwhat Ġkind Ġof Ġdescriptions Ġwe Ġwould Ġlike Ġour Ġmodels Ġto Ġproduce . </s> Acknowled gments </s> Thanks Ġto ĠPie k ĠV oss en Ġand ĠAnt s ke ĠF ok kens Ġfor Ġdiscussion , Ġand Ġto ĠDesmond ĠElliott Ġand Ġan Ġanonymous Ġreviewer Ġfor Ġcomments Ġon Ġan Ġearlier Ġversion Ġof Ġthis Ġpaper . ĠThis Ġresearch Ġwas Ġsupported Ġby Ġthe ĠNetherlands ĠOrganization Ġfor ĠScientific ĠResearch Ġ( NW O ) Ġvia Ġthe ĠSp ino za - pri ze Ġawarded Ġto ĠPie k ĠV oss en Ġ( SP I Ġ30 - 673 , Ġ2014 - 2019 ).