sentences that were selected from the Wikipedia corpus provided by culotta2006integrating
EXTRACTIVE
seven of the originally defined relation types: political_affiliation, education, founder, wife/husband, job_title, nationality, and employer
EXTRACTIVE
We provide the first dataset of simultaneous eye movement and brain activity recordings to analyze and compare normal reading to task-specific reading during annotation. The Zurich Cognitive Language Processing Corpus (ZuCo) 2.0, including raw and preprocessed eye-tracking and electroencephalography (EEG) data of 18 subjects, as well as the recording and preprocessing scripts, is publicly available at https://osf.io/2urht/. It contains physiological data of each subject reading 739 English sentences from Wikipedia (see example in Figure FIGREF1). We want to highlight the re-use potential of this data. In addition to the psycholinguistic motivation, this corpus is especially tailored for training and evaluating machine learning algorithms for NLP purposes. We conduct a detailed technical validation of the data as proof of the quality of the recordings. During the recording session, the participants read 739 sentences that were selected from the Wikipedia corpus provided by culotta2006integrating. This corpus was chosen because it provides annotations of semantic relations. We included seven of the originally defined relation types: political_affiliation, education, founder, wife/husband, job_title, nationality, and employer. The sentences were chosen in the same length range as ZuCo 1.0, and with similar Flesch reading ease scores. The dataset statistics are shown in Table TABREF2.
During the recording session, the participants read 739 sentences that were selected from the Wikipedia corpus provided by culotta2006integrating. This corpus was chosen because it provides annotations of semantic relations. We included seven of the originally defined relation types: political_affiliation, education, founder, wife/husband, job_title, nationality, and employer. The sentences were chosen in the same length range as ZuCo 1.0, and with similar Flesch reading ease scores. The dataset statistics are shown in Table TABREF2.
1912.00903
Introduction </s> How Ġhumans Ġprocess Ġlanguage Ġhas Ġbecome Ġincreasingly Ġrelevant Ġin Ġnatural Ġlanguage Ġprocessing Ġsince Ġphysiological Ġdata Ġduring Ġlanguage Ġunderstanding Ġis Ġmore Ġaccessible Ġand Ġrecorded Ġwith Ġless Ġeffort . ĠIn Ġthis Ġwork , Ġwe Ġfocus Ġon Ġeye - tracking Ġand Ġelectro ence phal ography Ġ( EE G ) Ġrecordings Ġto Ġcapture Ġthe Ġreading Ġprocess . ĠOn Ġone Ġhand , Ġeye Ġmovement Ġdata Ġprovides Ġmillisec ond - acc urate Ġrecords Ġabout Ġwhere Ġhumans Ġlook Ġwhen Ġthey Ġare Ġreading , Ġand Ġis Ġhighly Ġcorrelated Ġwith Ġthe Ġcognitive Ġload Ġassociated Ġwith Ġdifferent Ġstages Ġof Ġtext Ġprocessing . ĠOn Ġthe Ġother Ġhand , ĠEEG Ġrecords Ġelectrical Ġbrain Ġactivity Ġacross Ġthe Ġscalp Ġand Ġis Ġa Ġdirect Ġmeasure Ġof Ġphysiological Ġprocesses , Ġincluding Ġlanguage Ġprocessing . ĠThe Ġcombination Ġof Ġboth Ġmeasurement Ġmethods Ġenables Ġus Ġto Ġstudy Ġthe Ġlanguage Ġunderstanding Ġprocess Ġin Ġa Ġmore Ġnatural Ġsetting , Ġwhere Ġparticipants Ġread Ġfull Ġsentences Ġat Ġa Ġtime , Ġin Ġtheir Ġown Ġspeed . ĠEye - tracking Ġthen Ġpermits Ġus Ġto Ġdefine Ġexact Ġword Ġboundaries Ġin Ġthe Ġtimeline Ġof Ġa Ġsubject Ġreading Ġa Ġsentence , Ġallowing Ġthe Ġextraction Ġof Ġbrain Ġactivity Ġsignals Ġfor Ġeach Ġword . </s> Human Ġcognitive Ġlanguage Ġprocessing Ġdata Ġis Ġimmensely Ġuseful Ġfor ĠN LP : ĠNot Ġonly Ġcan Ġit Ġbe Ġlever aged Ġto Ġimprove ĠN LP Ġapplications Ġ( e . g . Ġbar rett 2016 weak ly Ġfor Ġpart - of - speech Ġtagging Ġor Ġk ler ke 2016 impro ving Ġfor Ġsentence Ġcompression ), Ġbut Ġalso Ġto Ġevaluate Ġstate - of - the - art Ġmachine Ġlearning Ġsystems . ĠFor Ġexample , Ġh ollen stein 2019 c ogn ival Ġevaluate Ġword Ġembed d ings , Ġor Ġsch w artz 2019 inducing Ġfine - t une Ġlanguage Ġmodels Ġwith Ġbrain - relevant Ġbias . </s> Additionally , Ġthe Ġavailability Ġof Ġlabelled Ġdata Ġplays Ġa Ġcrucial Ġrole Ġin Ġall Ġsupervised Ġmachine Ġlearning Ġapplications . ĠPhys iological Ġdata Ġcan Ġbe Ġused Ġto Ġunderstand Ġand Ġimprove Ġthe Ġlab elling Ġprocess Ġ( e . g . Ġto kun aga 2017 eye ), Ġand , Ġfor Ġinstance , Ġto Ġbuild Ġcost Ġmodels Ġfor Ġactive Ġlearning Ġscenarios ĠB IB REF 0 . ĠIs Ġit Ġpossible Ġto Ġreplace Ġthis Ġexpensive Ġmanual Ġwork Ġwith Ġmodels Ġtrained Ġon Ġphysiological Ġactivity Ġdata Ġrecorded Ġfrom Ġhumans Ġwhile Ġreading ? ĠThat Ġis Ġto Ġsay , Ġcan Ġwe Ġfind Ġand Ġextract Ġrelevant Ġaspects Ġof Ġtext Ġunderstanding Ġand Ġannotation Ġdirectly Ġfrom Ġthe Ġsource , Ġi . e . Ġeye - tracking Ġand Ġbrain Ġactivity Ġsignals Ġduring Ġreading ? </s> Mot ivated Ġby Ġthese Ġquestions Ġand Ġour Ġpreviously Ġreleased Ġdataset , ĠZ u Co Ġ1 . 0 ĠB IB REF 1 , Ġwe Ġdeveloped Ġthis Ġnew Ġcorpus , Ġwhere Ġwe Ġspecifically Ġaim Ġto Ġcollect Ġrecordings Ġduring Ġnatural Ġreading Ġas Ġwell Ġas Ġduring Ġannotation . </s> We Ġprovide Ġthe Ġfirst Ġdataset Ġof Ġsimultaneous Ġeye Ġmovement Ġand Ġbrain Ġactivity Ġrecordings Ġto Ġanalyze Ġand Ġcompare Ġnormal Ġreading Ġto Ġtask - specific Ġreading Ġduring Ġannotation . ĠThe ĠZurich ĠCognitive ĠLanguage ĠProcessing ĠCorpus Ġ( Z u Co ) Ġ2 . 0 , Ġincluding Ġraw Ġand Ġpre process ed Ġeye - tracking Ġand Ġelectro ence phal ography Ġ( EE G ) Ġdata Ġof Ġ18 Ġsubjects , Ġas Ġwell Ġas Ġthe Ġrecording Ġand Ġpre processing Ġscripts , Ġis Ġpublicly Ġavailable Ġat Ġhttps :// os f . io / 2 ur ht /. ĠIt Ġcontains Ġphysiological Ġdata Ġof Ġeach Ġsubject Ġreading Ġ7 39 ĠEnglish Ġsentences Ġfrom ĠWikipedia Ġ( see Ġexample Ġin ĠFigure ĠFIG REF 1 ). ĠWe Ġwant Ġto Ġhighlight Ġthe Ġre - use Ġpotential Ġof Ġthis Ġdata . ĠIn Ġaddition Ġto Ġthe Ġpsych ol ingu istic Ġmotivation , Ġthis Ġcorpus Ġis Ġespecially Ġtailored Ġfor Ġtraining Ġand Ġevaluating Ġmachine Ġlearning Ġalgorithms Ġfor ĠN LP Ġpurposes . ĠWe Ġconduct Ġa Ġdetailed Ġtechnical Ġvalidation Ġof Ġthe Ġdata Ġas Ġproof Ġof Ġthe Ġquality Ġof Ġthe Ġrecordings . </s> Related ĠWork </s> Some Ġeye - tracking Ġcorpor a Ġof Ġnatural Ġreading Ġ( e . g . Ġthe ĠDund ee ĠB IB REF 2 , ĠProv o ĠB IB REF 3 Ġand ĠG EC O Ġcorpus ĠB IB REF 4 ), Ġand Ġa Ġfew ĠEEG Ġcorpor a Ġ( for Ġexample , Ġthe ĠU CL Ġcorpus ĠB IB REF 5 ) Ġare Ġavailable . ĠIt Ġhas Ġbeen Ġshown Ġthat Ġthis Ġtype Ġof Ġcognitive Ġprocessing Ġdata Ġis Ġuseful Ġfor Ġimproving Ġand Ġevaluating ĠN LP Ġmethods Ġ( e . g . Ġbar rett 2018 sequence , h ollen stein 2019 c ogn ival , Ġh ale 2018 finding ). ĠHowever , Ġbefore Ġthe ĠZurich ĠCognitive ĠLanguage ĠProcessing ĠCorpus Ġ( Z u Co Ġ1 . 0 ), Ġthere Ġwas Ġno Ġavailable Ġdata Ġfor Ġsimultaneous Ġeye - tracking Ġand ĠEEG Ġrecordings Ġof Ġnatural Ġreading . Ġdim igen 2011 core g istration Ġstudied Ġthe Ġlinguistic Ġeffects Ġof Ġeye Ġmovements Ġand ĠEEG Ġco - reg istration Ġin Ġnatural Ġreading Ġand Ġshowed Ġthat Ġthey Ġaccurately Ġrepresent Ġlex ical Ġprocessing . ĠMoreover , Ġthe Ġsimultaneous Ġrecordings Ġare Ġcrucial Ġto Ġextract Ġword - level Ġbrain Ġactivity Ġsignals . </s> While Ġthe Ġabove Ġmentioned Ġstudies Ġanalyze Ġand Ġleverage Ġnatural Ġreading , Ġsome ĠN LP Ġwork Ġhas Ġused Ġeye - tracking Ġduring Ġannotation Ġ( but , Ġas Ġof Ġyet , Ġnot ĠEEG Ġdata ). Ġmish ra 2016 p redict ing Ġand Ġj oshi 2014 me asuring Ġrecorded Ġeye - tracking Ġduring Ġbinary Ġsentiment Ġannotation Ġ( positive / negative ). ĠThis Ġdata Ġwas Ġused Ġto Ġdetermine Ġthe Ġannotation Ġcomplexity Ġof Ġthe Ġtext Ġpassages Ġbased Ġon Ġeye Ġmovement Ġmetrics Ġand Ġfor Ġsarc asm Ġdetection ĠB IB REF 6 . ĠMoreover , Ġeye - tracking Ġhas Ġbeen Ġused Ġto Ġanalyze Ġthe Ġword Ġsense Ġannotation Ġprocess Ġin ĠHindi ĠB IB REF 7 , Ġnamed Ġentity Ġannotation Ġin ĠJapanese ĠB IB REF 8 , Ġand Ġto Ġleverage Ġannot ator Ġgaze Ġbehaviour Ġfor Ġcore ference Ġresolution ĠB IB REF 9 . ĠFinally , Ġto man ek 2010 c ognitive Ġused Ġeye - tracking Ġdata Ġduring Ġentity Ġannotation Ġto Ġbuild Ġa Ġcost Ġmodel Ġfor Ġactive Ġlearning . ĠHowever , Ġuntil Ġnow Ġthere Ġis Ġno Ġavailable Ġdata Ġor Ġresearch Ġthat Ġanaly zes Ġthe Ġdifferences Ġin Ġthe Ġhuman Ġprocessing Ġof Ġnormal Ġreading Ġversus Ġannotation . </s> Related ĠWork Ġ: :: ĠZ u Co 1 . 0 </s> In Ġprevious Ġwork , Ġwe Ġrecorded Ġa Ġfirst Ġdataset Ġof Ġsimultaneous Ġeye - tracking Ġand ĠEEG Ġduring Ġnatural Ġreading ĠB IB REF 1 . ĠZ u Co Ġ1 . 0 Ġconsists Ġof Ġthree Ġreading Ġtasks , Ġtwo Ġof Ġwhich Ġcontain Ġvery Ġsimilar Ġreading Ġmaterial Ġand Ġexperiments Ġas Ġpresented Ġin Ġthe Ġcurrent Ġwork . ĠHowever , Ġthe Ġmain Ġdifference Ġand Ġreason Ġfor Ġrecording ĠZ u Co Ġ2 . 0 , Ġconsists Ġin Ġthe Ġexperiment Ġprocedure . ĠFor ĠZ u Co Ġ1 . 0 Ġthe Ġnormal Ġreading Ġand Ġtask - specific Ġreading Ġparad ig ms Ġwere Ġrecorded Ġin Ġdifferent Ġsessions Ġon Ġdifferent Ġdays . ĠTherefore , Ġthe Ġrecorded Ġdata Ġis Ġnot Ġappropriate Ġas Ġa Ġmeans Ġof Ġcomparison Ġbetween Ġnatural Ġreading Ġand Ġannotation , Ġsince Ġthe Ġdifferences Ġin Ġthe Ġbrain Ġactivity Ġdata Ġmight Ġresult Ġmostly Ġfrom Ġthe Ġdifferent Ġsessions Ġdue Ġto Ġthe Ġsensitivity Ġof ĠEEG . ĠThis , Ġand Ġextending Ġthe Ġdataset Ġwith Ġmore Ġsentences Ġand Ġmore Ġsubjects , Ġwere Ġthe Ġmain Ġfactors Ġfor Ġrecording Ġthe Ġcurrent Ġcorpus . ĠWe Ġpurposefully Ġmaintained Ġan Ġoverlap Ġof Ġsome Ġsentences Ġbetween Ġboth Ġdatasets Ġto Ġallow Ġadditional Ġanalyses Ġ( details Ġare Ġdescribed Ġin ĠSection ĠSEC REF 7 ). </s> Corp us ĠConstruction </s> In Ġthis Ġsection Ġwe Ġdescribe Ġthe Ġcontents Ġand Ġexperimental Ġdesign Ġof Ġthe ĠZ u Co Ġ2 . 0 Ġcorpus . </s> Corp us ĠConstruction Ġ: :: ĠParticipants </s> We Ġrecorded Ġdata Ġfrom Ġ19 Ġparticipants Ġand Ġdiscarded Ġthe Ġdata Ġof Ġone Ġof Ġthem Ġdue Ġto Ġtechnical Ġdifficulties Ġwith Ġthe Ġeye - tracking Ġcalibration . ĠHence , Ġwe Ġshare Ġthe Ġdata Ġof Ġ18 Ġparticipants . ĠAll Ġparticipants Ġare Ġhealthy Ġadults Ġ( mean Ġage Ġ= Ġ34 Ġ( SD = 8 . 3 ), Ġ10 Ġfemales ). ĠTheir Ġnative Ġlanguage Ġis ĠEnglish , Ġoriginating Ġfrom ĠAustralia , ĠCanada , ĠUK , ĠUSA Ġor ĠSouth ĠAfrica . ĠTwo Ġparticipants Ġare Ġleft - handed Ġand Ġthree Ġparticipants Ġwear Ġglasses Ġfor Ġreading . ĠDetails Ġon Ġsubject Ġdemographics Ġcan Ġbe Ġfound Ġin ĠTable ĠT AB REF 4 . ĠAll Ġparticipants Ġgave Ġwritten Ġconsent Ġfor Ġtheir Ġparticipation Ġand Ġthe Ġre - use Ġof Ġthe Ġdata Ġprior Ġto Ġthe Ġstart Ġof Ġthe Ġexperiments . ĠThe Ġstudy Ġwas Ġapproved Ġby Ġthe ĠEthics ĠCommission Ġof Ġthe ĠUniversity Ġof ĠZurich . </s> Corp us ĠConstruction Ġ: :: ĠReading Ġmaterials </s> During Ġthe Ġrecording Ġsession , Ġthe Ġparticipants Ġread Ġ7 39 Ġsentences Ġthat Ġwere Ġselected Ġfrom Ġthe ĠWikipedia Ġcorpus Ġprovided Ġby Ġcul otta 2006 integ rating . ĠThis Ġcorpus Ġwas Ġchosen Ġbecause Ġit Ġprovides Ġannotations Ġof Ġsemantic Ġrelations . ĠWe Ġincluded Ġseven Ġof Ġthe Ġoriginally Ġdefined Ġrelation Ġtypes : Ġpolitical _ aff iliation , Ġeducation , Ġfounder , Ġwife / husband , Ġjob _ title , Ġnationality , Ġand Ġemployer . ĠThe Ġsentences Ġwere Ġchosen Ġin Ġthe Ġsame Ġlength Ġrange Ġas ĠZ u Co Ġ1 . 0 , Ġand Ġwith Ġsimilar ĠF les ch Ġreading Ġease Ġscores . ĠThe Ġdataset Ġstatistics Ġare Ġshown Ġin ĠTable ĠT AB REF 2 . </s> Of Ġthe Ġ7 39 Ġsentences , Ġthe Ġparticipants Ġread Ġ349 Ġsentences Ġin Ġa Ġnormal Ġreading Ġparadigm , Ġand Ġ390 Ġsentences Ġin Ġa Ġtask - specific Ġreading Ġparadigm , Ġin Ġwhich Ġthey Ġhad Ġto Ġdetermine Ġwhether Ġa Ġcertain Ġrelation Ġtype Ġoccurred Ġin Ġthe Ġsentence Ġor Ġnot . ĠTable ĠT AB REF 3 Ġshows Ġthe Ġdistribution Ġof Ġthe Ġdifferent Ġrelation Ġtypes Ġin Ġthe Ġsentences Ġof Ġthe Ġtask - specific Ġannotation Ġparadigm . </s> Pur pose fully , Ġthere Ġare Ġ63 Ġdupl icates Ġbetween Ġthe Ġnormal Ġreading Ġand Ġthe Ġtask - specific Ġsentences Ġ( 8 % Ġof Ġall Ġsentences ). ĠThe Ġintention Ġof Ġthese Ġduplicate Ġsentences Ġis Ġto Ġprovide Ġa Ġset Ġof Ġsentences Ġread Ġtwice Ġby Ġall Ġparticipants Ġwith Ġa Ġdifferent Ġtask Ġin Ġmind . ĠHence , Ġthis Ġenables Ġthe Ġcomparison Ġof Ġeye - tracking Ġand Ġbrain Ġactivity Ġdata Ġwhen Ġreading Ġnormally Ġand Ġwhen Ġannot ating Ġspecific Ġrelations Ġ( see Ġexamples Ġin ĠSection ĠSEC REF 4 ). </s> Furthermore , Ġthere Ġis Ġalso Ġan Ġoverlap Ġin Ġthe Ġsentences Ġbetween ĠZ u Co Ġ1 . 0 Ġand ĠZ u Co Ġ2 . 0 . Ġ100 Ġnormal Ġreading Ġand Ġ85 Ġtask - specific Ġsentences Ġrecorded Ġfor Ġthis Ġdataset Ġwere Ġalready Ġrecorded Ġin ĠZ u Co Ġ1 . 0 . ĠThis Ġallows Ġfor Ġcomparisons Ġbetween Ġthe Ġdifferent Ġrecording Ġprocedures Ġ( i . e . Ġsession - specific Ġeffects ) Ġand Ġbetween Ġmore Ġparticipants Ġ( subject - specific Ġeffects ). </s> Corp us ĠConstruction Ġ: :: ĠExperimental Ġdesign </s> As Ġmentioned Ġabove , Ġwe Ġrecorded Ġtwo Ġdifferent Ġreading Ġtasks Ġfor Ġthe ĠZ u Co Ġ2 . 0 Ġdataset . ĠDuring Ġboth Ġtasks Ġthe Ġparticipants Ġwere Ġable Ġto Ġread Ġin Ġtheir Ġown Ġspeed , Ġusing Ġa Ġcontrol Ġpad Ġto Ġmove Ġto Ġthe Ġnext Ġsentence Ġand Ġto Ġanswer Ġthe Ġcontrol Ġquestions , Ġwhich Ġallowed Ġfor Ġnatural Ġreading . ĠSince Ġeach Ġsubject Ġreads Ġat Ġtheir Ġown Ġpersonal Ġpace , Ġthe Ġreading Ġspeed Ġbetween Ġvaries Ġbetween Ġsubjects . ĠTable ĠT AB REF 4 Ġshows Ġthe Ġaverage Ġreading Ġspeed Ġfor Ġeach Ġtask , Ġi . e . Ġthe Ġaverage Ġnumber Ġof Ġseconds Ġa Ġsubject Ġspends Ġper Ġsentence Ġbefore Ġswitching Ġto Ġthe Ġnext Ġone . </s> All Ġ7 39 Ġsentences Ġwere Ġrecorded Ġin Ġa Ġsingle Ġsession Ġfor Ġeach Ġparticipant . ĠThe Ġduration Ġof Ġthe Ġrecording Ġsessions Ġwas Ġbetween Ġ100 Ġand Ġ180 Ġminutes , Ġdepending Ġon Ġthe Ġtime Ġrequired Ġto Ġset Ġup Ġand Ġcalibr ate Ġthe Ġdevices , Ġand Ġthe Ġpersonal Ġreading Ġspeed Ġof Ġthe Ġparticipants . </s> We Ġrecorded Ġ14 Ġblocks Ġof Ġapprox . Ġ50 Ġsentences , Ġalternating Ġbetween Ġtasks : Ġ50 Ġsentences Ġof Ġnormal Ġreading , Ġfollowed Ġby Ġ50 Ġsentences Ġof Ġtask - specific Ġreading . ĠThe Ġorder Ġof Ġblocks Ġand Ġsentences Ġwithin Ġblocks Ġwas Ġidentical Ġfor Ġall Ġsubjects . ĠEach Ġsentence Ġblock Ġwas Ġpreceded Ġby Ġa Ġpractice Ġround Ġof Ġthree Ġsentences . </s> Corp us ĠConstruction Ġ: :: ĠExperimental Ġdesign Ġ: :: ĠNormal Ġreading Ġ( NR ) </s> In Ġthe Ġfirst Ġtask , Ġparticipants Ġwere Ġinstructed Ġto Ġread Ġthe Ġsentences Ġnaturally , Ġwithout Ġany Ġspecific Ġtask Ġother Ġthan Ġcomprehension . ĠParticipants Ġwere Ġtold Ġto Ġread Ġthe Ġsentences Ġnormally Ġwithout Ġany Ġspecial Ġinstructions . ĠFigure ĠFIG REF 8 Ġ( left ) Ġshows Ġan Ġexample Ġsentence Ġas Ġit Ġwas Ġdepicted Ġon Ġthe Ġscreen Ġduring Ġrecording . ĠAs Ġshown Ġin ĠFigure ĠFIG REF 8 Ġ( middle ), Ġthe Ġcontrol Ġcondition Ġfor Ġthis Ġtask Ġconsisted Ġof Ġsingle - choice Ġquestions Ġabout Ġthe Ġcontent Ġof Ġthe Ġprevious Ġsentence . Ġ12 % Ġof Ġrandomly Ġselected Ġsentences Ġwere Ġfollowed Ġby Ġsuch Ġa Ġcomprehension Ġquestion Ġwith Ġthree Ġanswer Ġoptions Ġon Ġa Ġseparate Ġscreen . </s> Corp us ĠConstruction Ġ: :: ĠExperimental Ġdesign Ġ: :: ĠTask - specific Ġreading Ġ( TS R ) </s> In Ġthe Ġsecond Ġtask , Ġthe Ġparticipants Ġwere Ġinstructed Ġto Ġsearch Ġfor Ġa Ġspecific Ġrelation Ġin Ġeach Ġsentence Ġthey Ġread . ĠInstead Ġof Ġcomprehension Ġquestions , Ġthe Ġparticipants Ġhad Ġto Ġdecide Ġfor Ġeach Ġsentence Ġwhether Ġit Ġcontains Ġthe Ġrelation Ġor Ġnot , Ġi . e . Ġthey Ġwere Ġactively Ġannot ating Ġeach Ġsentence . ĠFigure ĠFIG REF 8 Ġ( right ) Ġshows Ġan Ġexample Ġscreen Ġfor Ġthis Ġtask . Ġ17 % Ġof Ġthe Ġsentences Ġdid Ġnot Ġinclude Ġthe Ġrelation Ġtype Ġand Ġwere Ġused Ġas Ġcontrol Ġconditions . ĠAll Ġsentences Ġwithin Ġone Ġblock Ġinvolved Ġthe Ġsame Ġrelation Ġtype . ĠThe Ġblocks Ġstarted Ġwith Ġa Ġpractice Ġround , Ġwhich Ġdescribed Ġthe Ġrelation Ġand Ġwas Ġfollowed Ġby Ġthree Ġsample Ġsentences , Ġso Ġthat Ġthe Ġparticipants Ġwould Ġbe Ġfamiliar Ġwith Ġthe Ġrespective Ġrelation Ġtype . </s> Corp us ĠConstruction Ġ: :: ĠL ingu istic Ġassessment </s> As Ġa Ġlinguistic Ġassessment , Ġthe Ġvocabulary Ġand Ġlanguage Ġproficiency Ġof Ġthe Ġparticipants Ġwas Ġtested Ġwith Ġthe ĠLex T ALE Ġtest Ġ( Lex ical ĠTest Ġfor ĠAdvanced ĠLear ners Ġof ĠEnglish , Ġle m h ofer 2012 introdu cing ). ĠThis Ġis Ġan Ġun speed ed Ġlex ical Ġdecision Ġtask Ġdesigned Ġfor Ġintermediate Ġto Ġhighly Ġproficient Ġlanguage Ġusers . ĠThe Ġaverage ĠLex T ALE Ġscore Ġover Ġall Ġparticipants Ġwas Ġ88 . 54 %. ĠMoreover , Ġwe Ġalso Ġreport Ġthe Ġscores Ġthe Ġparticipants Ġachieved Ġwith Ġtheir Ġanswers Ġto Ġthe Ġreading Ġcomprehension Ġcontrol Ġquestions Ġand Ġtheir Ġrelation Ġannotations . ĠThe Ġdetailed Ġscores Ġfor Ġall Ġparticipants Ġare Ġalso Ġpresented Ġin ĠTable ĠT AB REF 4 . </s> Corp us ĠConstruction Ġ: :: ĠData Ġacquisition </s> Data Ġacquisition Ġtook Ġplace Ġin Ġa Ġsound - atten uated Ġand Ġdark Ġexperiment Ġroom . ĠParticipants Ġwere Ġseated Ġat Ġa Ġdistance Ġof Ġ68 cm Ġfrom Ġa Ġ24 - inch Ġmonitor Ġwith Ġa Ġresolution Ġof Ġ800 x 600 Ġpixels . ĠA Ġstable Ġhead Ġposition Ġwas Ġensured Ġvia Ġa Ġchin Ġrest . ĠParticipants Ġwere Ġinstructed Ġto Ġstay Ġas Ġstill Ġas Ġpossible Ġduring Ġthe Ġtasks Ġto Ġavoid Ġmotor ĠEEG Ġartifacts . ĠParticipants Ġwere Ġalso Ġoffered Ġsnacks Ġand Ġwater Ġduring Ġthe Ġbreaks Ġand Ġwere Ġencouraged Ġto Ġrest . ĠAll Ġsentences Ġwere Ġpresented Ġat Ġthe Ġsame Ġposition Ġon Ġthe Ġscreen Ġand Ġcould Ġspan Ġmultiple Ġlines . ĠThe Ġsentences Ġwere Ġpresented Ġin Ġblack Ġon Ġa Ġlight Ġgrey Ġbackground Ġwith Ġfont Ġsize Ġ20 - point ĠA rial , Ġresulting Ġin Ġa Ġletter Ġheight Ġof Ġ0 . 8 Ġmm . ĠThe Ġexperiment Ġwas Ġprogrammed Ġin ĠMAT LAB Ġ2016 b ĠB IB REF 10 , Ġusing ĠPsych Tool box ĠB IB REF 11 . ĠParticipants Ġcompleted Ġthe Ġtasks Ġsitting Ġalone Ġin Ġthe Ġroom , Ġwhile Ġtwo Ġresearch Ġassistants Ġwere Ġmonitoring Ġtheir Ġprogress Ġin Ġthe Ġadjoining Ġroom . ĠAll Ġrecording Ġscripts Ġincluding Ġdetailed Ġparticipant Ġinstructions Ġare Ġavailable Ġalongside Ġthe Ġdata . </s> Corp us ĠConstruction Ġ: :: ĠData Ġacquisition Ġ: :: ĠEye - tracking Ġacquisition </s> Eye Ġposition Ġand Ġpupil Ġsize Ġwere Ġrecorded Ġwith Ġan Ġinfrared Ġvideo - based Ġeye Ġtracker Ġ( Eye Link Ġ1000 ĠPlus , ĠSR ĠResearch ) Ġat Ġa Ġsampling Ġrate Ġof Ġ500 ĠHz . ĠThe Ġeye Ġtracker Ġwas Ġcalibrated Ġwith Ġa Ġ9 - point Ġgrid Ġat Ġthe Ġbeginning Ġof Ġthe Ġsession Ġand Ġre - valid ated Ġbefore Ġeach Ġblock Ġof Ġsentences . </s> Corp us ĠConstruction Ġ: :: ĠData Ġacquisition Ġ: :: ĠEEG Ġacquisition </s> High - density ĠEEG Ġdata Ġwere Ġrecorded Ġat Ġa Ġsampling Ġrate Ġof Ġ500 ĠHz Ġwith Ġa Ġband pass Ġof Ġ0 . 1 Ġto Ġ100 ĠHz , Ġusing Ġa Ġ128 - channel ĠEEG ĠGe odes ic ĠHydro cel Ġsystem Ġ( Elect rical ĠGe odes ics ). ĠThe Ġrecording Ġreference Ġwas Ġset Ġat Ġelectrode ĠC z . ĠThe Ġhead Ġcircumference Ġof Ġeach Ġparticipant Ġwas Ġmeasured Ġto Ġselect Ġan Ġappropriately Ġsized ĠEEG Ġnet . ĠTo Ġensure Ġgood Ġcontact , Ġthe Ġimpedance Ġof Ġeach Ġelectrode Ġwas Ġchecked Ġprior Ġto Ġrecording , Ġand Ġwas Ġkept Ġbelow Ġ40 Ġk Oh m . ĠElectro de Ġimpedance Ġlevels Ġwere Ġchecked Ġafter Ġevery Ġthird Ġblock Ġof Ġ50 Ġsentences Ġ( app rox . Ġevery Ġ30 Ġmins ) Ġand Ġreduced Ġif Ġnecessary . </s> Corp us ĠConstruction Ġ: :: ĠPre processing Ġand Ġfeature Ġextraction Ġ: :: ĠEye - tracking </s> The Ġeye - tracking Ġdata Ġconsists Ġof Ġ( x , y ) Ġgaze Ġlocation Ġentries Ġfor Ġall Ġindividual Ġfix ations Ġ( Figure ĠFIG REF 1 b ). ĠCoord inates Ġwere Ġgiven Ġin Ġpixels Ġwith Ġrespect Ġto Ġthe Ġmonitor Ġcoordinates Ġ( the Ġupper Ġleft Ġcorner Ġof Ġthe Ġscreen Ġwas Ġ( 0 , 0 ) Ġand Ġdown / right Ġwas Ġpositive ). ĠWe Ġprovide Ġthis Ġraw Ġdata Ġas Ġwell Ġas Ġvarious Ġengineered Ġeye - tracking Ġfeatures . ĠFor Ġthis Ġfeature Ġextraction Ġonly Ġfix ations Ġwithin Ġthe Ġboundaries Ġof Ġeach Ġdisplayed Ġword Ġwere Ġextracted . ĠData Ġpoints Ġdistinctly Ġnot Ġassociated Ġwith Ġreading Ġ( minimum Ġdistance Ġof Ġ50 Ġpixels Ġto Ġthe Ġtext ) Ġwere Ġexcluded . ĠAdditionally , Ġfix ations Ġshorter Ġthan Ġ100 Ġms Ġwere Ġexcluded Ġfrom Ġthe Ġanalyses , Ġbecause Ġthese Ġare Ġunlikely Ġto Ġreflect Ġfix ations Ġrelevant Ġfor Ġreading ĠB IB REF 12 . ĠOn Ġthe Ġbasis Ġof Ġthe ĠG EC O Ġand ĠZ u Co Ġ1 . 0 Ġcorpor a , Ġwe Ġextracted Ġthe Ġfollowing Ġfeatures : Ġ( i ) Ġgaze Ġduration Ġ( GD ), Ġthe Ġsum Ġof Ġall Ġfix ations Ġon Ġthe Ġcurrent Ġword Ġin Ġthe Ġfirst - pass Ġreading Ġbefore Ġthe Ġeye Ġmoves Ġout Ġof Ġthe Ġword ; Ġ( ii ) Ġtotal Ġreading Ġtime Ġ( TR T ), Ġthe Ġsum Ġof Ġall Ġfixation Ġd urations Ġon Ġthe Ġcurrent Ġword , Ġincluding Ġregress ions ; Ġ( iii ) Ġfirst Ġfixation Ġduration Ġ( FF D ), Ġthe Ġduration Ġof Ġthe Ġfirst Ġfixation Ġon Ġthe Ġprevailing Ġword ; Ġ( iv ) Ġsingle Ġfixation Ġduration Ġ( SF D ), Ġthe Ġduration Ġof Ġthe Ġfirst Ġand Ġonly Ġfixation Ġon Ġthe Ġcurrent Ġword ; Ġand Ġ( v ) Ġgo - past Ġtime Ġ( G PT ), Ġthe Ġsum Ġof Ġall Ġfix ations Ġprior Ġto Ġprogressing Ġto Ġthe Ġright Ġof Ġthe Ġcurrent Ġword , Ġincluding Ġregress ions Ġto Ġprevious Ġwords Ġthat Ġoriginated Ġfrom Ġthe Ġcurrent Ġword . ĠFor Ġeach Ġof Ġthese Ġeye - tracking Ġfeatures Ġwe Ġadditionally Ġcomputed Ġthe Ġpupil Ġsize . ĠFurthermore , Ġwe Ġextracted Ġthe Ġnumber Ġof Ġfix ations Ġand Ġmean Ġpupil Ġsize Ġfor Ġeach Ġword Ġand Ġsentence . </s> Corp us ĠConstruction Ġ: :: ĠPre processing Ġand Ġfeature Ġextraction Ġ: :: ĠEEG </s> The ĠEEG Ġdata Ġshared Ġin Ġthis Ġproject Ġare Ġavailable Ġas Ġraw Ġdata , Ġbut Ġalso Ġpre process ed Ġwith ĠAutom agic Ġ( version Ġ1 . 4 . 6 , Ġped ron i 2019 aut om agic ), Ġa Ġtool Ġfor Ġautomatic ĠEEG Ġdata Ġcleaning Ġand Ġvalidation . Ġ105 ĠEEG Ġchannels Ġ( i . e . Ġelectrodes ) Ġwere Ġused Ġfrom Ġthe Ġscalp Ġrecordings . Ġ9 ĠE OG Ġchannels Ġwere Ġused Ġfor Ġartifact Ġremoval Ġand Ġadditional Ġ14 Ġchannels Ġlying Ġmainly Ġon Ġthe Ġneck Ġand Ġface Ġwere Ġdiscarded Ġbefore Ġdata Ġanalysis . ĠBad Ġchannels Ġwere Ġidentified Ġand Ġinterpol ated . ĠWe Ġused Ġthe ĠMultiple ĠArtifact ĠRe jection ĠAl gorithm Ġ( M ARA ), Ġa Ġsupervised Ġmachine Ġlearning Ġalgorithm Ġthat Ġevaluates ĠI CA Ġcomponents , Ġfor Ġautomatic Ġartifact Ġrejection . ĠMAR A Ġhas Ġbeen Ġtrained Ġon Ġmanual Ġcomponent Ġclass ifications , Ġand Ġthus Ġcaptures Ġa Ġwide Ġrange Ġof Ġartifacts . ĠMAR A Ġis Ġespecially Ġeffective Ġat Ġdetecting Ġand Ġremoving Ġeye Ġand Ġmuscle Ġartifact Ġcomponents . ĠThe Ġeffect Ġof Ġthis Ġpre processing Ġcan Ġbe Ġseen Ġin ĠFigure ĠFIG REF 1 d . </s> After Ġpre processing , Ġwe Ġsynchronized Ġthe ĠEEG Ġand Ġeye - tracking Ġdata Ġto Ġenable ĠEEG Ġanalyses Ġtime - locked Ġto Ġthe Ġon sets Ġof Ġfix ations . ĠTo Ġcompute Ġoscill atory Ġpower Ġmeasures , Ġwe Ġband - pass Ġfiltered Ġthe Ġcontinuous ĠEEG Ġsignals Ġacross Ġan Ġentire Ġreading Ġtask Ġfor Ġfive Ġdifferent Ġfrequency Ġbands Ġresulting Ġin Ġa Ġtime - series Ġfor Ġeach Ġfrequency Ġband . ĠThe Ġindependent Ġfrequency Ġbands Ġwere Ġdetermined Ġas Ġfollows : Ġthe ta $ _ 1 $ Ġ( 4 âĢĵ 6 ĠHz ), Ġthe ta $ _ 2 $ Ġ( 6 . 5 âĢĵ 8 ĠHz ), Ġalpha $ _ 1 $ Ġ( 8 . 5 âĢĵ 10 ĠHz ), Ġalpha $ _ 2 $ Ġ( 10 . 5 âĢĵ 13 ĠHz ), Ġbeta $ _ 1 $ Ġ( 13 . 5 âĢĵ 18 ĠHz ), Ġbeta $ _ 2 $ Ġ( 18 . 5 âĢĵ 30 ĠHz ), Ġgamma $ _ 1 $ Ġ( 30 . 5 âĢĵ 40 ĠHz ), Ġand Ġgamma $ _ 2 $ Ġ( 40 âĢĵ 49 . 5 ĠHz ). ĠWe Ġthen Ġapplied Ġa ĠHilbert Ġtransformation Ġto Ġeach Ġof Ġthese Ġtime - series . ĠWe Ġspecifically Ġchose Ġthe ĠHilbert Ġtransformation Ġto Ġmaintain Ġthe Ġtemporal Ġinformation Ġof Ġthe Ġamplitude Ġof Ġthe Ġfrequency Ġbands , Ġto Ġenable Ġthe Ġpower Ġof Ġthe Ġdifferent Ġfrequencies Ġfor Ġtime Ġsegments Ġdefined Ġthrough Ġthe Ġfix ations Ġfrom Ġthe Ġeye - tracking Ġrecording . ĠThus , Ġfor Ġeach Ġeye - tracking Ġfeature Ġwe Ġcomputed Ġthe Ġcorresponding ĠEEG Ġfeature Ġin Ġeach Ġfrequency Ġband . ĠFurthermore , Ġwe Ġextracted Ġsentence - level ĠEEG Ġfeatures Ġby Ġcalculating Ġthe Ġpower Ġin Ġeach Ġfrequency Ġband , Ġand Ġadditionally , Ġthe Ġdifference Ġof Ġthe Ġpower Ġspect ra Ġbetween Ġfrontal Ġleft Ġand Ġright Ġhom ologue Ġelectrodes Ġpairs . ĠFor Ġeach Ġeye - tracking Ġbased ĠEEG Ġfeature , Ġall Ġchannels Ġwere Ġsubject Ġto Ġan Ġartifact Ġrejection Ġcriterion Ġof Ġ$ 90 \ mu ĠV $ Ġto Ġexclude Ġtrials Ġwith Ġtransient Ġnoise . </s> Data ĠVal idation </s> The Ġaim Ġof Ġthe Ġtechnical Ġvalidation Ġof Ġthe Ġdata Ġis Ġto Ġguarantee Ġgood Ġrecording Ġquality Ġand Ġto Ġreplicate Ġfindings Ġof Ġprevious Ġstudies Ġinvestigating Ġco - reg istration Ġof ĠEEG Ġand Ġeye Ġmovement Ġdata Ġduring Ġnatural Ġreading Ġtasks Ġ( e . g . Ġdim igen 2011 core g istration ). ĠWe Ġalso Ġcompare Ġthe Ġresults Ġto ĠZ u Co Ġ1 . 0 ĠB IB REF 1 , Ġwhich Ġallows Ġa Ġmore Ġdirect Ġcomparison Ġdue Ġto Ġthe Ġanalogous Ġrecording Ġprocedure . </s> Data ĠVal idation Ġ: :: ĠEye - tracking </s> We Ġvalidated Ġthe Ġrecorded Ġeye - tracking Ġdata Ġby Ġanalyzing Ġthe Ġfix ations Ġmade Ġby Ġall Ġsubjects Ġthrough Ġtheir Ġreading Ġspeed Ġand Ġomission Ġrate Ġon Ġsentence Ġlevel . ĠThe Ġomission Ġrate Ġis Ġdefined Ġas Ġthe Ġpercentage Ġof Ġwords Ġthat Ġis Ġnot Ġfix ated Ġin Ġa Ġsentence . ĠFigure ĠFIG REF 10 Ġ( middle ) Ġshows Ġthe Ġmean Ġreading Ġspeed Ġover Ġall Ġsubjects , Ġmeasured Ġin Ġseconds Ġper Ġsentence Ġand ĠFigure ĠFIG REF 10 Ġ( right ) Ġshows Ġthe Ġmean Ġomission Ġrates Ġaggreg ated Ġover Ġall Ġsubjects Ġfor Ġeach Ġtask . ĠClearly , Ġthe Ġparticipants Ġmade Ġless Ġfix ations Ġduring Ġthe Ġtask - specific Ġreading , Ġwhich Ġlead Ġto Ġfaster Ġreading Ġspeed . </s> Moreover , Ġwe Ġcorrobor ated Ġthese Ġsentence - level Ġmetrics Ġby Ġvisual izing Ġthe Ġskipping Ġproportion Ġon Ġword Ġlevel Ġ( Figure ĠFIG REF 13 ). ĠThe Ġskipping Ġproportion Ġis Ġthe Ġaverage Ġrate Ġof Ġwords Ġbeing Ġskipped Ġ( i . e . Ġnot Ġbeing Ġfix ated ) Ġin Ġa Ġsentence . ĠAs Ġexpected , Ġthis Ġalso Ġincreases Ġin Ġthe Ġtask - specific Ġreading . </s> Although Ġthe Ġreading Ġmaterial Ġis Ġfrom Ġthe Ġsame Ġsource Ġand Ġof Ġthe Ġsame Ġlength Ġrange Ġ( see ĠFigure ĠFIG REF 10 Ġ( left )), Ġin Ġthe Ġfirst Ġtask Ġ( NR ) Ġpassive Ġreading Ġwas Ġrecorded , Ġwhile Ġin Ġthe Ġsecond Ġtask Ġ( TS R ) Ġthe Ġsubjects Ġhad Ġto Ġannot ate Ġa Ġspecific Ġrelation Ġtype Ġin Ġeach Ġsentence . ĠThus , Ġthe Ġtask - specific Ġannotation Ġreading Ġlead Ġto Ġshorter Ġpasses Ġbecause Ġthe Ġgoal Ġwas Ġmerely Ġto Ġrecognize Ġa Ġrelation Ġin Ġthe Ġtext , Ġbut Ġnot Ġnecessarily Ġto Ġprocess Ġthe Ġevery Ġword Ġin Ġeach Ġsentence . ĠThis Ġdistinct Ġreading Ġbehavior Ġis Ġshown Ġin ĠFigure ĠFIG REF 15 , Ġwhere Ġfix ations Ġoccur Ġuntil Ġthe Ġend Ġof Ġthe Ġsentence Ġduring Ġnormal Ġreading , Ġwhile Ġduring Ġtask - specific Ġreading Ġthe Ġfix ations Ġstop Ġafter Ġthe Ġdecisive Ġwords Ġto Ġdetect Ġa Ġgiven Ġrelation Ġtype . ĠFinally , Ġwe Ġalso Ġanalyzed Ġthe Ġaverage Ġreading Ġtimes Ġfor Ġeach Ġof Ġthe Ġextracted Ġeye - tracking Ġfeatures . ĠThe Ġmeans Ġand Ġdistributions Ġfor Ġboth Ġtasks Ġare Ġshown Ġin ĠFigure ĠFIG REF 21 . ĠThese Ġresults Ġare Ġin Ġline Ġwith Ġthe Ġrecorded Ġdata Ġin ĠZ u Co Ġ1 . 0 , Ġas Ġwell Ġas Ġwith Ġthe Ġfeatures Ġextracted Ġin Ġthe ĠG EC O Ġcorpus ĠB IB REF 4 . </s> Data ĠVal idation Ġ: :: ĠEEG </s> As Ġa Ġfirst Ġvalidation Ġstep , Ġwe Ġextracted Ġfixation - related Ġpotential s Ġ( FR Ps ), Ġwhere Ġthe ĠEEG Ġsignal Ġduring Ġall Ġfix ations Ġof Ġone Ġtask Ġare Ġaveraged . ĠFigure ĠFIG REF 24 Ġshows Ġthe Ġtime - series Ġof Ġthe Ġresulting ĠFR Ps Ġfor Ġtwo Ġelectrodes Ġ( PO 8 Ġand ĠC z ), Ġas Ġwell Ġas Ġtop ographies Ġof Ġthe Ġvoltage Ġdistributions Ġacross Ġthe Ġscalp Ġat Ġselected Ġpoints Ġin Ġtime . ĠThe Ġfive Ġcomponents Ġ( for Ġwhich Ġthe Ġscalp Ġtop ographies Ġare Ġplotted ) Ġare Ġhighly Ġsimilar Ġin Ġthe Ġtime - course Ġof Ġthe Ġchosen Ġelectrodes Ġto Ġdim igen 2011 core g istration Ġas Ġwell Ġas Ġto ĠZ u Co Ġ1 . 0 . </s> Moreover , Ġthese Ġprevious Ġstudies Ġwere Ġable Ġto Ġshow Ġan Ġeffect Ġof Ġfixation Ġduration Ġon Ġthe Ġresulting ĠFR Ps . ĠTo Ġshow Ġthis Ġdependency Ġwe Ġfollowed Ġtwo Ġapproaches . ĠFirst , Ġfor Ġeach Ġreading Ġtask , Ġall Ġsingle - trial ĠFR Ps Ġwere Ġordered Ġby Ġfixation Ġduration Ġand Ġa Ġvertical Ġsliding Ġtime - window Ġwas Ġused Ġto Ġsmooth Ġthe Ġdata ĠB IB REF 13 . ĠFigure ĠFIG REF 25 Ġ( bottom ) Ġshows Ġthe Ġresulting Ġplots . ĠIn Ġline Ġwith Ġthis Ġprevious Ġwork , Ġa Ġfirst Ġposit ivation Ġcan Ġbe Ġidentified Ġat Ġ100 Ġms Ġpost - fix ation Ġonset . ĠA Ġsecond Ġpositive Ġpeak Ġis Ġlocated Ġdependent Ġon Ġthe Ġduration Ġof Ġthe Ġfixation , Ġwhich Ġcan Ġbe Ġexplained Ġby Ġthe Ġtime - j ittered Ġsucceeding Ġfixation . ĠThe Ġsecond Ġapproach Ġis Ġbased Ġon Ġhe nd erson 2013 co Ġin Ġwhich Ġsingle Ġtrial ĠEEG Ġsegments Ġare Ġclustered Ġby Ġthe Ġduration Ġof Ġthe Ġcurrent Ġfixation . ĠAs Ġshown Ġin ĠFigure ĠFIG REF 25 Ġ( top ), Ġwe Ġchose Ġfour Ġclusters Ġand Ġaveraged Ġthe Ġdata Ġwithin Ġeach Ġcluster Ġto Ġfour Ġdistinct ĠFR Ps , Ġdepending Ġon Ġthe Ġfixation Ġduration . ĠAgain , Ġthe Ġsame Ġposit ivation Ġpeaks Ġbecome Ġapparent . ĠBoth Ġfindings Ġare Ġconsistent Ġwith Ġthe Ġprevious Ġwork Ġmentioned Ġand Ġwith Ġour Ġfindings Ġfrom ĠZ u Co Ġ1 . 0 . </s> Conclusion </s> We Ġpresented Ġa Ġnew , Ġfreely Ġavailable Ġcorpus Ġof Ġeye Ġmovement Ġand Ġelectrical Ġbrain Ġactivity Ġrecordings Ġduring Ġnatural Ġreading Ġas Ġwell Ġas Ġduring Ġannotation . ĠThis Ġis Ġthe Ġfirst Ġdataset Ġthat Ġallows Ġfor Ġthe Ġcomparison Ġbetween Ġthese Ġtwo Ġreading Ġparad ig ms . ĠWe Ġdescribed Ġthe Ġmaterials Ġand Ġexperiment Ġdesign Ġin Ġdetail Ġand Ġconducted Ġan Ġextensive Ġvalidation Ġto Ġensure Ġthe Ġquality Ġof Ġthe Ġrecorded Ġdata . ĠSince Ġthis Ġcorpus Ġis Ġtailored Ġto Ġcognitive ly - inspired ĠN LP , Ġthe Ġapplications Ġand Ġre - use Ġpotential s Ġof Ġthis Ġdata Ġare Ġextensive . ĠThe Ġprovided Ġword - level Ġand Ġsentence - level Ġeye - tracking Ġand ĠEEG Ġfeatures Ġcan Ġbe Ġused Ġto Ġimprove Ġand Ġevaluate ĠN LP Ġand Ġmachine Ġlearning Ġmethods , Ġfor Ġinstance , Ġto Ġevaluate Ġlinguistic Ġphenomena Ġin Ġneural Ġmodels Ġvia Ġpsych ol ingu istic Ġdata . ĠIn Ġaddition , Ġbecause Ġthe Ġsentences Ġcontains Ġsemantic Ġrelation Ġlabels Ġand Ġthe Ġannotations Ġof Ġall Ġparticipants , Ġit Ġcan Ġalso Ġbe Ġwidely Ġused Ġfor Ġrelation Ġextraction Ġand Ġclassification . ĠFinally , Ġthe Ġtwo Ġcarefully Ġconstructed Ġreading Ġparad ig ms Ġallow Ġfor Ġthe Ġcomparison Ġbetween Ġnormal Ġreading Ġand Ġreading Ġduring Ġannotation , Ġwhich Ġcan Ġbe Ġrelevant Ġto Ġimprove Ġthe Ġmanual Ġlab elling Ġprocess Ġas Ġwell Ġas Ġthe Ġquality Ġof Ġthe Ġannotations Ġfor Ġsupervised Ġmachine Ġlearning .
Introduction How humans process language has become increasingly relevant in natural language processing since physiological data during language understanding is more accessible and recorded with less effort. In this work, we focus on eye-tracking and electroencephalography (EEG) recordings to capture the reading process. On one hand, eye movement data provides millisecond-accurate records about where humans look when they are reading, and is highly correlated with the cognitive load associated with different stages of text processing. On the other hand, EEG records electrical brain activity across the scalp and is a direct measure of physiological processes, including language processing. The combination of both measurement methods enables us to study the language understanding process in a more natural setting, where participants read full sentences at a time, in their own speed. Eye-tracking then permits us to define exact word boundaries in the timeline of a subject reading a sentence, allowing the extraction of brain activity signals for each word. Human cognitive language processing data is immensely useful for NLP: Not only can it be leveraged to improve NLP applications (e.g. barrett2016weakly for part-of-speech tagging or klerke2016improving for sentence compression), but also to evaluate state-of-the-art machine learning systems. For example, hollenstein2019cognival evaluate word embeddings, or schwartz2019inducing fine-tune language models with brain-relevant bias. Additionally, the availability of labelled data plays a crucial role in all supervised machine learning applications. Physiological data can be used to understand and improve the labelling process (e.g. tokunaga2017eye), and, for instance, to build cost models for active learning scenarios BIBREF0. Is it possible to replace this expensive manual work with models trained on physiological activity data recorded from humans while reading? That is to say, can we find and extract relevant aspects of text understanding and annotation directly from the source, i.e. eye-tracking and brain activity signals during reading? Motivated by these questions and our previously released dataset, ZuCo 1.0 BIBREF1, we developed this new corpus, where we specifically aim to collect recordings during natural reading as well as during annotation. We provide the first dataset of simultaneous eye movement and brain activity recordings to analyze and compare normal reading to task-specific reading during annotation. The Zurich Cognitive Language Processing Corpus (ZuCo) 2.0, including raw and preprocessed eye-tracking and electroencephalography (EEG) data of 18 subjects, as well as the recording and preprocessing scripts, is publicly available at https://osf.io/2urht/. It contains physiological data of each subject reading 739 English sentences from Wikipedia (see example in Figure FIGREF1). We want to highlight the re-use potential of this data. In addition to the psycholinguistic motivation, this corpus is especially tailored for training and evaluating machine learning algorithms for NLP purposes. We conduct a detailed technical validation of the data as proof of the quality of the recordings. Related Work Some eye-tracking corpora of natural reading (e.g. the Dundee BIBREF2, Provo BIBREF3 and GECO corpus BIBREF4), and a few EEG corpora (for example, the UCL corpus BIBREF5) are available. It has been shown that this type of cognitive processing data is useful for improving and evaluating NLP methods (e.g. barrett2018sequence,hollenstein2019cognival, hale2018finding). However, before the Zurich Cognitive Language Processing Corpus (ZuCo 1.0), there was no available data for simultaneous eye-tracking and EEG recordings of natural reading. dimigen2011coregistration studied the linguistic effects of eye movements and EEG co-registration in natural reading and showed that they accurately represent lexical processing. Moreover, the simultaneous recordings are crucial to extract word-level brain activity signals. While the above mentioned studies analyze and leverage natural reading, some NLP work has used eye-tracking during annotation (but, as of yet, not EEG data). mishra2016predicting and joshi2014measuring recorded eye-tracking during binary sentiment annotation (positive/negative). This data was used to determine the annotation complexity of the text passages based on eye movement metrics and for sarcasm detection BIBREF6. Moreover, eye-tracking has been used to analyze the word sense annotation process in Hindi BIBREF7, named entity annotation in Japanese BIBREF8, and to leverage annotator gaze behaviour for coreference resolution BIBREF9. Finally, tomanek2010cognitive used eye-tracking data during entity annotation to build a cost model for active learning. However, until now there is no available data or research that analyzes the differences in the human processing of normal reading versus annotation. Related Work ::: ZuCo1.0 In previous work, we recorded a first dataset of simultaneous eye-tracking and EEG during natural reading BIBREF1. ZuCo 1.0 consists of three reading tasks, two of which contain very similar reading material and experiments as presented in the current work. However, the main difference and reason for recording ZuCo 2.0, consists in the experiment procedure. For ZuCo 1.0 the normal reading and task-specific reading paradigms were recorded in different sessions on different days. Therefore, the recorded data is not appropriate as a means of comparison between natural reading and annotation, since the differences in the brain activity data might result mostly from the different sessions due to the sensitivity of EEG. This, and extending the dataset with more sentences and more subjects, were the main factors for recording the current corpus. We purposefully maintained an overlap of some sentences between both datasets to allow additional analyses (details are described in Section SECREF7). Corpus Construction In this section we describe the contents and experimental design of the ZuCo 2.0 corpus. Corpus Construction ::: Participants We recorded data from 19 participants and discarded the data of one of them due to technical difficulties with the eye-tracking calibration. Hence, we share the data of 18 participants. All participants are healthy adults (mean age = 34 (SD=8.3), 10 females). Their native language is English, originating from Australia, Canada, UK, USA or South Africa. Two participants are left-handed and three participants wear glasses for reading. Details on subject demographics can be found in Table TABREF4. All participants gave written consent for their participation and the re-use of the data prior to the start of the experiments. The study was approved by the Ethics Commission of the University of Zurich. Corpus Construction ::: Reading materials During the recording session, the participants read 739 sentences that were selected from the Wikipedia corpus provided by culotta2006integrating. This corpus was chosen because it provides annotations of semantic relations. We included seven of the originally defined relation types: political_affiliation, education, founder, wife/husband, job_title, nationality, and employer. The sentences were chosen in the same length range as ZuCo 1.0, and with similar Flesch reading ease scores. The dataset statistics are shown in Table TABREF2. Of the 739 sentences, the participants read 349 sentences in a normal reading paradigm, and 390 sentences in a task-specific reading paradigm, in which they had to determine whether a certain relation type occurred in the sentence or not. Table TABREF3 shows the distribution of the different relation types in the sentences of the task-specific annotation paradigm. Purposefully, there are 63 duplicates between the normal reading and the task-specific sentences (8% of all sentences). The intention of these duplicate sentences is to provide a set of sentences read twice by all participants with a different task in mind. Hence, this enables the comparison of eye-tracking and brain activity data when reading normally and when annotating specific relations (see examples in Section SECREF4). Furthermore, there is also an overlap in the sentences between ZuCo 1.0 and ZuCo 2.0. 100 normal reading and 85 task-specific sentences recorded for this dataset were already recorded in ZuCo 1.0. This allows for comparisons between the different recording procedures (i.e. session-specific effects) and between more participants (subject-specific effects). Corpus Construction ::: Experimental design As mentioned above, we recorded two different reading tasks for the ZuCo 2.0 dataset. During both tasks the participants were able to read in their own speed, using a control pad to move to the next sentence and to answer the control questions, which allowed for natural reading. Since each subject reads at their own personal pace, the reading speed between varies between subjects. Table TABREF4 shows the average reading speed for each task, i.e. the average number of seconds a subject spends per sentence before switching to the next one. All 739 sentences were recorded in a single session for each participant. The duration of the recording sessions was between 100 and 180 minutes, depending on the time required to set up and calibrate the devices, and the personal reading speed of the participants. We recorded 14 blocks of approx. 50 sentences, alternating between tasks: 50 sentences of normal reading, followed by 50 sentences of task-specific reading. The order of blocks and sentences within blocks was identical for all subjects. Each sentence block was preceded by a practice round of three sentences. Corpus Construction ::: Experimental design ::: Normal reading (NR) In the first task, participants were instructed to read the sentences naturally, without any specific task other than comprehension. Participants were told to read the sentences normally without any special instructions. Figure FIGREF8 (left) shows an example sentence as it was depicted on the screen during recording. As shown in Figure FIGREF8 (middle), the control condition for this task consisted of single-choice questions about the content of the previous sentence. 12% of randomly selected sentences were followed by such a comprehension question with three answer options on a separate screen. Corpus Construction ::: Experimental design ::: Task-specific reading (TSR) In the second task, the participants were instructed to search for a specific relation in each sentence they read. Instead of comprehension questions, the participants had to decide for each sentence whether it contains the relation or not, i.e. they were actively annotating each sentence. Figure FIGREF8 (right) shows an example screen for this task. 17% of the sentences did not include the relation type and were used as control conditions. All sentences within one block involved the same relation type. The blocks started with a practice round, which described the relation and was followed by three sample sentences, so that the participants would be familiar with the respective relation type. Corpus Construction ::: Linguistic assessment As a linguistic assessment, the vocabulary and language proficiency of the participants was tested with the LexTALE test (Lexical Test for Advanced Learners of English, lemhofer2012introducing). This is an unspeeded lexical decision task designed for intermediate to highly proficient language users. The average LexTALE score over all participants was 88.54%. Moreover, we also report the scores the participants achieved with their answers to the reading comprehension control questions and their relation annotations. The detailed scores for all participants are also presented in Table TABREF4. Corpus Construction ::: Data acquisition Data acquisition took place in a sound-attenuated and dark experiment room. Participants were seated at a distance of 68cm from a 24-inch monitor with a resolution of 800x600 pixels. A stable head position was ensured via a chin rest. Participants were instructed to stay as still as possible during the tasks to avoid motor EEG artifacts. Participants were also offered snacks and water during the breaks and were encouraged to rest. All sentences were presented at the same position on the screen and could span multiple lines. The sentences were presented in black on a light grey background with font size 20-point Arial, resulting in a letter height of 0.8 mm. The experiment was programmed in MATLAB 2016b BIBREF10, using PsychToolbox BIBREF11. Participants completed the tasks sitting alone in the room, while two research assistants were monitoring their progress in the adjoining room. All recording scripts including detailed participant instructions are available alongside the data. Corpus Construction ::: Data acquisition ::: Eye-tracking acquisition Eye position and pupil size were recorded with an infrared video-based eye tracker (EyeLink 1000 Plus, SR Research) at a sampling rate of 500 Hz. The eye tracker was calibrated with a 9-point grid at the beginning of the session and re-validated before each block of sentences. Corpus Construction ::: Data acquisition ::: EEG acquisition High-density EEG data were recorded at a sampling rate of 500 Hz with a bandpass of 0.1 to 100 Hz, using a 128-channel EEG Geodesic Hydrocel system (Electrical Geodesics). The recording reference was set at electrode Cz. The head circumference of each participant was measured to select an appropriately sized EEG net. To ensure good contact, the impedance of each electrode was checked prior to recording, and was kept below 40 kOhm. Electrode impedance levels were checked after every third block of 50 sentences (approx. every 30 mins) and reduced if necessary. Corpus Construction ::: Preprocessing and feature extraction ::: Eye-tracking The eye-tracking data consists of (x,y) gaze location entries for all individual fixations (Figure FIGREF1b). Coordinates were given in pixels with respect to the monitor coordinates (the upper left corner of the screen was (0,0) and down/right was positive). We provide this raw data as well as various engineered eye-tracking features. For this feature extraction only fixations within the boundaries of each displayed word were extracted. Data points distinctly not associated with reading (minimum distance of 50 pixels to the text) were excluded. Additionally, fixations shorter than 100 ms were excluded from the analyses, because these are unlikely to reflect fixations relevant for reading BIBREF12. On the basis of the GECO and ZuCo 1.0 corpora, we extracted the following features: (i) gaze duration (GD), the sum of all fixations on the current word in the first-pass reading before the eye moves out of the word; (ii) total reading time (TRT), the sum of all fixation durations on the current word, including regressions; (iii) first fixation duration (FFD), the duration of the first fixation on the prevailing word; (iv) single fixation duration (SFD), the duration of the first and only fixation on the current word; and (v) go-past time (GPT), the sum of all fixations prior to progressing to the right of the current word, including regressions to previous words that originated from the current word. For each of these eye-tracking features we additionally computed the pupil size. Furthermore, we extracted the number of fixations and mean pupil size for each word and sentence. Corpus Construction ::: Preprocessing and feature extraction ::: EEG The EEG data shared in this project are available as raw data, but also preprocessed with Automagic (version 1.4.6, pedroni2019automagic), a tool for automatic EEG data cleaning and validation. 105 EEG channels (i.e. electrodes) were used from the scalp recordings. 9 EOG channels were used for artifact removal and additional 14 channels lying mainly on the neck and face were discarded before data analysis. Bad channels were identified and interpolated. We used the Multiple Artifact Rejection Algorithm (MARA), a supervised machine learning algorithm that evaluates ICA components, for automatic artifact rejection. MARA has been trained on manual component classifications, and thus captures a wide range of artifacts. MARA is especially effective at detecting and removing eye and muscle artifact components. The effect of this preprocessing can be seen in Figure FIGREF1d. After preprocessing, we synchronized the EEG and eye-tracking data to enable EEG analyses time-locked to the onsets of fixations. To compute oscillatory power measures, we band-pass filtered the continuous EEG signals across an entire reading task for five different frequency bands resulting in a time-series for each frequency band. The independent frequency bands were determined as follows: theta$_1$ (4–6 Hz), theta$_2$ (6.5–8 Hz), alpha$_1$ (8.5–10 Hz), alpha$_2$ (10.5–13 Hz), beta$_1$ (13.5–18 Hz), beta$_2$ (18.5–30 Hz), gamma$_1$ (30.5–40 Hz), and gamma$_2$ (40–49.5 Hz). We then applied a Hilbert transformation to each of these time-series. We specifically chose the Hilbert transformation to maintain the temporal information of the amplitude of the frequency bands, to enable the power of the different frequencies for time segments defined through the fixations from the eye-tracking recording. Thus, for each eye-tracking feature we computed the corresponding EEG feature in each frequency band. Furthermore, we extracted sentence-level EEG features by calculating the power in each frequency band, and additionally, the difference of the power spectra between frontal left and right homologue electrodes pairs. For each eye-tracking based EEG feature, all channels were subject to an artifact rejection criterion of $90\mu V$ to exclude trials with transient noise. Data Validation The aim of the technical validation of the data is to guarantee good recording quality and to replicate findings of previous studies investigating co-registration of EEG and eye movement data during natural reading tasks (e.g. dimigen2011coregistration). We also compare the results to ZuCo 1.0 BIBREF1, which allows a more direct comparison due to the analogous recording procedure. Data Validation ::: Eye-tracking We validated the recorded eye-tracking data by analyzing the fixations made by all subjects through their reading speed and omission rate on sentence level. The omission rate is defined as the percentage of words that is not fixated in a sentence. Figure FIGREF10 (middle) shows the mean reading speed over all subjects, measured in seconds per sentence and Figure FIGREF10 (right) shows the mean omission rates aggregated over all subjects for each task. Clearly, the participants made less fixations during the task-specific reading, which lead to faster reading speed. Moreover, we corroborated these sentence-level metrics by visualizing the skipping proportion on word level (Figure FIGREF13). The skipping proportion is the average rate of words being skipped (i.e. not being fixated) in a sentence. As expected, this also increases in the task-specific reading. Although the reading material is from the same source and of the same length range (see Figure FIGREF10 (left)), in the first task (NR) passive reading was recorded, while in the second task (TSR) the subjects had to annotate a specific relation type in each sentence. Thus, the task-specific annotation reading lead to shorter passes because the goal was merely to recognize a relation in the text, but not necessarily to process the every word in each sentence. This distinct reading behavior is shown in Figure FIGREF15, where fixations occur until the end of the sentence during normal reading, while during task-specific reading the fixations stop after the decisive words to detect a given relation type. Finally, we also analyzed the average reading times for each of the extracted eye-tracking features. The means and distributions for both tasks are shown in Figure FIGREF21. These results are in line with the recorded data in ZuCo 1.0, as well as with the features extracted in the GECO corpus BIBREF4. Data Validation ::: EEG As a first validation step, we extracted fixation-related potentials (FRPs), where the EEG signal during all fixations of one task are averaged. Figure FIGREF24 shows the time-series of the resulting FRPs for two electrodes (PO8 and Cz), as well as topographies of the voltage distributions across the scalp at selected points in time. The five components (for which the scalp topographies are plotted) are highly similar in the time-course of the chosen electrodes to dimigen2011coregistration as well as to ZuCo 1.0. Moreover, these previous studies were able to show an effect of fixation duration on the resulting FRPs. To show this dependency we followed two approaches. First, for each reading task, all single-trial FRPs were ordered by fixation duration and a vertical sliding time-window was used to smooth the data BIBREF13. Figure FIGREF25 (bottom) shows the resulting plots. In line with this previous work, a first positivation can be identified at 100 ms post-fixation onset. A second positive peak is located dependent on the duration of the fixation, which can be explained by the time-jittered succeeding fixation. The second approach is based on henderson2013co in which single trial EEG segments are clustered by the duration of the current fixation. As shown in Figure FIGREF25 (top), we chose four clusters and averaged the data within each cluster to four distinct FRPs, depending on the fixation duration. Again, the same positivation peaks become apparent. Both findings are consistent with the previous work mentioned and with our findings from ZuCo 1.0. Conclusion We presented a new, freely available corpus of eye movement and electrical brain activity recordings during natural reading as well as during annotation. This is the first dataset that allows for the comparison between these two reading paradigms. We described the materials and experiment design in detail and conducted an extensive validation to ensure the quality of the recorded data. Since this corpus is tailored to cognitively-inspired NLP, the applications and re-use potentials of this data are extensive. The provided word-level and sentence-level eye-tracking and EEG features can be used to improve and evaluate NLP and machine learning methods, for instance, to evaluate linguistic phenomena in neural models via psycholinguistic data. In addition, because the sentences contains semantic relation labels and the annotations of all participants, it can also be widely used for relation extraction and classification. Finally, the two carefully constructed reading paradigms allow for the comparison between normal reading and reading during annotation, which can be relevant to improve the manual labelling process as well as the quality of the annotations for supervised machine learning.
What kind of sentences were read?
ac7f6497be4bcca64e75f28934b207c9e8097576
What Ġkind Ġof Ġsentences Ġwere Ġread ?
sent ences Ġthat Ġwere Ġselected Ġfrom Ġthe ĠWikipedia Ġcorpus Ġprovided Ġby Ġcul otta 2006 integ rating
<s> What Ġkind Ġof Ġsentences Ġwere Ġread ? </s> Introduction </s> How Ġhumans Ġprocess Ġlanguage Ġhas Ġbecome Ġincreasingly Ġrelevant Ġin Ġnatural Ġlanguage Ġprocessing Ġsince Ġphysiological Ġdata Ġduring Ġlanguage Ġunderstanding Ġis Ġmore Ġaccessible Ġand Ġrecorded Ġwith Ġless Ġeffort . ĠIn Ġthis Ġwork , Ġwe Ġfocus Ġon Ġeye - tracking Ġand Ġelectro ence phal ography Ġ( EE G ) Ġrecordings Ġto Ġcapture Ġthe Ġreading Ġprocess . ĠOn Ġone Ġhand , Ġeye Ġmovement Ġdata Ġprovides Ġmillisec ond - acc urate Ġrecords Ġabout Ġwhere Ġhumans Ġlook Ġwhen Ġthey Ġare Ġreading , Ġand Ġis Ġhighly Ġcorrelated Ġwith Ġthe Ġcognitive Ġload Ġassociated Ġwith Ġdifferent Ġstages Ġof Ġtext Ġprocessing . ĠOn Ġthe Ġother Ġhand , ĠEEG Ġrecords Ġelectrical Ġbrain Ġactivity Ġacross Ġthe Ġscalp Ġand Ġis Ġa Ġdirect Ġmeasure Ġof Ġphysiological Ġprocesses , Ġincluding Ġlanguage Ġprocessing . ĠThe Ġcombination Ġof Ġboth Ġmeasurement Ġmethods Ġenables Ġus Ġto Ġstudy Ġthe Ġlanguage Ġunderstanding Ġprocess Ġin Ġa Ġmore Ġnatural Ġsetting , Ġwhere Ġparticipants Ġread Ġfull Ġsentences Ġat Ġa Ġtime , Ġin Ġtheir Ġown Ġspeed . ĠEye - tracking Ġthen Ġpermits Ġus Ġto Ġdefine Ġexact Ġword Ġboundaries Ġin Ġthe Ġtimeline Ġof Ġa Ġsubject Ġreading Ġa Ġsentence , Ġallowing Ġthe Ġextraction Ġof Ġbrain Ġactivity Ġsignals Ġfor Ġeach Ġword . </s> Human Ġcognitive Ġlanguage Ġprocessing Ġdata Ġis Ġimmensely Ġuseful Ġfor ĠN LP : ĠNot Ġonly Ġcan Ġit Ġbe Ġlever aged Ġto Ġimprove ĠN LP Ġapplications Ġ( e . g . Ġbar rett 2016 weak ly Ġfor Ġpart - of - speech Ġtagging Ġor Ġk ler ke 2016 impro ving Ġfor Ġsentence Ġcompression ), Ġbut Ġalso Ġto Ġevaluate Ġstate - of - the - art Ġmachine Ġlearning Ġsystems . ĠFor Ġexample , Ġh ollen stein 2019 c ogn ival Ġevaluate Ġword Ġembed d ings , Ġor Ġsch w artz 2019 inducing Ġfine - t une Ġlanguage Ġmodels Ġwith Ġbrain - relevant Ġbias . </s> Additionally , Ġthe Ġavailability Ġof Ġlabelled Ġdata Ġplays Ġa Ġcrucial Ġrole Ġin Ġall Ġsupervised Ġmachine Ġlearning Ġapplications . ĠPhys iological Ġdata Ġcan Ġbe Ġused Ġto Ġunderstand Ġand Ġimprove Ġthe Ġlab elling Ġprocess Ġ( e . g . Ġto kun aga 2017 eye ), Ġand , Ġfor Ġinstance , Ġto Ġbuild Ġcost Ġmodels Ġfor Ġactive Ġlearning Ġscenarios ĠB IB REF 0 . ĠIs Ġit Ġpossible Ġto Ġreplace Ġthis Ġexpensive Ġmanual Ġwork Ġwith Ġmodels Ġtrained Ġon Ġphysiological Ġactivity Ġdata Ġrecorded Ġfrom Ġhumans Ġwhile Ġreading ? ĠThat Ġis Ġto Ġsay , Ġcan Ġwe Ġfind Ġand Ġextract Ġrelevant Ġaspects Ġof Ġtext Ġunderstanding Ġand Ġannotation Ġdirectly Ġfrom Ġthe Ġsource , Ġi . e . Ġeye - tracking Ġand Ġbrain Ġactivity Ġsignals Ġduring Ġreading ? </s> Mot ivated Ġby Ġthese Ġquestions Ġand Ġour Ġpreviously Ġreleased Ġdataset , ĠZ u Co Ġ1 . 0 ĠB IB REF 1 , Ġwe Ġdeveloped Ġthis Ġnew Ġcorpus , Ġwhere Ġwe Ġspecifically Ġaim Ġto Ġcollect Ġrecordings Ġduring Ġnatural Ġreading Ġas Ġwell Ġas Ġduring Ġannotation . </s> We Ġprovide Ġthe Ġfirst Ġdataset Ġof Ġsimultaneous Ġeye Ġmovement Ġand Ġbrain Ġactivity Ġrecordings Ġto Ġanalyze Ġand Ġcompare Ġnormal Ġreading Ġto Ġtask - specific Ġreading Ġduring Ġannotation . ĠThe ĠZurich ĠCognitive ĠLanguage ĠProcessing ĠCorpus Ġ( Z u Co ) Ġ2 . 0 , Ġincluding Ġraw Ġand Ġpre process ed Ġeye - tracking Ġand Ġelectro ence phal ography Ġ( EE G ) Ġdata Ġof Ġ18 Ġsubjects , Ġas Ġwell Ġas Ġthe Ġrecording Ġand Ġpre processing Ġscripts , Ġis Ġpublicly Ġavailable Ġat Ġhttps :// os f . io / 2 ur ht /. ĠIt Ġcontains Ġphysiological Ġdata Ġof Ġeach Ġsubject Ġreading Ġ7 39 ĠEnglish Ġsentences Ġfrom ĠWikipedia Ġ( see Ġexample Ġin ĠFigure ĠFIG REF 1 ). ĠWe Ġwant Ġto Ġhighlight Ġthe Ġre - use Ġpotential Ġof Ġthis Ġdata . ĠIn Ġaddition Ġto Ġthe Ġpsych ol ingu istic Ġmotivation , Ġthis Ġcorpus Ġis Ġespecially Ġtailored Ġfor Ġtraining Ġand Ġevaluating Ġmachine Ġlearning Ġalgorithms Ġfor ĠN LP Ġpurposes . ĠWe Ġconduct Ġa Ġdetailed Ġtechnical Ġvalidation Ġof Ġthe Ġdata Ġas Ġproof Ġof Ġthe Ġquality Ġof Ġthe Ġrecordings . </s> Related ĠWork </s> Some Ġeye - tracking Ġcorpor a Ġof Ġnatural Ġreading Ġ( e . g . Ġthe ĠDund ee ĠB IB REF 2 , ĠProv o ĠB IB REF 3 Ġand ĠG EC O Ġcorpus ĠB IB REF 4 ), Ġand Ġa Ġfew ĠEEG Ġcorpor a Ġ( for Ġexample , Ġthe ĠU CL Ġcorpus ĠB IB REF 5 ) Ġare Ġavailable . ĠIt Ġhas Ġbeen Ġshown Ġthat Ġthis Ġtype Ġof Ġcognitive Ġprocessing Ġdata Ġis Ġuseful Ġfor Ġimproving Ġand Ġevaluating ĠN LP Ġmethods Ġ( e . g . Ġbar rett 2018 sequence , h ollen stein 2019 c ogn ival , Ġh ale 2018 finding ). ĠHowever , Ġbefore Ġthe ĠZurich ĠCognitive ĠLanguage ĠProcessing ĠCorpus Ġ( Z u Co Ġ1 . 0 ), Ġthere Ġwas Ġno Ġavailable Ġdata Ġfor Ġsimultaneous Ġeye - tracking Ġand ĠEEG Ġrecordings Ġof Ġnatural Ġreading . Ġdim igen 2011 core g istration Ġstudied Ġthe Ġlinguistic Ġeffects Ġof Ġeye Ġmovements Ġand ĠEEG Ġco - reg istration Ġin Ġnatural Ġreading Ġand Ġshowed Ġthat Ġthey Ġaccurately Ġrepresent Ġlex ical Ġprocessing . ĠMoreover , Ġthe Ġsimultaneous Ġrecordings Ġare Ġcrucial Ġto Ġextract Ġword - level Ġbrain Ġactivity Ġsignals . </s> While Ġthe Ġabove Ġmentioned Ġstudies Ġanalyze Ġand Ġleverage Ġnatural Ġreading , Ġsome ĠN LP Ġwork Ġhas Ġused Ġeye - tracking Ġduring Ġannotation Ġ( but , Ġas Ġof Ġyet , Ġnot ĠEEG Ġdata ). Ġmish ra 2016 p redict ing Ġand Ġj oshi 2014 me asuring Ġrecorded Ġeye - tracking Ġduring Ġbinary Ġsentiment Ġannotation Ġ( positive / negative ). ĠThis Ġdata Ġwas Ġused Ġto Ġdetermine Ġthe Ġannotation Ġcomplexity Ġof Ġthe Ġtext Ġpassages Ġbased Ġon Ġeye Ġmovement Ġmetrics Ġand Ġfor Ġsarc asm Ġdetection ĠB IB REF 6 . ĠMoreover , Ġeye - tracking Ġhas Ġbeen Ġused Ġto Ġanalyze Ġthe Ġword Ġsense Ġannotation Ġprocess Ġin ĠHindi ĠB IB REF 7 , Ġnamed Ġentity Ġannotation Ġin ĠJapanese ĠB IB REF 8 , Ġand Ġto Ġleverage Ġannot ator Ġgaze Ġbehaviour Ġfor Ġcore ference Ġresolution ĠB IB REF 9 . ĠFinally , Ġto man ek 2010 c ognitive Ġused Ġeye - tracking Ġdata Ġduring Ġentity Ġannotation Ġto Ġbuild Ġa Ġcost Ġmodel Ġfor Ġactive Ġlearning . ĠHowever , Ġuntil Ġnow Ġthere Ġis Ġno Ġavailable Ġdata Ġor Ġresearch Ġthat Ġanaly zes Ġthe Ġdifferences Ġin Ġthe Ġhuman Ġprocessing Ġof Ġnormal Ġreading Ġversus Ġannotation . </s> Related ĠWork Ġ: :: ĠZ u Co 1 . 0 </s> In Ġprevious Ġwork , Ġwe Ġrecorded Ġa Ġfirst Ġdataset Ġof Ġsimultaneous Ġeye - tracking Ġand ĠEEG Ġduring Ġnatural Ġreading ĠB IB REF 1 . ĠZ u Co Ġ1 . 0 Ġconsists Ġof Ġthree Ġreading Ġtasks , Ġtwo Ġof Ġwhich Ġcontain Ġvery Ġsimilar Ġreading Ġmaterial Ġand Ġexperiments Ġas Ġpresented Ġin Ġthe Ġcurrent Ġwork . ĠHowever , Ġthe Ġmain Ġdifference Ġand Ġreason Ġfor Ġrecording ĠZ u Co Ġ2 . 0 , Ġconsists Ġin Ġthe Ġexperiment Ġprocedure . ĠFor ĠZ u Co Ġ1 . 0 Ġthe Ġnormal Ġreading Ġand Ġtask - specific Ġreading Ġparad ig ms Ġwere Ġrecorded Ġin Ġdifferent Ġsessions Ġon Ġdifferent Ġdays . ĠTherefore , Ġthe Ġrecorded Ġdata Ġis Ġnot Ġappropriate Ġas Ġa Ġmeans Ġof Ġcomparison Ġbetween Ġnatural Ġreading Ġand Ġannotation , Ġsince Ġthe Ġdifferences Ġin Ġthe Ġbrain Ġactivity Ġdata Ġmight Ġresult Ġmostly Ġfrom Ġthe Ġdifferent Ġsessions Ġdue Ġto Ġthe Ġsensitivity Ġof ĠEEG . ĠThis , Ġand Ġextending Ġthe Ġdataset Ġwith Ġmore Ġsentences Ġand Ġmore Ġsubjects , Ġwere Ġthe Ġmain Ġfactors Ġfor Ġrecording Ġthe Ġcurrent Ġcorpus . ĠWe Ġpurposefully Ġmaintained Ġan Ġoverlap Ġof Ġsome Ġsentences Ġbetween Ġboth Ġdatasets Ġto Ġallow Ġadditional Ġanalyses Ġ( details Ġare Ġdescribed Ġin ĠSection ĠSEC REF 7 ). </s> Corp us ĠConstruction </s> In Ġthis Ġsection Ġwe Ġdescribe Ġthe Ġcontents Ġand Ġexperimental Ġdesign Ġof Ġthe ĠZ u Co Ġ2 . 0 Ġcorpus . </s> Corp us ĠConstruction Ġ: :: ĠParticipants </s> We Ġrecorded Ġdata Ġfrom Ġ19 Ġparticipants Ġand Ġdiscarded Ġthe Ġdata Ġof Ġone Ġof Ġthem Ġdue Ġto Ġtechnical Ġdifficulties Ġwith Ġthe Ġeye - tracking Ġcalibration . ĠHence , Ġwe Ġshare Ġthe Ġdata Ġof Ġ18 Ġparticipants . ĠAll Ġparticipants Ġare Ġhealthy Ġadults Ġ( mean Ġage Ġ= Ġ34 Ġ( SD = 8 . 3 ), Ġ10 Ġfemales ). ĠTheir Ġnative Ġlanguage Ġis ĠEnglish , Ġoriginating Ġfrom ĠAustralia , ĠCanada , ĠUK , ĠUSA Ġor ĠSouth ĠAfrica . ĠTwo Ġparticipants Ġare Ġleft - handed Ġand Ġthree Ġparticipants Ġwear Ġglasses Ġfor Ġreading . ĠDetails Ġon Ġsubject Ġdemographics Ġcan Ġbe Ġfound Ġin ĠTable ĠT AB REF 4 . ĠAll Ġparticipants Ġgave Ġwritten Ġconsent Ġfor Ġtheir Ġparticipation Ġand Ġthe Ġre - use Ġof Ġthe Ġdata Ġprior Ġto Ġthe Ġstart Ġof Ġthe Ġexperiments . ĠThe Ġstudy Ġwas Ġapproved Ġby Ġthe ĠEthics ĠCommission Ġof Ġthe ĠUniversity Ġof ĠZurich . </s> Corp us ĠConstruction Ġ: :: ĠReading Ġmaterials </s> During Ġthe Ġrecording Ġsession , Ġthe Ġparticipants Ġread Ġ7 39 Ġsentences Ġthat Ġwere Ġselected Ġfrom Ġthe ĠWikipedia Ġcorpus Ġprovided Ġby Ġcul otta 2006 integ rating . ĠThis Ġcorpus Ġwas Ġchosen Ġbecause Ġit Ġprovides Ġannotations Ġof Ġsemantic Ġrelations . ĠWe Ġincluded Ġseven Ġof Ġthe Ġoriginally Ġdefined Ġrelation Ġtypes : Ġpolitical _ aff iliation , Ġeducation , Ġfounder , Ġwife / husband , Ġjob _ title , Ġnationality , Ġand Ġemployer . ĠThe Ġsentences Ġwere Ġchosen Ġin Ġthe Ġsame Ġlength Ġrange Ġas ĠZ u Co Ġ1 . 0 , Ġand Ġwith Ġsimilar ĠF les ch Ġreading Ġease Ġscores . ĠThe Ġdataset Ġstatistics Ġare Ġshown Ġin ĠTable ĠT AB REF 2 . </s> Of Ġthe Ġ7 39 Ġsentences , Ġthe Ġparticipants Ġread Ġ349 Ġsentences Ġin Ġa Ġnormal Ġreading Ġparadigm , Ġand Ġ390 Ġsentences Ġin Ġa Ġtask - specific Ġreading Ġparadigm , Ġin Ġwhich Ġthey Ġhad Ġto Ġdetermine Ġwhether Ġa Ġcertain Ġrelation Ġtype Ġoccurred Ġin Ġthe Ġsentence Ġor Ġnot . ĠTable ĠT AB REF 3 Ġshows Ġthe Ġdistribution Ġof Ġthe Ġdifferent Ġrelation Ġtypes Ġin Ġthe Ġsentences Ġof Ġthe Ġtask - specific Ġannotation Ġparadigm . </s> Pur pose fully , Ġthere Ġare Ġ63 Ġdupl icates Ġbetween Ġthe Ġnormal Ġreading Ġand Ġthe Ġtask - specific Ġsentences Ġ( 8 % Ġof Ġall Ġsentences ). ĠThe Ġintention Ġof Ġthese Ġduplicate Ġsentences Ġis Ġto Ġprovide Ġa Ġset Ġof Ġsentences Ġread Ġtwice Ġby Ġall Ġparticipants Ġwith Ġa Ġdifferent Ġtask Ġin Ġmind . ĠHence , Ġthis Ġenables Ġthe Ġcomparison Ġof Ġeye - tracking Ġand Ġbrain Ġactivity Ġdata Ġwhen Ġreading Ġnormally Ġand Ġwhen Ġannot ating Ġspecific Ġrelations Ġ( see Ġexamples Ġin ĠSection ĠSEC REF 4 ). </s> Furthermore , Ġthere Ġis Ġalso Ġan Ġoverlap Ġin Ġthe Ġsentences Ġbetween ĠZ u Co Ġ1 . 0 Ġand ĠZ u Co Ġ2 . 0 . Ġ100 Ġnormal Ġreading Ġand Ġ85 Ġtask - specific Ġsentences Ġrecorded Ġfor Ġthis Ġdataset Ġwere Ġalready Ġrecorded Ġin ĠZ u Co Ġ1 . 0 . ĠThis Ġallows Ġfor Ġcomparisons Ġbetween Ġthe Ġdifferent Ġrecording Ġprocedures Ġ( i . e . Ġsession - specific Ġeffects ) Ġand Ġbetween Ġmore Ġparticipants Ġ( subject - specific Ġeffects ). </s> Corp us ĠConstruction Ġ: :: ĠExperimental Ġdesign </s> As Ġmentioned Ġabove , Ġwe Ġrecorded Ġtwo Ġdifferent Ġreading Ġtasks Ġfor Ġthe ĠZ u Co Ġ2 . 0 Ġdataset . ĠDuring Ġboth Ġtasks Ġthe Ġparticipants Ġwere Ġable Ġto Ġread Ġin Ġtheir Ġown Ġspeed , Ġusing Ġa Ġcontrol Ġpad Ġto Ġmove Ġto Ġthe Ġnext Ġsentence Ġand Ġto Ġanswer Ġthe Ġcontrol Ġquestions , Ġwhich Ġallowed Ġfor Ġnatural Ġreading . ĠSince Ġeach Ġsubject Ġreads Ġat Ġtheir Ġown Ġpersonal Ġpace , Ġthe Ġreading Ġspeed Ġbetween Ġvaries Ġbetween Ġsubjects . ĠTable ĠT AB REF 4 Ġshows Ġthe Ġaverage Ġreading Ġspeed Ġfor Ġeach Ġtask , Ġi . e . Ġthe Ġaverage Ġnumber Ġof Ġseconds Ġa Ġsubject Ġspends Ġper Ġsentence Ġbefore Ġswitching Ġto Ġthe Ġnext Ġone . </s> All Ġ7 39 Ġsentences Ġwere Ġrecorded Ġin Ġa Ġsingle Ġsession Ġfor Ġeach Ġparticipant . ĠThe Ġduration Ġof Ġthe Ġrecording Ġsessions Ġwas Ġbetween Ġ100 Ġand Ġ180 Ġminutes , Ġdepending Ġon Ġthe Ġtime Ġrequired Ġto Ġset Ġup Ġand Ġcalibr ate Ġthe Ġdevices , Ġand Ġthe Ġpersonal Ġreading Ġspeed Ġof Ġthe Ġparticipants . </s> We Ġrecorded Ġ14 Ġblocks Ġof Ġapprox . Ġ50 Ġsentences , Ġalternating Ġbetween Ġtasks : Ġ50 Ġsentences Ġof Ġnormal Ġreading , Ġfollowed Ġby Ġ50 Ġsentences Ġof Ġtask - specific Ġreading . ĠThe Ġorder Ġof Ġblocks Ġand Ġsentences Ġwithin Ġblocks Ġwas Ġidentical Ġfor Ġall Ġsubjects . ĠEach Ġsentence Ġblock Ġwas Ġpreceded Ġby Ġa Ġpractice Ġround Ġof Ġthree Ġsentences . </s> Corp us ĠConstruction Ġ: :: ĠExperimental Ġdesign Ġ: :: ĠNormal Ġreading Ġ( NR ) </s> In Ġthe Ġfirst Ġtask , Ġparticipants Ġwere Ġinstructed Ġto Ġread Ġthe Ġsentences Ġnaturally , Ġwithout Ġany Ġspecific Ġtask Ġother Ġthan Ġcomprehension . ĠParticipants Ġwere Ġtold Ġto Ġread Ġthe Ġsentences Ġnormally Ġwithout Ġany Ġspecial Ġinstructions . ĠFigure ĠFIG REF 8 Ġ( left ) Ġshows Ġan Ġexample Ġsentence Ġas Ġit Ġwas Ġdepicted Ġon Ġthe Ġscreen Ġduring Ġrecording . ĠAs Ġshown Ġin ĠFigure ĠFIG REF 8 Ġ( middle ), Ġthe Ġcontrol Ġcondition Ġfor Ġthis Ġtask Ġconsisted Ġof Ġsingle - choice Ġquestions Ġabout Ġthe Ġcontent Ġof Ġthe Ġprevious Ġsentence . Ġ12 % Ġof Ġrandomly Ġselected Ġsentences Ġwere Ġfollowed Ġby Ġsuch Ġa Ġcomprehension Ġquestion Ġwith Ġthree Ġanswer Ġoptions Ġon Ġa Ġseparate Ġscreen . </s> Corp us ĠConstruction Ġ: :: ĠExperimental Ġdesign Ġ: :: ĠTask - specific Ġreading Ġ( TS R ) </s> In Ġthe Ġsecond Ġtask , Ġthe Ġparticipants Ġwere Ġinstructed Ġto Ġsearch Ġfor Ġa Ġspecific Ġrelation Ġin Ġeach Ġsentence Ġthey Ġread . ĠInstead Ġof Ġcomprehension Ġquestions , Ġthe Ġparticipants Ġhad Ġto Ġdecide Ġfor Ġeach Ġsentence Ġwhether Ġit Ġcontains Ġthe Ġrelation Ġor Ġnot , Ġi . e . Ġthey Ġwere Ġactively Ġannot ating Ġeach Ġsentence . ĠFigure ĠFIG REF 8 Ġ( right ) Ġshows Ġan Ġexample Ġscreen Ġfor Ġthis Ġtask . Ġ17 % Ġof Ġthe Ġsentences Ġdid Ġnot Ġinclude Ġthe Ġrelation Ġtype Ġand Ġwere Ġused Ġas Ġcontrol Ġconditions . ĠAll Ġsentences Ġwithin Ġone Ġblock Ġinvolved Ġthe Ġsame Ġrelation Ġtype . ĠThe Ġblocks Ġstarted Ġwith Ġa Ġpractice Ġround , Ġwhich Ġdescribed Ġthe Ġrelation Ġand Ġwas Ġfollowed Ġby Ġthree Ġsample Ġsentences , Ġso Ġthat Ġthe Ġparticipants Ġwould Ġbe Ġfamiliar Ġwith Ġthe Ġrespective Ġrelation Ġtype . </s> Corp us ĠConstruction Ġ: :: ĠL ingu istic Ġassessment </s> As Ġa Ġlinguistic Ġassessment , Ġthe Ġvocabulary Ġand Ġlanguage Ġproficiency Ġof Ġthe Ġparticipants Ġwas Ġtested Ġwith Ġthe ĠLex T ALE Ġtest Ġ( Lex ical ĠTest Ġfor ĠAdvanced ĠLear ners Ġof ĠEnglish , Ġle m h ofer 2012 introdu cing ). ĠThis Ġis Ġan Ġun speed ed Ġlex ical Ġdecision Ġtask Ġdesigned Ġfor Ġintermediate Ġto Ġhighly Ġproficient Ġlanguage Ġusers . ĠThe Ġaverage ĠLex T ALE Ġscore Ġover Ġall Ġparticipants Ġwas Ġ88 . 54 %. ĠMoreover , Ġwe Ġalso Ġreport Ġthe Ġscores Ġthe Ġparticipants Ġachieved Ġwith Ġtheir Ġanswers Ġto Ġthe Ġreading Ġcomprehension Ġcontrol Ġquestions Ġand Ġtheir Ġrelation Ġannotations . ĠThe Ġdetailed Ġscores Ġfor Ġall Ġparticipants Ġare Ġalso Ġpresented Ġin ĠTable ĠT AB REF 4 . </s> Corp us ĠConstruction Ġ: :: ĠData Ġacquisition </s> Data Ġacquisition Ġtook Ġplace Ġin Ġa Ġsound - atten uated Ġand Ġdark Ġexperiment Ġroom . ĠParticipants Ġwere Ġseated Ġat Ġa Ġdistance Ġof Ġ68 cm Ġfrom Ġa Ġ24 - inch Ġmonitor Ġwith Ġa Ġresolution Ġof Ġ800 x 600 Ġpixels . ĠA Ġstable Ġhead Ġposition Ġwas Ġensured Ġvia Ġa Ġchin Ġrest . ĠParticipants Ġwere Ġinstructed Ġto Ġstay Ġas Ġstill Ġas Ġpossible Ġduring Ġthe Ġtasks Ġto Ġavoid Ġmotor ĠEEG Ġartifacts . ĠParticipants Ġwere Ġalso Ġoffered Ġsnacks Ġand Ġwater Ġduring Ġthe Ġbreaks Ġand Ġwere Ġencouraged Ġto Ġrest . ĠAll Ġsentences Ġwere Ġpresented Ġat Ġthe Ġsame Ġposition Ġon Ġthe Ġscreen Ġand Ġcould Ġspan Ġmultiple Ġlines . ĠThe Ġsentences Ġwere Ġpresented Ġin Ġblack Ġon Ġa Ġlight Ġgrey Ġbackground Ġwith Ġfont Ġsize Ġ20 - point ĠA rial , Ġresulting Ġin Ġa Ġletter Ġheight Ġof Ġ0 . 8 Ġmm . ĠThe Ġexperiment Ġwas Ġprogrammed Ġin ĠMAT LAB Ġ2016 b ĠB IB REF 10 , Ġusing ĠPsych Tool box ĠB IB REF 11 . ĠParticipants Ġcompleted Ġthe Ġtasks Ġsitting Ġalone Ġin Ġthe Ġroom , Ġwhile Ġtwo Ġresearch Ġassistants Ġwere Ġmonitoring Ġtheir Ġprogress Ġin Ġthe Ġadjoining Ġroom . ĠAll Ġrecording Ġscripts Ġincluding Ġdetailed Ġparticipant Ġinstructions Ġare Ġavailable Ġalongside Ġthe Ġdata . </s> Corp us ĠConstruction Ġ: :: ĠData Ġacquisition Ġ: :: ĠEye - tracking Ġacquisition </s> Eye Ġposition Ġand Ġpupil Ġsize Ġwere Ġrecorded Ġwith Ġan Ġinfrared Ġvideo - based Ġeye Ġtracker Ġ( Eye Link Ġ1000 ĠPlus , ĠSR ĠResearch ) Ġat Ġa Ġsampling Ġrate Ġof Ġ500 ĠHz . ĠThe Ġeye Ġtracker Ġwas Ġcalibrated Ġwith Ġa Ġ9 - point Ġgrid Ġat Ġthe Ġbeginning Ġof Ġthe Ġsession Ġand Ġre - valid ated Ġbefore Ġeach Ġblock Ġof Ġsentences . </s> Corp us ĠConstruction Ġ: :: ĠData Ġacquisition Ġ: :: ĠEEG Ġacquisition </s> High - density ĠEEG Ġdata Ġwere Ġrecorded Ġat Ġa Ġsampling Ġrate Ġof Ġ500 ĠHz Ġwith Ġa Ġband pass Ġof Ġ0 . 1 Ġto Ġ100 ĠHz , Ġusing Ġa Ġ128 - channel ĠEEG ĠGe odes ic ĠHydro cel Ġsystem Ġ( Elect rical ĠGe odes ics ). ĠThe Ġrecording Ġreference Ġwas Ġset Ġat Ġelectrode ĠC z . ĠThe Ġhead Ġcircumference Ġof Ġeach Ġparticipant Ġwas Ġmeasured Ġto Ġselect Ġan Ġappropriately Ġsized ĠEEG Ġnet . ĠTo Ġensure Ġgood Ġcontact , Ġthe Ġimpedance Ġof Ġeach Ġelectrode Ġwas Ġchecked Ġprior Ġto Ġrecording , Ġand Ġwas Ġkept Ġbelow Ġ40 Ġk Oh m . ĠElectro de Ġimpedance Ġlevels Ġwere Ġchecked Ġafter Ġevery Ġthird Ġblock Ġof Ġ50 Ġsentences Ġ( app rox . Ġevery Ġ30 Ġmins ) Ġand Ġreduced Ġif Ġnecessary . </s> Corp us ĠConstruction Ġ: :: ĠPre processing Ġand Ġfeature Ġextraction Ġ: :: ĠEye - tracking </s> The Ġeye - tracking Ġdata Ġconsists Ġof Ġ( x , y ) Ġgaze Ġlocation Ġentries Ġfor Ġall Ġindividual Ġfix ations Ġ( Figure ĠFIG REF 1 b ). ĠCoord inates Ġwere Ġgiven Ġin Ġpixels Ġwith Ġrespect Ġto Ġthe Ġmonitor Ġcoordinates Ġ( the Ġupper Ġleft Ġcorner Ġof Ġthe Ġscreen Ġwas Ġ( 0 , 0 ) Ġand Ġdown / right Ġwas Ġpositive ). ĠWe Ġprovide Ġthis Ġraw Ġdata Ġas Ġwell Ġas Ġvarious Ġengineered Ġeye - tracking Ġfeatures . ĠFor Ġthis Ġfeature Ġextraction Ġonly Ġfix ations Ġwithin Ġthe Ġboundaries Ġof Ġeach Ġdisplayed Ġword Ġwere Ġextracted . ĠData Ġpoints Ġdistinctly Ġnot Ġassociated Ġwith Ġreading Ġ( minimum Ġdistance Ġof Ġ50 Ġpixels Ġto Ġthe Ġtext ) Ġwere Ġexcluded . ĠAdditionally , Ġfix ations Ġshorter Ġthan Ġ100 Ġms Ġwere Ġexcluded Ġfrom Ġthe Ġanalyses , Ġbecause Ġthese Ġare Ġunlikely Ġto Ġreflect Ġfix ations Ġrelevant Ġfor Ġreading ĠB IB REF 12 . ĠOn Ġthe Ġbasis Ġof Ġthe ĠG EC O Ġand ĠZ u Co Ġ1 . 0 Ġcorpor a , Ġwe Ġextracted Ġthe Ġfollowing Ġfeatures : Ġ( i ) Ġgaze Ġduration Ġ( GD ), Ġthe Ġsum Ġof Ġall Ġfix ations Ġon Ġthe Ġcurrent Ġword Ġin Ġthe Ġfirst - pass Ġreading Ġbefore Ġthe Ġeye Ġmoves Ġout Ġof Ġthe Ġword ; Ġ( ii ) Ġtotal Ġreading Ġtime Ġ( TR T ), Ġthe Ġsum Ġof Ġall Ġfixation Ġd urations Ġon Ġthe Ġcurrent Ġword , Ġincluding Ġregress ions ; Ġ( iii ) Ġfirst Ġfixation Ġduration Ġ( FF D ), Ġthe Ġduration Ġof Ġthe Ġfirst Ġfixation Ġon Ġthe Ġprevailing Ġword ; Ġ( iv ) Ġsingle Ġfixation Ġduration Ġ( SF D ), Ġthe Ġduration Ġof Ġthe Ġfirst Ġand Ġonly Ġfixation Ġon Ġthe Ġcurrent Ġword ; Ġand Ġ( v ) Ġgo - past Ġtime Ġ( G PT ), Ġthe Ġsum Ġof Ġall Ġfix ations Ġprior Ġto Ġprogressing Ġto Ġthe Ġright Ġof Ġthe Ġcurrent Ġword , Ġincluding Ġregress ions Ġto Ġprevious Ġwords Ġthat Ġoriginated Ġfrom Ġthe Ġcurrent Ġword . ĠFor Ġeach Ġof Ġthese Ġeye - tracking Ġfeatures Ġwe Ġadditionally Ġcomputed Ġthe Ġpupil Ġsize . ĠFurthermore , Ġwe Ġextracted Ġthe Ġnumber Ġof Ġfix ations Ġand Ġmean Ġpupil Ġsize Ġfor Ġeach Ġword Ġand Ġsentence . </s> Corp us ĠConstruction Ġ: :: ĠPre processing Ġand Ġfeature Ġextraction Ġ: :: ĠEEG </s> The ĠEEG Ġdata Ġshared Ġin Ġthis Ġproject Ġare Ġavailable Ġas Ġraw Ġdata , Ġbut Ġalso Ġpre process ed Ġwith ĠAutom agic Ġ( version Ġ1 . 4 . 6 , Ġped ron i 2019 aut om agic ), Ġa Ġtool Ġfor Ġautomatic ĠEEG Ġdata Ġcleaning Ġand Ġvalidation . Ġ105 ĠEEG Ġchannels Ġ( i . e . Ġelectrodes ) Ġwere Ġused Ġfrom Ġthe Ġscalp Ġrecordings . Ġ9 ĠE OG Ġchannels Ġwere Ġused Ġfor Ġartifact Ġremoval Ġand Ġadditional Ġ14 Ġchannels Ġlying Ġmainly Ġon Ġthe Ġneck Ġand Ġface Ġwere Ġdiscarded Ġbefore Ġdata Ġanalysis . ĠBad Ġchannels Ġwere Ġidentified Ġand Ġinterpol ated . ĠWe Ġused Ġthe ĠMultiple ĠArtifact ĠRe jection ĠAl gorithm Ġ( M ARA ), Ġa Ġsupervised Ġmachine Ġlearning Ġalgorithm Ġthat Ġevaluates ĠI CA Ġcomponents , Ġfor Ġautomatic Ġartifact Ġrejection . ĠMAR A Ġhas Ġbeen Ġtrained Ġon Ġmanual Ġcomponent Ġclass ifications , Ġand Ġthus Ġcaptures Ġa Ġwide Ġrange Ġof Ġartifacts . ĠMAR A Ġis Ġespecially Ġeffective Ġat Ġdetecting Ġand Ġremoving Ġeye Ġand Ġmuscle Ġartifact Ġcomponents . ĠThe Ġeffect Ġof Ġthis Ġpre processing Ġcan Ġbe Ġseen Ġin ĠFigure ĠFIG REF 1 d . </s> After Ġpre processing , Ġwe Ġsynchronized Ġthe ĠEEG Ġand Ġeye - tracking Ġdata Ġto Ġenable ĠEEG Ġanalyses Ġtime - locked Ġto Ġthe Ġon sets Ġof Ġfix ations . ĠTo Ġcompute Ġoscill atory Ġpower Ġmeasures , Ġwe Ġband - pass Ġfiltered Ġthe Ġcontinuous ĠEEG Ġsignals Ġacross Ġan Ġentire Ġreading Ġtask Ġfor Ġfive Ġdifferent Ġfrequency Ġbands Ġresulting Ġin Ġa Ġtime - series Ġfor Ġeach Ġfrequency Ġband . ĠThe Ġindependent Ġfrequency Ġbands Ġwere Ġdetermined Ġas Ġfollows : Ġthe ta $ _ 1 $ Ġ( 4 âĢĵ 6 ĠHz ), Ġthe ta $ _ 2 $ Ġ( 6 . 5 âĢĵ 8 ĠHz ), Ġalpha $ _ 1 $ Ġ( 8 . 5 âĢĵ 10 ĠHz ), Ġalpha $ _ 2 $ Ġ( 10 . 5 âĢĵ 13 ĠHz ), Ġbeta $ _ 1 $ Ġ( 13 . 5 âĢĵ 18 ĠHz ), Ġbeta $ _ 2 $ Ġ( 18 . 5 âĢĵ 30 ĠHz ), Ġgamma $ _ 1 $ Ġ( 30 . 5 âĢĵ 40 ĠHz ), Ġand Ġgamma $ _ 2 $ Ġ( 40 âĢĵ 49 . 5 ĠHz ). ĠWe Ġthen Ġapplied Ġa ĠHilbert Ġtransformation Ġto Ġeach Ġof Ġthese Ġtime - series . ĠWe Ġspecifically Ġchose Ġthe ĠHilbert Ġtransformation Ġto Ġmaintain Ġthe Ġtemporal Ġinformation Ġof Ġthe Ġamplitude Ġof Ġthe Ġfrequency Ġbands , Ġto Ġenable Ġthe Ġpower Ġof Ġthe Ġdifferent Ġfrequencies Ġfor Ġtime Ġsegments Ġdefined Ġthrough Ġthe Ġfix ations Ġfrom Ġthe Ġeye - tracking Ġrecording . ĠThus , Ġfor Ġeach Ġeye - tracking Ġfeature Ġwe Ġcomputed Ġthe Ġcorresponding ĠEEG Ġfeature Ġin Ġeach Ġfrequency Ġband . ĠFurthermore , Ġwe Ġextracted Ġsentence - level ĠEEG Ġfeatures Ġby Ġcalculating Ġthe Ġpower Ġin Ġeach Ġfrequency Ġband , Ġand Ġadditionally , Ġthe Ġdifference Ġof Ġthe Ġpower Ġspect ra Ġbetween Ġfrontal Ġleft Ġand Ġright Ġhom ologue Ġelectrodes Ġpairs . ĠFor Ġeach Ġeye - tracking Ġbased ĠEEG Ġfeature , Ġall Ġchannels Ġwere Ġsubject Ġto Ġan Ġartifact Ġrejection Ġcriterion Ġof Ġ$ 90 \ mu ĠV $ Ġto Ġexclude Ġtrials Ġwith Ġtransient Ġnoise . </s> Data ĠVal idation </s> The Ġaim Ġof Ġthe Ġtechnical Ġvalidation Ġof Ġthe Ġdata Ġis Ġto Ġguarantee Ġgood Ġrecording Ġquality Ġand Ġto Ġreplicate Ġfindings Ġof Ġprevious Ġstudies Ġinvestigating Ġco - reg istration Ġof ĠEEG Ġand Ġeye Ġmovement Ġdata Ġduring Ġnatural Ġreading Ġtasks Ġ( e . g . Ġdim igen 2011 core g istration ). ĠWe Ġalso Ġcompare Ġthe Ġresults Ġto ĠZ u Co Ġ1 . 0 ĠB IB REF 1 , Ġwhich Ġallows Ġa Ġmore Ġdirect Ġcomparison Ġdue Ġto Ġthe Ġanalogous Ġrecording Ġprocedure . </s> Data ĠVal idation Ġ: :: ĠEye - tracking </s> We Ġvalidated Ġthe Ġrecorded Ġeye - tracking Ġdata Ġby Ġanalyzing Ġthe Ġfix ations Ġmade Ġby Ġall Ġsubjects Ġthrough Ġtheir Ġreading Ġspeed Ġand Ġomission Ġrate Ġon Ġsentence Ġlevel . ĠThe Ġomission Ġrate Ġis Ġdefined Ġas Ġthe Ġpercentage Ġof Ġwords Ġthat Ġis Ġnot Ġfix ated Ġin Ġa Ġsentence . ĠFigure ĠFIG REF 10 Ġ( middle ) Ġshows Ġthe Ġmean Ġreading Ġspeed Ġover Ġall Ġsubjects , Ġmeasured Ġin Ġseconds Ġper Ġsentence Ġand ĠFigure ĠFIG REF 10 Ġ( right ) Ġshows Ġthe Ġmean Ġomission Ġrates Ġaggreg ated Ġover Ġall Ġsubjects Ġfor Ġeach Ġtask . ĠClearly , Ġthe Ġparticipants Ġmade Ġless Ġfix ations Ġduring Ġthe Ġtask - specific Ġreading , Ġwhich Ġlead Ġto Ġfaster Ġreading Ġspeed . </s> Moreover , Ġwe Ġcorrobor ated Ġthese Ġsentence - level Ġmetrics Ġby Ġvisual izing Ġthe Ġskipping Ġproportion Ġon Ġword Ġlevel Ġ( Figure ĠFIG REF 13 ). ĠThe Ġskipping Ġproportion Ġis Ġthe Ġaverage Ġrate Ġof Ġwords Ġbeing Ġskipped Ġ( i . e . Ġnot Ġbeing Ġfix ated ) Ġin Ġa Ġsentence . ĠAs Ġexpected , Ġthis Ġalso Ġincreases Ġin Ġthe Ġtask - specific Ġreading . </s> Although Ġthe Ġreading Ġmaterial Ġis Ġfrom Ġthe Ġsame Ġsource Ġand Ġof Ġthe Ġsame Ġlength Ġrange Ġ( see ĠFigure ĠFIG REF 10 Ġ( left )), Ġin Ġthe Ġfirst Ġtask Ġ( NR ) Ġpassive Ġreading Ġwas Ġrecorded , Ġwhile Ġin Ġthe Ġsecond Ġtask Ġ( TS R ) Ġthe Ġsubjects Ġhad Ġto Ġannot ate Ġa Ġspecific Ġrelation Ġtype Ġin Ġeach Ġsentence . ĠThus , Ġthe Ġtask - specific Ġannotation Ġreading Ġlead Ġto Ġshorter Ġpasses Ġbecause Ġthe Ġgoal Ġwas Ġmerely Ġto Ġrecognize Ġa Ġrelation Ġin Ġthe Ġtext , Ġbut Ġnot Ġnecessarily Ġto Ġprocess Ġthe Ġevery Ġword Ġin Ġeach Ġsentence . ĠThis Ġdistinct Ġreading Ġbehavior Ġis Ġshown Ġin ĠFigure ĠFIG REF 15 , Ġwhere Ġfix ations Ġoccur Ġuntil Ġthe Ġend Ġof Ġthe Ġsentence Ġduring Ġnormal Ġreading , Ġwhile Ġduring Ġtask - specific Ġreading Ġthe Ġfix ations Ġstop Ġafter Ġthe Ġdecisive Ġwords Ġto Ġdetect Ġa Ġgiven Ġrelation Ġtype . ĠFinally , Ġwe Ġalso Ġanalyzed Ġthe Ġaverage Ġreading Ġtimes Ġfor Ġeach Ġof Ġthe Ġextracted Ġeye - tracking Ġfeatures . ĠThe Ġmeans Ġand Ġdistributions Ġfor Ġboth Ġtasks Ġare Ġshown Ġin ĠFigure ĠFIG REF 21 . ĠThese Ġresults Ġare Ġin Ġline Ġwith Ġthe Ġrecorded Ġdata Ġin ĠZ u Co Ġ1 . 0 , Ġas Ġwell Ġas Ġwith Ġthe Ġfeatures Ġextracted Ġin Ġthe ĠG EC O Ġcorpus ĠB IB REF 4 . </s> Data ĠVal idation Ġ: :: ĠEEG </s> As Ġa Ġfirst Ġvalidation Ġstep , Ġwe Ġextracted Ġfixation - related Ġpotential s Ġ( FR Ps ), Ġwhere Ġthe ĠEEG Ġsignal Ġduring Ġall Ġfix ations Ġof Ġone Ġtask Ġare Ġaveraged . ĠFigure ĠFIG REF 24 Ġshows Ġthe Ġtime - series Ġof Ġthe Ġresulting ĠFR Ps Ġfor Ġtwo Ġelectrodes Ġ( PO 8 Ġand ĠC z ), Ġas Ġwell Ġas Ġtop ographies Ġof Ġthe Ġvoltage Ġdistributions Ġacross Ġthe Ġscalp Ġat Ġselected Ġpoints Ġin Ġtime . ĠThe Ġfive Ġcomponents Ġ( for Ġwhich Ġthe Ġscalp Ġtop ographies Ġare Ġplotted ) Ġare Ġhighly Ġsimilar Ġin Ġthe Ġtime - course Ġof Ġthe Ġchosen Ġelectrodes Ġto Ġdim igen 2011 core g istration Ġas Ġwell Ġas Ġto ĠZ u Co Ġ1 . 0 . </s> Moreover , Ġthese Ġprevious Ġstudies Ġwere Ġable Ġto Ġshow Ġan Ġeffect Ġof Ġfixation Ġduration Ġon Ġthe Ġresulting ĠFR Ps . ĠTo Ġshow Ġthis Ġdependency Ġwe Ġfollowed Ġtwo Ġapproaches . ĠFirst , Ġfor Ġeach Ġreading Ġtask , Ġall Ġsingle - trial ĠFR Ps Ġwere Ġordered Ġby Ġfixation Ġduration Ġand Ġa Ġvertical Ġsliding Ġtime - window Ġwas Ġused Ġto Ġsmooth Ġthe Ġdata ĠB IB REF 13 . ĠFigure ĠFIG REF 25 Ġ( bottom ) Ġshows Ġthe Ġresulting Ġplots . ĠIn Ġline Ġwith Ġthis Ġprevious Ġwork , Ġa Ġfirst Ġposit ivation Ġcan Ġbe Ġidentified Ġat Ġ100 Ġms Ġpost - fix ation Ġonset . ĠA Ġsecond Ġpositive Ġpeak Ġis Ġlocated Ġdependent Ġon Ġthe Ġduration Ġof Ġthe Ġfixation , Ġwhich Ġcan Ġbe Ġexplained Ġby Ġthe Ġtime - j ittered Ġsucceeding Ġfixation . ĠThe Ġsecond Ġapproach Ġis Ġbased Ġon Ġhe nd erson 2013 co Ġin Ġwhich Ġsingle Ġtrial ĠEEG Ġsegments Ġare Ġclustered Ġby Ġthe Ġduration Ġof Ġthe Ġcurrent Ġfixation . ĠAs Ġshown Ġin ĠFigure ĠFIG REF 25 Ġ( top ), Ġwe Ġchose Ġfour Ġclusters Ġand Ġaveraged Ġthe Ġdata Ġwithin Ġeach Ġcluster Ġto Ġfour Ġdistinct ĠFR Ps , Ġdepending Ġon Ġthe Ġfixation Ġduration . ĠAgain , Ġthe Ġsame Ġposit ivation Ġpeaks Ġbecome Ġapparent . ĠBoth Ġfindings Ġare Ġconsistent Ġwith Ġthe Ġprevious Ġwork Ġmentioned Ġand Ġwith Ġour Ġfindings Ġfrom ĠZ u Co Ġ1 . 0 . </s> Conclusion </s> We Ġpresented Ġa Ġnew , Ġfreely Ġavailable Ġcorpus Ġof Ġeye Ġmovement Ġand Ġelectrical Ġbrain Ġactivity Ġrecordings Ġduring Ġnatural Ġreading Ġas Ġwell Ġas Ġduring Ġannotation . ĠThis Ġis Ġthe Ġfirst Ġdataset Ġthat Ġallows Ġfor Ġthe Ġcomparison Ġbetween Ġthese Ġtwo Ġreading Ġparad ig ms . ĠWe Ġdescribed Ġthe Ġmaterials Ġand Ġexperiment Ġdesign Ġin Ġdetail Ġand Ġconducted Ġan Ġextensive Ġvalidation Ġto Ġensure Ġthe Ġquality Ġof Ġthe Ġrecorded Ġdata . ĠSince Ġthis Ġcorpus Ġis Ġtailored Ġto Ġcognitive ly - inspired ĠN LP , Ġthe Ġapplications Ġand Ġre - use Ġpotential s Ġof Ġthis Ġdata Ġare Ġextensive . ĠThe Ġprovided Ġword - level Ġand Ġsentence - level Ġeye - tracking Ġand ĠEEG Ġfeatures Ġcan Ġbe Ġused Ġto Ġimprove Ġand Ġevaluate ĠN LP Ġand Ġmachine Ġlearning Ġmethods , Ġfor Ġinstance , Ġto Ġevaluate Ġlinguistic Ġphenomena Ġin Ġneural Ġmodels Ġvia Ġpsych ol ingu istic Ġdata . ĠIn Ġaddition , Ġbecause Ġthe Ġsentences Ġcontains Ġsemantic Ġrelation Ġlabels Ġand Ġthe Ġannotations Ġof Ġall Ġparticipants , Ġit Ġcan Ġalso Ġbe Ġwidely Ġused Ġfor Ġrelation Ġextraction Ġand Ġclassification . ĠFinally , Ġthe Ġtwo Ġcarefully Ġconstructed Ġreading Ġparad ig ms Ġallow Ġfor Ġthe Ġcomparison Ġbetween Ġnormal Ġreading Ġand Ġreading Ġduring Ġannotation , Ġwhich Ġcan Ġbe Ġrelevant Ġto Ġimprove Ġthe Ġmanual Ġlab elling Ġprocess Ġas Ġwell Ġas Ġthe Ġquality Ġof Ġthe Ġannotations Ġfor Ġsupervised Ġmachine Ġlearning .