paragraphs, lines, Information on physical page segmentation (for PDFs only), paragraph segmentation, and line segmentation
EXTRACTIVE
paragraph, lines, textspan element (paragraph segmentation, line segmentation, Information on physical page segmentation(for PDF only))
ABSTRACTIVE
The paper at hand introduces a corpus developed for use in automatic readability assessment and automatic text simplification of German. The focus of this publication is on representing information that is valuable for these tasks but that hitherto has largely been ignored in machine learning approaches centering around simplified language, specifically, text structure (e.g., paragraphs, lines), typography (e.g., font type, font style), and image (content, position, and dimensions) information. The importance of considering such information has repeatedly been asserted theoretically BIBREF11, BIBREF12, BIBREF0. The remainder of this paper is structured as follows: Section SECREF2 presents previous corpora used for automatic readability assessment and text simplification. Section SECREF3 describes our corpus, introducing its novel aspects and presenting the primary data (Section SECREF7), the metadata (Section SECREF10), the secondary data (Section SECREF28), the profile (Section SECREF35), and the results of machine learning experiments carried out on the corpus (Section SECREF37). Information on physical page segmentation (for PDFs only), paragraph segmentation, and line segmentation was added as part of a textspan element in the textstructure layer
The paper at hand introduces a corpus developed for use in automatic readability assessment and automatic text simplification of German. The focus of this publication is on representing information that is valuable for these tasks but that hitherto has largely been ignored in machine learning approaches centering around simplified language, specifically, text structure (e.g., paragraphs, lines), typography (e.g., font type, font style), and image (content, position, and dimensions) information. The importance of considering such information has repeatedly been asserted theoretically BIBREF11, BIBREF12, BIBREF0. The remainder of this paper is structured as follows: Section SECREF2 presents previous corpora used for automatic readability assessment and text simplification. Section SECREF3 describes our corpus, introducing its novel aspects and presenting the primary data (Section SECREF7), the metadata (Section SECREF10), the secondary data (Section SECREF28), the profile (Section SECREF35), and the results of machine learning experiments carried out on the corpus (Section SECREF37). Information on physical page segmentation (for PDFs only), paragraph segmentation, and line segmentation was added as part of a textspan element in the textstructure layer
1909.09067
Introduction </s> Sim pl ified Ġlanguage Ġis Ġa Ġvariety Ġof Ġstandard Ġlanguage Ġcharacterized Ġby Ġreduced Ġlex ical Ġand Ġsynt actic Ġcomplexity , Ġthe Ġaddition Ġof Ġexplanations Ġfor Ġdifficult Ġconcepts , Ġand Ġclearly Ġstructured Ġlayout . ĠAmong Ġthe Ġtarget Ġgroups Ġof Ġsimplified Ġlanguage Ġcommonly Ġmentioned Ġare Ġpersons Ġwith Ġcognitive Ġimpairment Ġor Ġlearning Ġdisabilities , Ġpre ling ually Ġdeaf Ġpersons , Ġfunctionally Ġilliter ate Ġpersons , Ġand Ġforeign Ġlanguage Ġlearners ĠB IB REF 0 . </s> Two Ġnatural Ġlanguage Ġprocessing Ġtasks Ġdeal Ġwith Ġthe Ġconcept Ġof Ġsimplified Ġlanguage : Ġautomatic Ġread ability Ġassessment Ġand Ġautomatic Ġtext Ġsimpl ification . ĠRead ability Ġassessment Ġrefers Ġto Ġthe Ġprocess Ġof Ġdetermining Ġthe Ġlevel Ġof Ġdifficulty Ġof Ġa Ġtext , Ġe . g ., Ġalong Ġread ability Ġmeasures , Ġschool Ġgrades , Ġor Ġlevels Ġof Ġthe ĠCommon ĠEuropean ĠFramework Ġof ĠReference Ġfor ĠLanguages Ġ( CE FR ) ĠB IB REF 1 . ĠRead ability Ġmeasures , Ġin Ġtheir Ġtraditional Ġform , Ġtake Ġinto Ġaccount Ġonly Ġsurface Ġfeatures . ĠFor Ġexample , Ġthe ĠF les ch ĠReading ĠE ase ĠScore ĠB IB REF 2 Ġmeasures Ġthe Ġlength Ġof Ġwords Ġ( in Ġsyll ables ) Ġand Ġsentences Ġ( in Ġwords ). ĠWhile Ġread ability Ġhas Ġbeen Ġshown Ġto Ġcorrelate Ġwith Ġsuch Ġfeatures Ġto Ġsome Ġextent ĠB IB REF 3 , Ġa Ġconsensus Ġhas Ġemerged Ġaccording Ġto Ġwhich Ġthey Ġare Ġnot Ġsufficient Ġto Ġaccount Ġfor Ġall Ġof Ġthe Ġcomplexity Ġinherent Ġin Ġa Ġtext . ĠAs Ġ[ p . Ġ26 18 ] k au ch ak - et - al - 2014 Ġstate , ĠâĢ ľ the Ġusability Ġof Ġread ability Ġformulas Ġis Ġlimited Ġand Ġthere Ġis Ġlittle Ġevidence Ġthat Ġthe Ġoutput Ġof Ġthese Ġtools Ġdirectly Ġresults Ġin Ġimproved Ġunderstanding Ġby Ġreaders âĢ Ŀ . ĠRecently , Ġmore Ġsophisticated Ġmodels Ġemploying Ġ( d eeper ) Ġlinguistic Ġfeatures Ġsuch Ġas Ġlex ical , Ġsemantic , Ġmorph ological , Ġmorph os ynt actic , Ġsynt actic , Ġpragmatic , Ġdiscourse , Ġpsych ol ingu istic , Ġand Ġlanguage Ġmodel Ġfeatures Ġhave Ġbeen Ġproposed ĠB IB REF 4 , ĠB IB REF 5 , ĠB IB REF 6 , ĠB IB REF 7 , ĠB IB REF 8 . </s> Aut omatic Ġtext Ġsimpl ification Ġwas Ġinitiated Ġin Ġthe Ġlate Ġ1990 s ĠB IB REF 9 , ĠB IB REF 10 Ġand Ġsince Ġthen Ġhas Ġbeen Ġapproached Ġby Ġmeans Ġof Ġrule - based Ġand Ġstatistical Ġmethods . ĠAs Ġpart Ġof Ġa Ġrule - based Ġapproach , Ġthe Ġoperations Ġcarried Ġout Ġtypically Ġinclude Ġreplacing Ġcomplex Ġlex ical Ġand Ġsynt actic Ġunits Ġby Ġsimpler Ġones . ĠA Ġstatistical Ġapproach Ġgenerally Ġconceptual izes Ġthe Ġsimpl ification Ġtask Ġas Ġone Ġof Ġconverting Ġa Ġstandard - language Ġinto Ġa Ġsimplified - language Ġtext Ġusing Ġmachine Ġtranslation . Ġn is io i - et - al - 2017 Ġintroduced Ġneural Ġmachine Ġtranslation Ġto Ġautomatic Ġtext Ġsimpl ification . ĠResearch Ġon Ġautomatic Ġtext Ġsimpl ification Ġis Ġcomparatively Ġwidespread Ġfor Ġlanguages Ġsuch Ġas ĠEnglish , ĠSwedish , ĠSpanish , Ġand ĠBrazilian ĠPortuguese . ĠTo Ġthe Ġauthors ' Ġknowledge , Ġno Ġproductive Ġsystem Ġexists Ġfor ĠGerman . Ġs uter - 2015 , Ġs uter - et - al - 2016 Ġpresented Ġa Ġprototype Ġof Ġa Ġrule - based Ġsystem Ġfor ĠGerman . </s> Machine Ġlearning Ġapproaches Ġto Ġboth Ġread ability Ġassessment Ġand Ġtext Ġsimpl ification Ġrely Ġon Ġdata Ġsystematically Ġprepared Ġin Ġthe Ġform Ġof Ġcorpor a . ĠSpecifically , Ġfor Ġautomatic Ġtext Ġsimpl ification Ġvia Ġmachine Ġtranslation , Ġpairs Ġof Ġstandard - language / sim pl ified - language Ġtexts Ġaligned Ġat Ġthe Ġsentence Ġlevel Ġ( i . e ., Ġparallel Ġcorpor a ) Ġare Ġneeded . </s> The Ġpaper Ġat Ġhand Ġintroduces Ġa Ġcorpus Ġdeveloped Ġfor Ġuse Ġin Ġautomatic Ġread ability Ġassessment Ġand Ġautomatic Ġtext Ġsimpl ification Ġof ĠGerman . ĠThe Ġfocus Ġof Ġthis Ġpublication Ġis Ġon Ġrepresenting Ġinformation Ġthat Ġis Ġvaluable Ġfor Ġthese Ġtasks Ġbut Ġthat Ġhitherto Ġhas Ġlargely Ġbeen Ġignored Ġin Ġmachine Ġlearning Ġapproaches Ġcent ering Ġaround Ġsimplified Ġlanguage , Ġspecifically , Ġtext Ġstructure Ġ( e . g ., Ġparagraphs , Ġlines ), Ġtyp ography Ġ( e . g ., Ġfont Ġtype , Ġfont Ġstyle ), Ġand Ġimage Ġ( content , Ġposition , Ġand Ġdimensions ) Ġinformation . ĠThe Ġimportance Ġof Ġconsidering Ġsuch Ġinformation Ġhas Ġrepeatedly Ġbeen Ġasserted Ġtheoretically ĠB IB REF 11 , ĠB IB REF 12 , ĠB IB REF 0 . ĠThe Ġremainder Ġof Ġthis Ġpaper Ġis Ġstructured Ġas Ġfollows : ĠSection ĠSEC REF 2 Ġpresents Ġprevious Ġcorpor a Ġused Ġfor Ġautomatic Ġread ability Ġassessment Ġand Ġtext Ġsimpl ification . ĠSection ĠSEC REF 3 Ġdescribes Ġour Ġcorpus , Ġintroducing Ġits Ġnovel Ġaspects Ġand Ġpresenting Ġthe Ġprimary Ġdata Ġ( Section ĠSEC REF 7 ), Ġthe Ġmetadata Ġ( Section ĠSEC REF 10 ), Ġthe Ġsecondary Ġdata Ġ( Section ĠSEC REF 28 ), Ġthe Ġprofile Ġ( Section ĠSEC REF 35 ), Ġand Ġthe Ġresults Ġof Ġmachine Ġlearning Ġexperiments Ġcarried Ġout Ġon Ġthe Ġcorpus Ġ( Section ĠSEC REF 37 ). </s> Previous ĠCorpor a Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification </s> A Ġnumber Ġof Ġcorpor a Ġfor Ġuse Ġin Ġautomatic Ġread ability Ġassessment Ġand Ġautomatic Ġtext Ġsimpl ification Ġexist . ĠThe Ġmost Ġwell - known Ġexample Ġis Ġthe ĠParallel ĠWikipedia ĠSimpl ification ĠCorpus Ġ( P W K P ) Ġcompiled Ġfrom Ġparallel Ġarticles Ġof Ġthe ĠEnglish ĠWikipedia Ġand ĠSimple ĠEnglish ĠWikipedia ĠB IB REF 13 Ġand Ġconsisting Ġof Ġaround Ġ108 , 000 Ġsentence Ġpairs . ĠThe Ġcorpus Ġprofile Ġis Ġshown Ġin ĠTable ĠT AB REF 2 . ĠWhile Ġthe Ġcorpus Ġrepresents Ġthe Ġlargest Ġdataset Ġinvolving Ġsimplified Ġlanguage Ġto Ġdate , Ġits Ġapplication Ġhas Ġbeen Ġcriticized Ġfor Ġvarious Ġreasons ĠB IB REF 15 , ĠB IB REF 14 , ĠB IB REF 16 ; Ġamong Ġthese , Ġthe Ġfact Ġthat ĠSimple ĠEnglish ĠWikipedia Ġarticles Ġare Ġnot Ġnecessarily Ġdirect Ġtranslations Ġof Ġarticles Ġfrom Ġthe ĠEnglish ĠWikipedia Ġstands Ġout . Ġh wang - et - al - 2015 Ġprovided Ġan Ġupdated Ġversion Ġof Ġthe Ġcorpus Ġthat Ġincludes Ġa Ġtotal Ġof Ġ280 , 000 Ġfull Ġand Ġpartial Ġmatches Ġbetween Ġthe Ġtwo ĠWikipedia Ġversions . ĠAnother Ġfrequently Ġused Ġdata Ġcollection Ġfor ĠEnglish Ġis Ġthe ĠNew sel a ĠCorpus ĠB IB REF 14 Ġconsisting Ġof Ġ1 , 130 Ġnews Ġarticles , Ġeach Ġsimplified Ġinto Ġfour Ġschool Ġgrade Ġlevels Ġby Ġprofessional Ġeditors . ĠTable ĠT AB REF 3 Ġshows Ġthe Ġprofile Ġof Ġthe ĠNew sel a ĠCorpus . ĠThe Ġtable Ġob vi ates Ġthat Ġthe Ġdifference Ġin Ġvocabulary Ġsize Ġbetween Ġthe ĠEnglish Ġand Ġthe Ġsimplified ĠEnglish Ġside Ġof Ġthe ĠPW K P ĠCorpus Ġamounts Ġto Ġonly Ġ18 %, Ġwhile Ġthe Ġcorresponding Ġnumber Ġfor Ġthe ĠEnglish Ġside Ġand Ġthe Ġlevel Ġrepresenting Ġthe Ġhighest Ġamount Ġof Ġsimpl ification Ġin Ġthe ĠNew sel a ĠCorpus Ġ( Simple - 4 ) Ġis Ġ50 . 8 %. ĠVoc abulary Ġsize Ġas Ġan Ġindicator Ġof Ġlex ical Ġrichness Ġis Ġgenerally Ġtaken Ġto Ġcorrelate Ġpositively Ġwith Ġcomplexity ĠB IB REF 17 . </s> gas per in - et - al - 2010 Ġcompiled Ġthe ĠPor Sim ples ĠCorpus Ġconsisting Ġof ĠBrazilian ĠPortuguese Ġtexts Ġ( 2 , 116 Ġsentences ), Ġeach Ġwith Ġa Ġnatural Ġand Ġa Ġstrong Ġsimpl ification , Ġresulting Ġin Ġaround Ġ4 , 500 Ġaligned Ġsentences . Ġdr nd are vic - s agg ion - 2012 , Ġbott - et - al - 2012 , Ġbott - s agg ion - 2012 Ġproduced Ġthe ĠSimple xt ĠCorpus Ġconsisting Ġof Ġ200 ĠSpanish / sim pl ified ĠSpanish Ġdocument Ġpairs , Ġamount ing Ġto Ġa Ġtotal Ġof Ġ1 , 149 Ġ( Spanish )/ 1 , 808 Ġ( sim pl ified ĠSpanish ) Ġsentences Ġ( approximately Ġ1 , 000 Ġaligned Ġsentences ). </s> kl aper - eb ling - vol k - 2013 Ġcreated Ġthe Ġfirst Ġparallel Ġcorpus Ġfor ĠGerman / sim pl ified ĠGerman , Ġconsisting Ġof Ġ256 Ġparallel Ġtexts Ġdownloaded Ġfrom Ġthe Ġweb Ġ( approximately Ġ70 , 000 Ġtokens ). </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman </s> Section ĠSEC REF 2 Ġdemonstrated Ġthat Ġthe Ġonly Ġcorpus Ġcontaining Ġsimplified ĠGerman Ġavailable Ġis Ġthat Ġof Ġk l aper - eb ling - vol k - 2013 . ĠSince Ġits Ġcreation , Ġa Ġnumber Ġof Ġlegal Ġand Ġpolitical Ġdevelopments Ġhave Ġspurred Ġthe Ġavailability Ġof Ġdata Ġin Ġsimplified ĠGerman . ĠAmong Ġthese Ġdevelopments Ġis Ġthe Ġintroduction Ġof Ġa Ġset Ġof Ġregulations Ġfor Ġaccessible Ġinformation Ġtechnology Ġ( Bar ri ere fre ie - Information ste chn ik - Ver ord n ung , ĠBIT V Ġ2 . 0 ) Ġin ĠGermany Ġand Ġthe Ġratification Ġof Ġthe ĠUnited ĠNations ĠConvention Ġon Ġthe ĠRights Ġof ĠPersons Ġwith ĠDis abilities Ġ( CR PD ) Ġin ĠSwitzerland . ĠThe Ġpaper Ġat Ġhand Ġintroduces Ġa Ġcorpus Ġthat Ġrepresents Ġan Ġenhancement Ġof Ġthe Ġcorpus Ġof Ġk l aper - eb ling - vol k - 2013 Ġin Ġthe Ġfollowing Ġways : </s> The Ġcorpus Ġcontains Ġmore Ġparallel Ġdata . </s> The Ġcorpus Ġadditionally Ġcontains Ġmon oling ual - only Ġdata Ġ( sim pl ified ĠGerman ). </s> The Ġcorpus Ġnewly Ġcontains Ġinformation Ġon Ġtext Ġstructure , Ġtyp ography , Ġand Ġimages . </s> The Ġsimplified ĠGerman Ġside Ġof Ġthe Ġparallel Ġdata Ġtogether Ġwith Ġthe Ġmon oling ual - only Ġdata Ġcan Ġbe Ġused Ġfor Ġautomatic Ġread ability Ġassessment . ĠThe Ġparallel Ġdata Ġin Ġthe Ġcorpus Ġis Ġuseful Ġboth Ġfor Ġder iving Ġrules Ġfor Ġa Ġrule - based Ġtext Ġsimpl ification Ġsystem Ġin Ġa Ġdata - driven Ġmanner Ġand Ġfor Ġtraining Ġa Ġdata - driven Ġmachine Ġtranslation Ġsystem . ĠA Ġdata Ġaug mentation Ġtechnique Ġsuch Ġas Ġback - translation ĠB IB REF 18 Ġcan Ġbe Ġapplied Ġto Ġthe Ġmon oling ual - only Ġdata Ġto Ġarrive Ġat Ġadditional Ġ( sy nt hetic ) Ġparallel Ġdata . </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠPrimary ĠData </s> The Ġcorpus Ġcontains ĠPDF s Ġand Ġweb pages Ġcollected Ġfrom Ġweb Ġsources Ġin ĠGermany , ĠAustria , Ġand ĠSwitzerland Ġat Ġthe Ġend Ġof Ġ2018 / begin ning Ġof Ġ2019 . ĠThe Ġweb Ġsources Ġmostly Ġconsist Ġof Ġwebsites Ġof Ġgovernments , Ġspecial ised Ġinstitutions , Ġtranslation Ġagencies , Ġand Ġnon - profit Ġorganisations Ġ( 92 Ġdifferent Ġdomains ). ĠThe Ġdocuments Ġcover Ġa Ġrange Ġof Ġtopics , Ġsuch Ġas Ġpolitics Ġ( e . g ., Ġinstructions Ġfor Ġvoting ), Ġhealth Ġ( e . g ., Ġwhat Ġto Ġdo Ġin Ġcase Ġof Ġpregnancy ), Ġand Ġculture Ġ( e . g ., Ġintroduction Ġto Ġart Ġmuseums ). </s> For Ġthe Ġweb pages , Ġa Ġstatic Ġdump Ġof Ġall Ġdocuments Ġwas Ġcreated . ĠFollowing Ġthis , Ġthe Ġdocuments Ġwere Ġmanually Ġchecked Ġto Ġverify Ġthe Ġlanguage . ĠThe Ġmain Ġcontent Ġwas Ġsubsequently Ġextracted , Ġi . e ., ĠHTML Ġmarkup Ġand Ġboiler plate Ġremoved Ġusing Ġthe ĠBeautiful ĠSoup Ġlibrary Ġfor ĠPython . ĠInformation Ġon Ġtext Ġstructure Ġ( e . g ., Ġparagraphs , Ġlines ) Ġand Ġtyp ography Ġ( e . g ., Ġbold face , Ġital ics ) Ġwas Ġretained . ĠSimilarly , Ġimage Ġinformation Ġ( content , Ġposition , Ġand Ġdimensions Ġof Ġan Ġimage ) Ġwas Ġpreserved . </s> For ĠPDF s , Ġthe ĠPD Fl ib ĠText Ġand ĠImage ĠExt raction ĠTool kit Ġ( T ET ) Ġwas Ġused Ġto Ġextract Ġthe Ġplain Ġtext Ġand Ġrecord Ġinformation Ġon Ġtext Ġstructure , Ġtyp ography , Ġand Ġimages . ĠThe Ġtool kit Ġproduces Ġoutput Ġin Ġan ĠXML Ġformat Ġ( T ET ML ). </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠMet adata </s> Met adata Ġwas Ġcollected Ġautomatically Ġfrom Ġthe ĠHTML Ġ( web pages ) Ġand ĠT ET ML Ġ( PDF s ) Ġfiles , Ġcomple mented Ġmanually , Ġand Ġrecorded Ġin Ġthe ĠOpen ĠLanguage ĠArchives ĠCommunity Ġ( OL AC ) ĠStandard . ĠOL AC Ġis Ġbased Ġon Ġa Ġreduced Ġversion Ġof Ġthe ĠDublin ĠCore ĠMet adata ĠElement ĠSet Ġ( DC M ES ). ĠOf Ġthe Ġ15 Ġelements Ġof Ġthis ĠâĢ ľ Simple ĠDublin ĠCore âĢ Ŀ Ġset , Ġthe Ġfollowing Ġ12 Ġwere Ġactively Ġused Ġalong Ġwith Ġcontrolled Ġvoc ab ular ies Ġof ĠOL AC Ġand ĠDublin ĠCore : </s> title : Ġtitle Ġof Ġthe Ġdocument , Ġwith Ġthe Ġlanguage Ġspecified Ġas Ġthe Ġvalue Ġof Ġan Ġxml : lang Ġattribute Ġand Ġalternatives Ġto Ġthe Ġoriginal Ġtitle Ġ( e . g ., Ġtranslations ) Ġstored Ġas Ġd ct er ms : altern ative Ġ( cf . ĠFigure ĠFIG REF 11 Ġfor Ġan Ġexample ) </s> cont ribut or : Ġall Ġperson Ġentities Ġlinked Ġto Ġthe Ġcreation Ġof Ġa Ġdocument , Ġwith Ġan Ġol ac : code Ġattribute Ġwith Ġvalues Ġfrom Ġthe ĠOL AC Ġrole Ġvocabulary Ġused Ġto Ġfurther Ġspecify Ġthe Ġrole Ġof Ġthe Ġcontributor , Ġe . g ., Ġauthor , Ġeditor , Ġpublisher , Ġor Ġtranslator </s> date : Ġdate Ġmentioned Ġin Ġthe Ġmetadata Ġof Ġthe ĠHTML Ġor ĠPDF Ġsource Ġor , Ġfor Ġnews Ġand Ġblog Ġarticles , Ġdate Ġmentioned Ġin Ġthe Ġbody Ġof Ġthe Ġtext , Ġin ĠW 3 C Ġdate Ġand Ġtime Ġformat </s> description : Ġvalue Ġof Ġthe Ġdescription Ġin Ġthe Ġmetadata Ġof Ġan ĠHTML Ġdocument Ġor Ġlist Ġof Ġsections Ġof Ġa ĠPDF Ġdocument , Ġusing Ġthe ĠDublin ĠCore Ġqualifier ĠTable Of Contents </s> format : Ġdistinction Ġbetween Ġthe ĠInternet ĠMedia ĠTypes Ġ( M IME Ġtypes ) Ġtext / html Ġ( for Ġweb pages ) Ġand Ġapplication / pdf Ġ( for ĠPDF s ) </s> ident ifier : ĠURL Ġof Ġthe Ġdocument Ġor ĠInternational ĠStandard ĠBook ĠNumber Ġ( IS BN ) Ġfor Ġbooks Ġor Ġbroch ures </s> language : Ġlanguage Ġof Ġthe Ġdocument Ġas Ġvalue Ġof Ġthe Ġattribute Ġol ac : code Ġ( i . e ., Ġde , Ġas Ġcon forming Ġto ĠISO Ġ6 39 ), Ġwith Ġthe ĠCE FR Ġlevel Ġas Ġoptional Ġelement Ġcontent </s> pub lisher : Ġorganization Ġor Ġperson Ġthat Ġmade Ġthe Ġdocument Ġavailable </s> relation : Ġused Ġto Ġestablish Ġa Ġlink Ġbetween Ġdocuments Ġin ĠGerman Ġand Ġsimplified ĠGerman Ġfor Ġthe Ġparallel Ġpart Ġof Ġthe Ġcorpus , Ġusing Ġthe ĠDublin ĠCore Ġqualifiers Ġhas Version Ġ( for Ġthe ĠGerman Ġtext ) Ġand Ġis Version Of Ġ( for Ġthe Ġsimplified ĠGerman Ġtext ) </s> rights : Ġany Ġpiece Ġof Ġinformation Ġabout Ġthe Ġrights Ġof Ġa Ġdocument , Ġas Ġfar Ġas Ġavailable Ġin Ġthe Ġsource </s> source : Ġsource Ġdocument , Ġi . e ., ĠHTML Ġfor Ġweb Ġdocuments Ġand ĠT ET ML Ġfor ĠPDF s </s> type : Ġnature Ġor Ġgenre Ġof Ġthe Ġcontent Ġof Ġthe Ġdocument , Ġwhich , Ġin Ġaccordance Ġwith Ġthe ĠDC MI ĠType ĠVoc abulary , Ġis ĠText Ġin Ġall Ġcases Ġand Ġadditionally ĠStill Image Ġin Ġcases Ġwhere Ġa Ġdocument Ġalso Ġcontains Ġimages . ĠAdditionally , Ġthe Ġlinguistic Ġtype Ġis Ġspecified Ġaccording Ġto Ġthe ĠOL AC ĠL ingu istic ĠData ĠType ĠVoc abulary , Ġas Ġeither Ġprimary _ text Ġ( app lies Ġto Ġmost Ġdocuments ) Ġor Ġlex icon Ġin Ġcases Ġwhere Ġa Ġdocument Ġrepresents Ġan Ġentry Ġof Ġa Ġsimplified Ġlanguage Ġvocabulary </s> The Ġelements Ġcoverage Ġ( to Ġdenote Ġthe Ġspatial Ġor Ġtemporal Ġscope Ġof Ġthe Ġcontent Ġof Ġa Ġresource ), Ġcreator Ġ( to Ġdenote Ġthe Ġauthor Ġof Ġa Ġtext , Ġsee Ġcontributor Ġabove ), Ġand Ġsubject Ġ( to Ġdenote Ġthe Ġtopic Ġof Ġthe Ġdocument Ġcontent ) Ġwere Ġnot Ġused . </s> Figure ĠFIG REF 11 Ġshows Ġan Ġexample Ġof ĠOL AC Ġmetadata . ĠThe Ġsource Ġdocument Ġdescribed Ġwith Ġthis Ġmetadata Ġrecord Ġis Ġa ĠPDF Ġstructured Ġinto Ġchapters , Ġwith Ġtext Ġcorresponding Ġto Ġthe ĠCE FR Ġlevel ĠA 2 Ġand Ġimages . ĠMet adata Ġin ĠOL AC Ġcan Ġbe Ġconverted Ġinto Ġthe Ġmetadata Ġstandard Ġof ĠCL AR IN Ġ( a ĠEuropean Ġresearch Ġinfrastructure Ġfor Ġlanguage Ġresources Ġand Ġtechnology ), Ġthe ĠComponent ĠMeta Data ĠInfrastructure Ġ( C MD I ). ĠThe ĠC MD I Ġstandard Ġwas Ġchosen Ġsince Ġit Ġis Ġthe Ġsupported Ġmetadata Ġversion Ġof ĠCL AR IN , Ġwhich Ġis Ġspecifically Ġpopular Ġin ĠGerman - speaking Ġcountries . </s> Information Ġon Ġthe Ġlanguage Ġlevel Ġof Ġa Ġsimplified ĠGerman Ġtext Ġ( typically ĠA 1 , ĠA 2 , Ġor ĠB 1 ) Ġis Ġparticularly Ġvaluable , Ġas Ġit Ġallows Ġfor Ġconducting Ġautomatic Ġread ability Ġassessment Ġand Ġgraded Ġautomatic Ġtext Ġsimpl ification Ġexperiments Ġon Ġthe Ġdata . Ġ52 Ġwebsites Ġand Ġ233 ĠPDF s Ġ( amount ing Ġto Ġapproximately Ġ26 , 000 Ġsentences ) Ġhave Ġan Ġexplicit Ġlanguage Ġlevel Ġlabel . </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠSecondary ĠData </s> An notations Ġwere Ġadded Ġin Ġthe ĠText ĠCorpus ĠFormat Ġby ĠWeb L icht Ġ( TC F ) Ġdeveloped Ġas Ġpart Ġof ĠCL AR IN . ĠTC F Ġsupports Ġstandoff Ġannotation , Ġwhich Ġallows Ġfor Ġrepresentation Ġof Ġannotations Ġwith Ġconflicting Ġhierarch ies . ĠTC F Ġdoes Ġnot Ġassign Ġa Ġseparate Ġfile Ġfor Ġeach Ġannotation Ġlayer ; Ġinstead , Ġthe Ġsource Ġtext Ġand Ġall Ġannotation Ġlayers Ġare Ġstored Ġjointly Ġin Ġa Ġsingle Ġfile . ĠA Ġtoken Ġlayer Ġacts Ġas Ġthe Ġkey Ġelement Ġto Ġwhich Ġall Ġother Ġannotation Ġlayers Ġare Ġlinked . </s> The Ġfollowing Ġtypes Ġof Ġannotations Ġwere Ġadded : Ġtext Ġstructure , Ġfonts , Ġimages , Ġtokens , Ġparts Ġof Ġspeech , Ġmorph ological Ġunits , Ġle mm as , Ġsentences , Ġand Ġdependency Ġpars es . ĠTC F Ġdoes Ġnot Ġreadily Ġaccommodate Ġthe Ġincorporation Ġof Ġall Ġof Ġthese Ġtypes Ġof Ġinformation . ĠWe Ġtherefore Ġextended Ġthe Ġformat Ġin Ġthe Ġfollowing Ġways : </s> Information Ġon Ġthe Ġfont Ġtype Ġand Ġfont Ġstyle Ġ( e . g ., Ġital ics , Ġbold Ġprint ) Ġof Ġa Ġtoken Ġand Ġits Ġposition Ġon Ġthe Ġphysical Ġpage Ġ( for ĠPDF s Ġonly ) Ġwas Ġspecified Ġas Ġattributes Ġto Ġthe Ġtoken Ġelements Ġof Ġthe Ġtokens Ġlayer Ġ( cf . ĠFigure ĠFIG REF 34 Ġfor Ġan Ġexample ) </s> Information Ġon Ġphysical Ġpage Ġsegment ation Ġ( for ĠPDF s Ġonly ), Ġparagraph Ġsegment ation , Ġand Ġline Ġsegment ation Ġwas Ġadded Ġas Ġpart Ġof Ġa Ġtext span Ġelement Ġin Ġthe Ġtext st ructure Ġlayer </s> A Ġseparate Ġimages Ġlayer Ġwas Ġintroduced Ġto Ġhold Ġimage Ġelements Ġthat Ġtake Ġas Ġattributes Ġthe Ġx Ġand Ġy Ġcoordinates Ġof Ġthe Ġimages , Ġtheir Ġdimensions Ġ( width Ġand Ġheight ), Ġand Ġthe Ġnumber Ġof Ġthe Ġpage Ġon Ġwhich Ġthey Ġoccur </s> A Ġseparate Ġfonts Ġlayer Ġwas Ġintroduced Ġto Ġpreserve Ġdetailed Ġinformation Ġon Ġthe Ġfont Ġconfigurations Ġreferenced Ġin Ġthe Ġtokens Ġlayer </s> L ingu istic Ġannotation Ġwas Ġadded Ġautomatically Ġusing Ġthe ĠPar Z u Ġdependency Ġparser Ġfor ĠGerman ĠB IB REF 19 Ġ( for Ġtokens Ġand Ġdependency Ġpars es ), Ġthe ĠNL T K Ġtool kit ĠB IB REF 20 Ġ( for Ġsentences ), Ġthe ĠTree T agger ĠB IB REF 21 Ġ( for Ġpart - of - speech Ġtags Ġand Ġle mm as ), Ġand ĠZ m orge ĠB IB REF 22 Ġ( for Ġmorph ological Ġunits ). ĠFigure ĠFIG REF 34 Ġshows Ġa Ġsample Ġcorpus Ġannotation . ĠTogether , Ġthe Ġmetadata Ġshown Ġin ĠFigure ĠFIG REF 11 Ġand Ġthe Ġannotations Ġpresented Ġin ĠFigure ĠFIG REF 34 Ġconstitute Ġa Ġcomplete ĠTC F Ġfile . </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠCorpus ĠProfile </s> The Ġresulting Ġcorpus Ġcontains Ġ6 , 217 Ġdocuments Ġ( 5 , 461 Ġmon oling ual Ġdocuments Ġplus Ġ378 Ġdocuments Ġfor Ġeach Ġside Ġof Ġthe Ġparallel Ġdata ). ĠTable ĠT AB REF 36 Ġshows Ġthe Ġcorpus Ġprofile . ĠThe Ġmon oling ual - only Ġdocuments Ġon Ġaverage Ġcontain Ġfewer Ġsentences Ġthan Ġthe Ġsimplified ĠGerman Ġside Ġof Ġthe Ġparallel Ġdata Ġ( average Ġdocument Ġlength Ġin Ġsentences Ġ31 . 64 Ġvs . Ġ55 . 75 ). ĠThe Ġaverage Ġsentence Ġlength Ġis Ġalmost Ġequal Ġ( app rox . Ġ11 Ġtokens ). ĠHence , Ġthe Ġmon oling ual - only Ġtexts Ġare Ġshorter Ġthan Ġthe Ġsimplified ĠGerman Ġtexts Ġin Ġthe Ġparallel Ġdata . ĠCompared Ġto Ġtheir ĠGerman Ġcounterparts , Ġthe Ġsimplified ĠGerman Ġtexts Ġin Ġthe Ġparallel Ġdata Ġhave Ġclearly Ġundergone Ġa Ġprocess Ġof Ġlex ical Ġsimpl ification : ĠThe Ġvocabulary Ġis Ġsmaller Ġby Ġ51 % Ġ( 33 , 384 Ġvs . Ġ16 , 352 Ġtypes ), Ġwhich Ġis Ġcomparable Ġto Ġthe Ġrate Ġof Ġreduction Ġreported Ġin ĠSection ĠSEC REF 2 Ġfor Ġthe ĠNew sel a ĠCorpus Ġ( 50 . 8 %). </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠEm pir ical Ġvalidation Ġof Ġthe Ġcorpus </s> b att ist i - 2019 Ġapplied Ġun super vised Ġmachine Ġlearning Ġtechniques Ġto Ġthe Ġsimplified ĠGerman Ġtexts Ġof Ġthe Ġcorpus Ġpresented Ġin Ġthis Ġpaper Ġwith Ġthe Ġaim Ġof Ġinvestigating Ġevidence Ġof Ġmultiple Ġcomplexity Ġlevels . ĠWhile Ġthe Ġdetailed Ġresults Ġare Ġbeyond Ġthe Ġscope Ġof Ġthis Ġpaper , Ġthe Ġauthor Ġfound Ġfeatures Ġbased Ġon Ġthe Ġstructural Ġinformation Ġthat Ġis Ġa Ġunique Ġproperty Ġof Ġthis Ġcorpus Ġ( e . g ., Ġnumber Ġof Ġimages , Ġnumber Ġof Ġparagraphs , Ġnumber Ġof Ġlines , Ġnumber Ġof Ġwords Ġof Ġa Ġspecific Ġfont Ġtype , Ġand Ġadherence Ġto Ġa Ġone - sent ence - per - line Ġrule ) Ġto Ġbe Ġpredictive Ġof Ġthe Ġlevel Ġof Ġdifficulty Ġof Ġa Ġsimplified ĠGerman Ġtext . ĠTo Ġour Ġknowledge , Ġthis Ġis Ġthe Ġfirst Ġstudy Ġto Ġdeliver Ġempirical Ġproof Ġof Ġthe Ġrelevance Ġof Ġsuch Ġfeatures . </s> Conclusion Ġand ĠOutlook </s> We Ġhave Ġintroduced Ġa Ġcorpus Ġcompiled Ġfor Ġuse Ġin Ġautomatic Ġread ability Ġassessment Ġand Ġautomatic Ġtext Ġsimpl ification Ġof ĠGerman . ĠWhile Ġsuch Ġtasks Ġhave Ġbeen Ġaddressed Ġfor Ġother Ġlanguages , Ġresearch Ġon ĠGerman Ġis Ġstill Ġscarce . ĠThe Ġfeatures Ġexploited Ġas Ġpart Ġof Ġmachine Ġlearning Ġapproaches Ġto Ġread ability Ġassessment Ġso Ġfar Ġtypically Ġinclude Ġsurface Ġand / or Ġ( d eeper ) Ġlinguistic Ġfeatures . ĠThe Ġcorpus Ġpresented Ġin Ġthis Ġpaper Ġadditionally Ġcontains Ġinformation Ġon Ġtext Ġstructure , Ġtyp ography , Ġand Ġimages . ĠThese Ġfeatures Ġhave Ġbeen Ġshown Ġto Ġbe Ġindicative Ġof Ġsimple Ġvs . Ġcomplex Ġtexts Ġboth Ġtheoretically Ġand , Ġusing Ġthe Ġcorpus Ġdescribed Ġin Ġthis Ġpaper , Ġempir ically . </s> Information Ġon Ġtext Ġstructure , Ġtyp ography , Ġand Ġimages Ġcan Ġalso Ġbe Ġlever aged Ġas Ġpart Ġof Ġa Ġneural Ġmachine Ġtranslation Ġapproach Ġto Ġtext Ġsimpl ification . ĠA Ġset Ġof Ġparallel Ġdocuments Ġused Ġin Ġmachine Ġtranslation Ġadditionally Ġrequires Ġsentence Ġalign ments , Ġwhich Ġare Ġstill Ġmissing Ġfrom Ġour Ġcorpus . ĠHence , Ġas Ġa Ġnext Ġstep , Ġwe Ġwill Ġinclude Ġsuch Ġinformation Ġusing Ġthe ĠCustom ized ĠAl ignment Ġfor ĠText ĠSimpl ification Ġ( C ATS ) Ġtool ĠB IB REF 23 .
Introduction Simplified language is a variety of standard language characterized by reduced lexical and syntactic complexity, the addition of explanations for difficult concepts, and clearly structured layout. Among the target groups of simplified language commonly mentioned are persons with cognitive impairment or learning disabilities, prelingually deaf persons, functionally illiterate persons, and foreign language learners BIBREF0. Two natural language processing tasks deal with the concept of simplified language: automatic readability assessment and automatic text simplification. Readability assessment refers to the process of determining the level of difficulty of a text, e.g., along readability measures, school grades, or levels of the Common European Framework of Reference for Languages (CEFR) BIBREF1. Readability measures, in their traditional form, take into account only surface features. For example, the Flesch Reading Ease Score BIBREF2 measures the length of words (in syllables) and sentences (in words). While readability has been shown to correlate with such features to some extent BIBREF3, a consensus has emerged according to which they are not sufficient to account for all of the complexity inherent in a text. As [p. 2618]kauchak-et-al-2014 state, “the usability of readability formulas is limited and there is little evidence that the output of these tools directly results in improved understanding by readers”. Recently, more sophisticated models employing (deeper) linguistic features such as lexical, semantic, morphological, morphosyntactic, syntactic, pragmatic, discourse, psycholinguistic, and language model features have been proposed BIBREF4, BIBREF5, BIBREF6, BIBREF7, BIBREF8. Automatic text simplification was initiated in the late 1990s BIBREF9, BIBREF10 and since then has been approached by means of rule-based and statistical methods. As part of a rule-based approach, the operations carried out typically include replacing complex lexical and syntactic units by simpler ones. A statistical approach generally conceptualizes the simplification task as one of converting a standard-language into a simplified-language text using machine translation. nisioi-et-al-2017 introduced neural machine translation to automatic text simplification. Research on automatic text simplification is comparatively widespread for languages such as English, Swedish, Spanish, and Brazilian Portuguese. To the authors' knowledge, no productive system exists for German. suter-2015, suter-et-al-2016 presented a prototype of a rule-based system for German. Machine learning approaches to both readability assessment and text simplification rely on data systematically prepared in the form of corpora. Specifically, for automatic text simplification via machine translation, pairs of standard-language/simplified-language texts aligned at the sentence level (i.e., parallel corpora) are needed. The paper at hand introduces a corpus developed for use in automatic readability assessment and automatic text simplification of German. The focus of this publication is on representing information that is valuable for these tasks but that hitherto has largely been ignored in machine learning approaches centering around simplified language, specifically, text structure (e.g., paragraphs, lines), typography (e.g., font type, font style), and image (content, position, and dimensions) information. The importance of considering such information has repeatedly been asserted theoretically BIBREF11, BIBREF12, BIBREF0. The remainder of this paper is structured as follows: Section SECREF2 presents previous corpora used for automatic readability assessment and text simplification. Section SECREF3 describes our corpus, introducing its novel aspects and presenting the primary data (Section SECREF7), the metadata (Section SECREF10), the secondary data (Section SECREF28), the profile (Section SECREF35), and the results of machine learning experiments carried out on the corpus (Section SECREF37). Previous Corpora for Automatic Readability Assessment and Automatic Text Simplification A number of corpora for use in automatic readability assessment and automatic text simplification exist. The most well-known example is the Parallel Wikipedia Simplification Corpus (PWKP) compiled from parallel articles of the English Wikipedia and Simple English Wikipedia BIBREF13 and consisting of around 108,000 sentence pairs. The corpus profile is shown in Table TABREF2. While the corpus represents the largest dataset involving simplified language to date, its application has been criticized for various reasons BIBREF15, BIBREF14, BIBREF16; among these, the fact that Simple English Wikipedia articles are not necessarily direct translations of articles from the English Wikipedia stands out. hwang-et-al-2015 provided an updated version of the corpus that includes a total of 280,000 full and partial matches between the two Wikipedia versions. Another frequently used data collection for English is the Newsela Corpus BIBREF14 consisting of 1,130 news articles, each simplified into four school grade levels by professional editors. Table TABREF3 shows the profile of the Newsela Corpus. The table obviates that the difference in vocabulary size between the English and the simplified English side of the PWKP Corpus amounts to only 18%, while the corresponding number for the English side and the level representing the highest amount of simplification in the Newsela Corpus (Simple-4) is 50.8%. Vocabulary size as an indicator of lexical richness is generally taken to correlate positively with complexity BIBREF17. gasperin-et-al-2010 compiled the PorSimples Corpus consisting of Brazilian Portuguese texts (2,116 sentences), each with a natural and a strong simplification, resulting in around 4,500 aligned sentences. drndarevic-saggion-2012, bott-et-al-2012, bott-saggion-2012 produced the Simplext Corpus consisting of 200 Spanish/simplified Spanish document pairs, amounting to a total of 1,149 (Spanish)/1,808 (simplified Spanish) sentences (approximately 1,000 aligned sentences). klaper-ebling-volk-2013 created the first parallel corpus for German/simplified German, consisting of 256 parallel texts downloaded from the web (approximately 70,000 tokens). Building a Corpus for Automatic Readability Assessment and Automatic Text Simplification of German Section SECREF2 demonstrated that the only corpus containing simplified German available is that of klaper-ebling-volk-2013. Since its creation, a number of legal and political developments have spurred the availability of data in simplified German. Among these developments is the introduction of a set of regulations for accessible information technology (Barrierefreie-Informationstechnik-Verordnung, BITV 2.0) in Germany and the ratification of the United Nations Convention on the Rights of Persons with Disabilities (CRPD) in Switzerland. The paper at hand introduces a corpus that represents an enhancement of the corpus of klaper-ebling-volk-2013 in the following ways: The corpus contains more parallel data. The corpus additionally contains monolingual-only data (simplified German). The corpus newly contains information on text structure, typography, and images. The simplified German side of the parallel data together with the monolingual-only data can be used for automatic readability assessment. The parallel data in the corpus is useful both for deriving rules for a rule-based text simplification system in a data-driven manner and for training a data-driven machine translation system. A data augmentation technique such as back-translation BIBREF18 can be applied to the monolingual-only data to arrive at additional (synthetic) parallel data. Building a Corpus for Automatic Readability Assessment and Automatic Text Simplification of German ::: Primary Data The corpus contains PDFs and webpages collected from web sources in Germany, Austria, and Switzerland at the end of 2018/beginning of 2019. The web sources mostly consist of websites of governments, specialised institutions, translation agencies, and non-profit organisations (92 different domains). The documents cover a range of topics, such as politics (e.g., instructions for voting), health (e.g., what to do in case of pregnancy), and culture (e.g., introduction to art museums). For the webpages, a static dump of all documents was created. Following this, the documents were manually checked to verify the language. The main content was subsequently extracted, i.e., HTML markup and boilerplate removed using the Beautiful Soup library for Python. Information on text structure (e.g., paragraphs, lines) and typography (e.g., boldface, italics) was retained. Similarly, image information (content, position, and dimensions of an image) was preserved. For PDFs, the PDFlib Text and Image Extraction Toolkit (TET) was used to extract the plain text and record information on text structure, typography, and images. The toolkit produces output in an XML format (TETML). Building a Corpus for Automatic Readability Assessment and Automatic Text Simplification of German ::: Metadata Metadata was collected automatically from the HTML (webpages) and TETML (PDFs) files, complemented manually, and recorded in the Open Language Archives Community (OLAC) Standard. OLAC is based on a reduced version of the Dublin Core Metadata Element Set (DCMES). Of the 15 elements of this “Simple Dublin Core” set, the following 12 were actively used along with controlled vocabularies of OLAC and Dublin Core: title: title of the document, with the language specified as the value of an xml:lang attribute and alternatives to the original title (e.g., translations) stored as dcterms:alternative (cf. Figure FIGREF11 for an example) contributor: all person entities linked to the creation of a document, with an olac:code attribute with values from the OLAC role vocabulary used to further specify the role of the contributor, e.g., author, editor, publisher, or translator date: date mentioned in the metadata of the HTML or PDF source or, for news and blog articles, date mentioned in the body of the text, in W3C date and time format description: value of the description in the metadata of an HTML document or list of sections of a PDF document, using the Dublin Core qualifier TableOfContents format: distinction between the Internet Media Types (MIME types) text/html (for webpages) and application/pdf (for PDFs) identifier: URL of the document or International Standard Book Number (ISBN) for books or brochures language: language of the document as value of the attribute olac:code (i.e., de, as conforming to ISO 639), with the CEFR level as optional element content publisher: organization or person that made the document available relation: used to establish a link between documents in German and simplified German for the parallel part of the corpus, using the Dublin Core qualifiers hasVersion (for the German text) and isVersionOf (for the simplified German text) rights: any piece of information about the rights of a document, as far as available in the source source: source document, i.e., HTML for web documents and TETML for PDFs type: nature or genre of the content of the document, which, in accordance with the DCMI Type Vocabulary, is Text in all cases and additionally StillImage in cases where a document also contains images. Additionally, the linguistic type is specified according to the OLAC Linguistic Data Type Vocabulary, as either primary_text (applies to most documents) or lexicon in cases where a document represents an entry of a simplified language vocabulary The elements coverage (to denote the spatial or temporal scope of the content of a resource), creator (to denote the author of a text, see contributor above), and subject (to denote the topic of the document content) were not used. Figure FIGREF11 shows an example of OLAC metadata. The source document described with this metadata record is a PDF structured into chapters, with text corresponding to the CEFR level A2 and images. Metadata in OLAC can be converted into the metadata standard of CLARIN (a European research infrastructure for language resources and technology), the Component MetaData Infrastructure (CMDI). The CMDI standard was chosen since it is the supported metadata version of CLARIN, which is specifically popular in German-speaking countries. Information on the language level of a simplified German text (typically A1, A2, or B1) is particularly valuable, as it allows for conducting automatic readability assessment and graded automatic text simplification experiments on the data. 52 websites and 233 PDFs (amounting to approximately 26,000 sentences) have an explicit language level label. Building a Corpus for Automatic Readability Assessment and Automatic Text Simplification of German ::: Secondary Data Annotations were added in the Text Corpus Format by WebLicht (TCF) developed as part of CLARIN. TCF supports standoff annotation, which allows for representation of annotations with conflicting hierarchies. TCF does not assign a separate file for each annotation layer; instead, the source text and all annotation layers are stored jointly in a single file. A token layer acts as the key element to which all other annotation layers are linked. The following types of annotations were added: text structure, fonts, images, tokens, parts of speech, morphological units, lemmas, sentences, and dependency parses. TCF does not readily accommodate the incorporation of all of these types of information. We therefore extended the format in the following ways: Information on the font type and font style (e.g., italics, bold print) of a token and its position on the physical page (for PDFs only) was specified as attributes to the token elements of the tokens layer (cf. Figure FIGREF34 for an example) Information on physical page segmentation (for PDFs only), paragraph segmentation, and line segmentation was added as part of a textspan element in the textstructure layer A separate images layer was introduced to hold image elements that take as attributes the x and y coordinates of the images, their dimensions (width and height), and the number of the page on which they occur A separate fonts layer was introduced to preserve detailed information on the font configurations referenced in the tokens layer Linguistic annotation was added automatically using the ParZu dependency parser for German BIBREF19 (for tokens and dependency parses), the NLTK toolkit BIBREF20 (for sentences), the TreeTagger BIBREF21 (for part-of-speech tags and lemmas), and Zmorge BIBREF22 (for morphological units). Figure FIGREF34 shows a sample corpus annotation. Together, the metadata shown in Figure FIGREF11 and the annotations presented in Figure FIGREF34 constitute a complete TCF file. Building a Corpus for Automatic Readability Assessment and Automatic Text Simplification of German ::: Corpus Profile The resulting corpus contains 6,217 documents (5,461 monolingual documents plus 378 documents for each side of the parallel data). Table TABREF36 shows the corpus profile. The monolingual-only documents on average contain fewer sentences than the simplified German side of the parallel data (average document length in sentences 31.64 vs. 55.75). The average sentence length is almost equal (approx. 11 tokens). Hence, the monolingual-only texts are shorter than the simplified German texts in the parallel data. Compared to their German counterparts, the simplified German texts in the parallel data have clearly undergone a process of lexical simplification: The vocabulary is smaller by 51% (33,384 vs. 16,352 types), which is comparable to the rate of reduction reported in Section SECREF2 for the Newsela Corpus (50.8%). Building a Corpus for Automatic Readability Assessment and Automatic Text Simplification of German ::: Empirical validation of the corpus battisti-2019 applied unsupervised machine learning techniques to the simplified German texts of the corpus presented in this paper with the aim of investigating evidence of multiple complexity levels. While the detailed results are beyond the scope of this paper, the author found features based on the structural information that is a unique property of this corpus (e.g., number of images, number of paragraphs, number of lines, number of words of a specific font type, and adherence to a one-sentence-per-line rule) to be predictive of the level of difficulty of a simplified German text. To our knowledge, this is the first study to deliver empirical proof of the relevance of such features. Conclusion and Outlook We have introduced a corpus compiled for use in automatic readability assessment and automatic text simplification of German. While such tasks have been addressed for other languages, research on German is still scarce. The features exploited as part of machine learning approaches to readability assessment so far typically include surface and/or (deeper) linguistic features. The corpus presented in this paper additionally contains information on text structure, typography, and images. These features have been shown to be indicative of simple vs. complex texts both theoretically and, using the corpus described in this paper, empirically. Information on text structure, typography, and images can also be leveraged as part of a neural machine translation approach to text simplification. A set of parallel documents used in machine translation additionally requires sentence alignments, which are still missing from our corpus. Hence, as a next step, we will include such information using the Customized Alignment for Text Simplification (CATS) tool BIBREF23.
Which information about text structure is included in the corpus?
9eabb54c2408dac24f00f92cf1061258c7ea2e1a
Which Ġinformation Ġabout Ġtext Ġstructure Ġis Ġincluded Ġin Ġthe Ġcorpus ?
paragraph s , Ġlines , ĠInformation Ġon Ġphysical Ġpage Ġsegment ation Ġ( for ĠPDF s Ġonly ), Ġparagraph Ġsegment ation , Ġand Ġline Ġsegment ation
<s> Which Ġinformation Ġabout Ġtext Ġstructure Ġis Ġincluded Ġin Ġthe Ġcorpus ? </s> Introduction </s> Sim pl ified Ġlanguage Ġis Ġa Ġvariety Ġof Ġstandard Ġlanguage Ġcharacterized Ġby Ġreduced Ġlex ical Ġand Ġsynt actic Ġcomplexity , Ġthe Ġaddition Ġof Ġexplanations Ġfor Ġdifficult Ġconcepts , Ġand Ġclearly Ġstructured Ġlayout . ĠAmong Ġthe Ġtarget Ġgroups Ġof Ġsimplified Ġlanguage Ġcommonly Ġmentioned Ġare Ġpersons Ġwith Ġcognitive Ġimpairment Ġor Ġlearning Ġdisabilities , Ġpre ling ually Ġdeaf Ġpersons , Ġfunctionally Ġilliter ate Ġpersons , Ġand Ġforeign Ġlanguage Ġlearners ĠB IB REF 0 . </s> Two Ġnatural Ġlanguage Ġprocessing Ġtasks Ġdeal Ġwith Ġthe Ġconcept Ġof Ġsimplified Ġlanguage : Ġautomatic Ġread ability Ġassessment Ġand Ġautomatic Ġtext Ġsimpl ification . ĠRead ability Ġassessment Ġrefers Ġto Ġthe Ġprocess Ġof Ġdetermining Ġthe Ġlevel Ġof Ġdifficulty Ġof Ġa Ġtext , Ġe . g ., Ġalong Ġread ability Ġmeasures , Ġschool Ġgrades , Ġor Ġlevels Ġof Ġthe ĠCommon ĠEuropean ĠFramework Ġof ĠReference Ġfor ĠLanguages Ġ( CE FR ) ĠB IB REF 1 . ĠRead ability Ġmeasures , Ġin Ġtheir Ġtraditional Ġform , Ġtake Ġinto Ġaccount Ġonly Ġsurface Ġfeatures . ĠFor Ġexample , Ġthe ĠF les ch ĠReading ĠE ase ĠScore ĠB IB REF 2 Ġmeasures Ġthe Ġlength Ġof Ġwords Ġ( in Ġsyll ables ) Ġand Ġsentences Ġ( in Ġwords ). ĠWhile Ġread ability Ġhas Ġbeen Ġshown Ġto Ġcorrelate Ġwith Ġsuch Ġfeatures Ġto Ġsome Ġextent ĠB IB REF 3 , Ġa Ġconsensus Ġhas Ġemerged Ġaccording Ġto Ġwhich Ġthey Ġare Ġnot Ġsufficient Ġto Ġaccount Ġfor Ġall Ġof Ġthe Ġcomplexity Ġinherent Ġin Ġa Ġtext . ĠAs Ġ[ p . Ġ26 18 ] k au ch ak - et - al - 2014 Ġstate , ĠâĢ ľ the Ġusability Ġof Ġread ability Ġformulas Ġis Ġlimited Ġand Ġthere Ġis Ġlittle Ġevidence Ġthat Ġthe Ġoutput Ġof Ġthese Ġtools Ġdirectly Ġresults Ġin Ġimproved Ġunderstanding Ġby Ġreaders âĢ Ŀ . ĠRecently , Ġmore Ġsophisticated Ġmodels Ġemploying Ġ( d eeper ) Ġlinguistic Ġfeatures Ġsuch Ġas Ġlex ical , Ġsemantic , Ġmorph ological , Ġmorph os ynt actic , Ġsynt actic , Ġpragmatic , Ġdiscourse , Ġpsych ol ingu istic , Ġand Ġlanguage Ġmodel Ġfeatures Ġhave Ġbeen Ġproposed ĠB IB REF 4 , ĠB IB REF 5 , ĠB IB REF 6 , ĠB IB REF 7 , ĠB IB REF 8 . </s> Aut omatic Ġtext Ġsimpl ification Ġwas Ġinitiated Ġin Ġthe Ġlate Ġ1990 s ĠB IB REF 9 , ĠB IB REF 10 Ġand Ġsince Ġthen Ġhas Ġbeen Ġapproached Ġby Ġmeans Ġof Ġrule - based Ġand Ġstatistical Ġmethods . ĠAs Ġpart Ġof Ġa Ġrule - based Ġapproach , Ġthe Ġoperations Ġcarried Ġout Ġtypically Ġinclude Ġreplacing Ġcomplex Ġlex ical Ġand Ġsynt actic Ġunits Ġby Ġsimpler Ġones . ĠA Ġstatistical Ġapproach Ġgenerally Ġconceptual izes Ġthe Ġsimpl ification Ġtask Ġas Ġone Ġof Ġconverting Ġa Ġstandard - language Ġinto Ġa Ġsimplified - language Ġtext Ġusing Ġmachine Ġtranslation . Ġn is io i - et - al - 2017 Ġintroduced Ġneural Ġmachine Ġtranslation Ġto Ġautomatic Ġtext Ġsimpl ification . ĠResearch Ġon Ġautomatic Ġtext Ġsimpl ification Ġis Ġcomparatively Ġwidespread Ġfor Ġlanguages Ġsuch Ġas ĠEnglish , ĠSwedish , ĠSpanish , Ġand ĠBrazilian ĠPortuguese . ĠTo Ġthe Ġauthors ' Ġknowledge , Ġno Ġproductive Ġsystem Ġexists Ġfor ĠGerman . Ġs uter - 2015 , Ġs uter - et - al - 2016 Ġpresented Ġa Ġprototype Ġof Ġa Ġrule - based Ġsystem Ġfor ĠGerman . </s> Machine Ġlearning Ġapproaches Ġto Ġboth Ġread ability Ġassessment Ġand Ġtext Ġsimpl ification Ġrely Ġon Ġdata Ġsystematically Ġprepared Ġin Ġthe Ġform Ġof Ġcorpor a . ĠSpecifically , Ġfor Ġautomatic Ġtext Ġsimpl ification Ġvia Ġmachine Ġtranslation , Ġpairs Ġof Ġstandard - language / sim pl ified - language Ġtexts Ġaligned Ġat Ġthe Ġsentence Ġlevel Ġ( i . e ., Ġparallel Ġcorpor a ) Ġare Ġneeded . </s> The Ġpaper Ġat Ġhand Ġintroduces Ġa Ġcorpus Ġdeveloped Ġfor Ġuse Ġin Ġautomatic Ġread ability Ġassessment Ġand Ġautomatic Ġtext Ġsimpl ification Ġof ĠGerman . ĠThe Ġfocus Ġof Ġthis Ġpublication Ġis Ġon Ġrepresenting Ġinformation Ġthat Ġis Ġvaluable Ġfor Ġthese Ġtasks Ġbut Ġthat Ġhitherto Ġhas Ġlargely Ġbeen Ġignored Ġin Ġmachine Ġlearning Ġapproaches Ġcent ering Ġaround Ġsimplified Ġlanguage , Ġspecifically , Ġtext Ġstructure Ġ( e . g ., Ġparagraphs , Ġlines ), Ġtyp ography Ġ( e . g ., Ġfont Ġtype , Ġfont Ġstyle ), Ġand Ġimage Ġ( content , Ġposition , Ġand Ġdimensions ) Ġinformation . ĠThe Ġimportance Ġof Ġconsidering Ġsuch Ġinformation Ġhas Ġrepeatedly Ġbeen Ġasserted Ġtheoretically ĠB IB REF 11 , ĠB IB REF 12 , ĠB IB REF 0 . ĠThe Ġremainder Ġof Ġthis Ġpaper Ġis Ġstructured Ġas Ġfollows : ĠSection ĠSEC REF 2 Ġpresents Ġprevious Ġcorpor a Ġused Ġfor Ġautomatic Ġread ability Ġassessment Ġand Ġtext Ġsimpl ification . ĠSection ĠSEC REF 3 Ġdescribes Ġour Ġcorpus , Ġintroducing Ġits Ġnovel Ġaspects Ġand Ġpresenting Ġthe Ġprimary Ġdata Ġ( Section ĠSEC REF 7 ), Ġthe Ġmetadata Ġ( Section ĠSEC REF 10 ), Ġthe Ġsecondary Ġdata Ġ( Section ĠSEC REF 28 ), Ġthe Ġprofile Ġ( Section ĠSEC REF 35 ), Ġand Ġthe Ġresults Ġof Ġmachine Ġlearning Ġexperiments Ġcarried Ġout Ġon Ġthe Ġcorpus Ġ( Section ĠSEC REF 37 ). </s> Previous ĠCorpor a Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification </s> A Ġnumber Ġof Ġcorpor a Ġfor Ġuse Ġin Ġautomatic Ġread ability Ġassessment Ġand Ġautomatic Ġtext Ġsimpl ification Ġexist . ĠThe Ġmost Ġwell - known Ġexample Ġis Ġthe ĠParallel ĠWikipedia ĠSimpl ification ĠCorpus Ġ( P W K P ) Ġcompiled Ġfrom Ġparallel Ġarticles Ġof Ġthe ĠEnglish ĠWikipedia Ġand ĠSimple ĠEnglish ĠWikipedia ĠB IB REF 13 Ġand Ġconsisting Ġof Ġaround Ġ108 , 000 Ġsentence Ġpairs . ĠThe Ġcorpus Ġprofile Ġis Ġshown Ġin ĠTable ĠT AB REF 2 . ĠWhile Ġthe Ġcorpus Ġrepresents Ġthe Ġlargest Ġdataset Ġinvolving Ġsimplified Ġlanguage Ġto Ġdate , Ġits Ġapplication Ġhas Ġbeen Ġcriticized Ġfor Ġvarious Ġreasons ĠB IB REF 15 , ĠB IB REF 14 , ĠB IB REF 16 ; Ġamong Ġthese , Ġthe Ġfact Ġthat ĠSimple ĠEnglish ĠWikipedia Ġarticles Ġare Ġnot Ġnecessarily Ġdirect Ġtranslations Ġof Ġarticles Ġfrom Ġthe ĠEnglish ĠWikipedia Ġstands Ġout . Ġh wang - et - al - 2015 Ġprovided Ġan Ġupdated Ġversion Ġof Ġthe Ġcorpus Ġthat Ġincludes Ġa Ġtotal Ġof Ġ280 , 000 Ġfull Ġand Ġpartial Ġmatches Ġbetween Ġthe Ġtwo ĠWikipedia Ġversions . ĠAnother Ġfrequently Ġused Ġdata Ġcollection Ġfor ĠEnglish Ġis Ġthe ĠNew sel a ĠCorpus ĠB IB REF 14 Ġconsisting Ġof Ġ1 , 130 Ġnews Ġarticles , Ġeach Ġsimplified Ġinto Ġfour Ġschool Ġgrade Ġlevels Ġby Ġprofessional Ġeditors . ĠTable ĠT AB REF 3 Ġshows Ġthe Ġprofile Ġof Ġthe ĠNew sel a ĠCorpus . ĠThe Ġtable Ġob vi ates Ġthat Ġthe Ġdifference Ġin Ġvocabulary Ġsize Ġbetween Ġthe ĠEnglish Ġand Ġthe Ġsimplified ĠEnglish Ġside Ġof Ġthe ĠPW K P ĠCorpus Ġamounts Ġto Ġonly Ġ18 %, Ġwhile Ġthe Ġcorresponding Ġnumber Ġfor Ġthe ĠEnglish Ġside Ġand Ġthe Ġlevel Ġrepresenting Ġthe Ġhighest Ġamount Ġof Ġsimpl ification Ġin Ġthe ĠNew sel a ĠCorpus Ġ( Simple - 4 ) Ġis Ġ50 . 8 %. ĠVoc abulary Ġsize Ġas Ġan Ġindicator Ġof Ġlex ical Ġrichness Ġis Ġgenerally Ġtaken Ġto Ġcorrelate Ġpositively Ġwith Ġcomplexity ĠB IB REF 17 . </s> gas per in - et - al - 2010 Ġcompiled Ġthe ĠPor Sim ples ĠCorpus Ġconsisting Ġof ĠBrazilian ĠPortuguese Ġtexts Ġ( 2 , 116 Ġsentences ), Ġeach Ġwith Ġa Ġnatural Ġand Ġa Ġstrong Ġsimpl ification , Ġresulting Ġin Ġaround Ġ4 , 500 Ġaligned Ġsentences . Ġdr nd are vic - s agg ion - 2012 , Ġbott - et - al - 2012 , Ġbott - s agg ion - 2012 Ġproduced Ġthe ĠSimple xt ĠCorpus Ġconsisting Ġof Ġ200 ĠSpanish / sim pl ified ĠSpanish Ġdocument Ġpairs , Ġamount ing Ġto Ġa Ġtotal Ġof Ġ1 , 149 Ġ( Spanish )/ 1 , 808 Ġ( sim pl ified ĠSpanish ) Ġsentences Ġ( approximately Ġ1 , 000 Ġaligned Ġsentences ). </s> kl aper - eb ling - vol k - 2013 Ġcreated Ġthe Ġfirst Ġparallel Ġcorpus Ġfor ĠGerman / sim pl ified ĠGerman , Ġconsisting Ġof Ġ256 Ġparallel Ġtexts Ġdownloaded Ġfrom Ġthe Ġweb Ġ( approximately Ġ70 , 000 Ġtokens ). </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman </s> Section ĠSEC REF 2 Ġdemonstrated Ġthat Ġthe Ġonly Ġcorpus Ġcontaining Ġsimplified ĠGerman Ġavailable Ġis Ġthat Ġof Ġk l aper - eb ling - vol k - 2013 . ĠSince Ġits Ġcreation , Ġa Ġnumber Ġof Ġlegal Ġand Ġpolitical Ġdevelopments Ġhave Ġspurred Ġthe Ġavailability Ġof Ġdata Ġin Ġsimplified ĠGerman . ĠAmong Ġthese Ġdevelopments Ġis Ġthe Ġintroduction Ġof Ġa Ġset Ġof Ġregulations Ġfor Ġaccessible Ġinformation Ġtechnology Ġ( Bar ri ere fre ie - Information ste chn ik - Ver ord n ung , ĠBIT V Ġ2 . 0 ) Ġin ĠGermany Ġand Ġthe Ġratification Ġof Ġthe ĠUnited ĠNations ĠConvention Ġon Ġthe ĠRights Ġof ĠPersons Ġwith ĠDis abilities Ġ( CR PD ) Ġin ĠSwitzerland . ĠThe Ġpaper Ġat Ġhand Ġintroduces Ġa Ġcorpus Ġthat Ġrepresents Ġan Ġenhancement Ġof Ġthe Ġcorpus Ġof Ġk l aper - eb ling - vol k - 2013 Ġin Ġthe Ġfollowing Ġways : </s> The Ġcorpus Ġcontains Ġmore Ġparallel Ġdata . </s> The Ġcorpus Ġadditionally Ġcontains Ġmon oling ual - only Ġdata Ġ( sim pl ified ĠGerman ). </s> The Ġcorpus Ġnewly Ġcontains Ġinformation Ġon Ġtext Ġstructure , Ġtyp ography , Ġand Ġimages . </s> The Ġsimplified ĠGerman Ġside Ġof Ġthe Ġparallel Ġdata Ġtogether Ġwith Ġthe Ġmon oling ual - only Ġdata Ġcan Ġbe Ġused Ġfor Ġautomatic Ġread ability Ġassessment . ĠThe Ġparallel Ġdata Ġin Ġthe Ġcorpus Ġis Ġuseful Ġboth Ġfor Ġder iving Ġrules Ġfor Ġa Ġrule - based Ġtext Ġsimpl ification Ġsystem Ġin Ġa Ġdata - driven Ġmanner Ġand Ġfor Ġtraining Ġa Ġdata - driven Ġmachine Ġtranslation Ġsystem . ĠA Ġdata Ġaug mentation Ġtechnique Ġsuch Ġas Ġback - translation ĠB IB REF 18 Ġcan Ġbe Ġapplied Ġto Ġthe Ġmon oling ual - only Ġdata Ġto Ġarrive Ġat Ġadditional Ġ( sy nt hetic ) Ġparallel Ġdata . </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠPrimary ĠData </s> The Ġcorpus Ġcontains ĠPDF s Ġand Ġweb pages Ġcollected Ġfrom Ġweb Ġsources Ġin ĠGermany , ĠAustria , Ġand ĠSwitzerland Ġat Ġthe Ġend Ġof Ġ2018 / begin ning Ġof Ġ2019 . ĠThe Ġweb Ġsources Ġmostly Ġconsist Ġof Ġwebsites Ġof Ġgovernments , Ġspecial ised Ġinstitutions , Ġtranslation Ġagencies , Ġand Ġnon - profit Ġorganisations Ġ( 92 Ġdifferent Ġdomains ). ĠThe Ġdocuments Ġcover Ġa Ġrange Ġof Ġtopics , Ġsuch Ġas Ġpolitics Ġ( e . g ., Ġinstructions Ġfor Ġvoting ), Ġhealth Ġ( e . g ., Ġwhat Ġto Ġdo Ġin Ġcase Ġof Ġpregnancy ), Ġand Ġculture Ġ( e . g ., Ġintroduction Ġto Ġart Ġmuseums ). </s> For Ġthe Ġweb pages , Ġa Ġstatic Ġdump Ġof Ġall Ġdocuments Ġwas Ġcreated . ĠFollowing Ġthis , Ġthe Ġdocuments Ġwere Ġmanually Ġchecked Ġto Ġverify Ġthe Ġlanguage . ĠThe Ġmain Ġcontent Ġwas Ġsubsequently Ġextracted , Ġi . e ., ĠHTML Ġmarkup Ġand Ġboiler plate Ġremoved Ġusing Ġthe ĠBeautiful ĠSoup Ġlibrary Ġfor ĠPython . ĠInformation Ġon Ġtext Ġstructure Ġ( e . g ., Ġparagraphs , Ġlines ) Ġand Ġtyp ography Ġ( e . g ., Ġbold face , Ġital ics ) Ġwas Ġretained . ĠSimilarly , Ġimage Ġinformation Ġ( content , Ġposition , Ġand Ġdimensions Ġof Ġan Ġimage ) Ġwas Ġpreserved . </s> For ĠPDF s , Ġthe ĠPD Fl ib ĠText Ġand ĠImage ĠExt raction ĠTool kit Ġ( T ET ) Ġwas Ġused Ġto Ġextract Ġthe Ġplain Ġtext Ġand Ġrecord Ġinformation Ġon Ġtext Ġstructure , Ġtyp ography , Ġand Ġimages . ĠThe Ġtool kit Ġproduces Ġoutput Ġin Ġan ĠXML Ġformat Ġ( T ET ML ). </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠMet adata </s> Met adata Ġwas Ġcollected Ġautomatically Ġfrom Ġthe ĠHTML Ġ( web pages ) Ġand ĠT ET ML Ġ( PDF s ) Ġfiles , Ġcomple mented Ġmanually , Ġand Ġrecorded Ġin Ġthe ĠOpen ĠLanguage ĠArchives ĠCommunity Ġ( OL AC ) ĠStandard . ĠOL AC Ġis Ġbased Ġon Ġa Ġreduced Ġversion Ġof Ġthe ĠDublin ĠCore ĠMet adata ĠElement ĠSet Ġ( DC M ES ). ĠOf Ġthe Ġ15 Ġelements Ġof Ġthis ĠâĢ ľ Simple ĠDublin ĠCore âĢ Ŀ Ġset , Ġthe Ġfollowing Ġ12 Ġwere Ġactively Ġused Ġalong Ġwith Ġcontrolled Ġvoc ab ular ies Ġof ĠOL AC Ġand ĠDublin ĠCore : </s> title : Ġtitle Ġof Ġthe Ġdocument , Ġwith Ġthe Ġlanguage Ġspecified Ġas Ġthe Ġvalue Ġof Ġan Ġxml : lang Ġattribute Ġand Ġalternatives Ġto Ġthe Ġoriginal Ġtitle Ġ( e . g ., Ġtranslations ) Ġstored Ġas Ġd ct er ms : altern ative Ġ( cf . ĠFigure ĠFIG REF 11 Ġfor Ġan Ġexample ) </s> cont ribut or : Ġall Ġperson Ġentities Ġlinked Ġto Ġthe Ġcreation Ġof Ġa Ġdocument , Ġwith Ġan Ġol ac : code Ġattribute Ġwith Ġvalues Ġfrom Ġthe ĠOL AC Ġrole Ġvocabulary Ġused Ġto Ġfurther Ġspecify Ġthe Ġrole Ġof Ġthe Ġcontributor , Ġe . g ., Ġauthor , Ġeditor , Ġpublisher , Ġor Ġtranslator </s> date : Ġdate Ġmentioned Ġin Ġthe Ġmetadata Ġof Ġthe ĠHTML Ġor ĠPDF Ġsource Ġor , Ġfor Ġnews Ġand Ġblog Ġarticles , Ġdate Ġmentioned Ġin Ġthe Ġbody Ġof Ġthe Ġtext , Ġin ĠW 3 C Ġdate Ġand Ġtime Ġformat </s> description : Ġvalue Ġof Ġthe Ġdescription Ġin Ġthe Ġmetadata Ġof Ġan ĠHTML Ġdocument Ġor Ġlist Ġof Ġsections Ġof Ġa ĠPDF Ġdocument , Ġusing Ġthe ĠDublin ĠCore Ġqualifier ĠTable Of Contents </s> format : Ġdistinction Ġbetween Ġthe ĠInternet ĠMedia ĠTypes Ġ( M IME Ġtypes ) Ġtext / html Ġ( for Ġweb pages ) Ġand Ġapplication / pdf Ġ( for ĠPDF s ) </s> ident ifier : ĠURL Ġof Ġthe Ġdocument Ġor ĠInternational ĠStandard ĠBook ĠNumber Ġ( IS BN ) Ġfor Ġbooks Ġor Ġbroch ures </s> language : Ġlanguage Ġof Ġthe Ġdocument Ġas Ġvalue Ġof Ġthe Ġattribute Ġol ac : code Ġ( i . e ., Ġde , Ġas Ġcon forming Ġto ĠISO Ġ6 39 ), Ġwith Ġthe ĠCE FR Ġlevel Ġas Ġoptional Ġelement Ġcontent </s> pub lisher : Ġorganization Ġor Ġperson Ġthat Ġmade Ġthe Ġdocument Ġavailable </s> relation : Ġused Ġto Ġestablish Ġa Ġlink Ġbetween Ġdocuments Ġin ĠGerman Ġand Ġsimplified ĠGerman Ġfor Ġthe Ġparallel Ġpart Ġof Ġthe Ġcorpus , Ġusing Ġthe ĠDublin ĠCore Ġqualifiers Ġhas Version Ġ( for Ġthe ĠGerman Ġtext ) Ġand Ġis Version Of Ġ( for Ġthe Ġsimplified ĠGerman Ġtext ) </s> rights : Ġany Ġpiece Ġof Ġinformation Ġabout Ġthe Ġrights Ġof Ġa Ġdocument , Ġas Ġfar Ġas Ġavailable Ġin Ġthe Ġsource </s> source : Ġsource Ġdocument , Ġi . e ., ĠHTML Ġfor Ġweb Ġdocuments Ġand ĠT ET ML Ġfor ĠPDF s </s> type : Ġnature Ġor Ġgenre Ġof Ġthe Ġcontent Ġof Ġthe Ġdocument , Ġwhich , Ġin Ġaccordance Ġwith Ġthe ĠDC MI ĠType ĠVoc abulary , Ġis ĠText Ġin Ġall Ġcases Ġand Ġadditionally ĠStill Image Ġin Ġcases Ġwhere Ġa Ġdocument Ġalso Ġcontains Ġimages . ĠAdditionally , Ġthe Ġlinguistic Ġtype Ġis Ġspecified Ġaccording Ġto Ġthe ĠOL AC ĠL ingu istic ĠData ĠType ĠVoc abulary , Ġas Ġeither Ġprimary _ text Ġ( app lies Ġto Ġmost Ġdocuments ) Ġor Ġlex icon Ġin Ġcases Ġwhere Ġa Ġdocument Ġrepresents Ġan Ġentry Ġof Ġa Ġsimplified Ġlanguage Ġvocabulary </s> The Ġelements Ġcoverage Ġ( to Ġdenote Ġthe Ġspatial Ġor Ġtemporal Ġscope Ġof Ġthe Ġcontent Ġof Ġa Ġresource ), Ġcreator Ġ( to Ġdenote Ġthe Ġauthor Ġof Ġa Ġtext , Ġsee Ġcontributor Ġabove ), Ġand Ġsubject Ġ( to Ġdenote Ġthe Ġtopic Ġof Ġthe Ġdocument Ġcontent ) Ġwere Ġnot Ġused . </s> Figure ĠFIG REF 11 Ġshows Ġan Ġexample Ġof ĠOL AC Ġmetadata . ĠThe Ġsource Ġdocument Ġdescribed Ġwith Ġthis Ġmetadata Ġrecord Ġis Ġa ĠPDF Ġstructured Ġinto Ġchapters , Ġwith Ġtext Ġcorresponding Ġto Ġthe ĠCE FR Ġlevel ĠA 2 Ġand Ġimages . ĠMet adata Ġin ĠOL AC Ġcan Ġbe Ġconverted Ġinto Ġthe Ġmetadata Ġstandard Ġof ĠCL AR IN Ġ( a ĠEuropean Ġresearch Ġinfrastructure Ġfor Ġlanguage Ġresources Ġand Ġtechnology ), Ġthe ĠComponent ĠMeta Data ĠInfrastructure Ġ( C MD I ). ĠThe ĠC MD I Ġstandard Ġwas Ġchosen Ġsince Ġit Ġis Ġthe Ġsupported Ġmetadata Ġversion Ġof ĠCL AR IN , Ġwhich Ġis Ġspecifically Ġpopular Ġin ĠGerman - speaking Ġcountries . </s> Information Ġon Ġthe Ġlanguage Ġlevel Ġof Ġa Ġsimplified ĠGerman Ġtext Ġ( typically ĠA 1 , ĠA 2 , Ġor ĠB 1 ) Ġis Ġparticularly Ġvaluable , Ġas Ġit Ġallows Ġfor Ġconducting Ġautomatic Ġread ability Ġassessment Ġand Ġgraded Ġautomatic Ġtext Ġsimpl ification Ġexperiments Ġon Ġthe Ġdata . Ġ52 Ġwebsites Ġand Ġ233 ĠPDF s Ġ( amount ing Ġto Ġapproximately Ġ26 , 000 Ġsentences ) Ġhave Ġan Ġexplicit Ġlanguage Ġlevel Ġlabel . </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠSecondary ĠData </s> An notations Ġwere Ġadded Ġin Ġthe ĠText ĠCorpus ĠFormat Ġby ĠWeb L icht Ġ( TC F ) Ġdeveloped Ġas Ġpart Ġof ĠCL AR IN . ĠTC F Ġsupports Ġstandoff Ġannotation , Ġwhich Ġallows Ġfor Ġrepresentation Ġof Ġannotations Ġwith Ġconflicting Ġhierarch ies . ĠTC F Ġdoes Ġnot Ġassign Ġa Ġseparate Ġfile Ġfor Ġeach Ġannotation Ġlayer ; Ġinstead , Ġthe Ġsource Ġtext Ġand Ġall Ġannotation Ġlayers Ġare Ġstored Ġjointly Ġin Ġa Ġsingle Ġfile . ĠA Ġtoken Ġlayer Ġacts Ġas Ġthe Ġkey Ġelement Ġto Ġwhich Ġall Ġother Ġannotation Ġlayers Ġare Ġlinked . </s> The Ġfollowing Ġtypes Ġof Ġannotations Ġwere Ġadded : Ġtext Ġstructure , Ġfonts , Ġimages , Ġtokens , Ġparts Ġof Ġspeech , Ġmorph ological Ġunits , Ġle mm as , Ġsentences , Ġand Ġdependency Ġpars es . ĠTC F Ġdoes Ġnot Ġreadily Ġaccommodate Ġthe Ġincorporation Ġof Ġall Ġof Ġthese Ġtypes Ġof Ġinformation . ĠWe Ġtherefore Ġextended Ġthe Ġformat Ġin Ġthe Ġfollowing Ġways : </s> Information Ġon Ġthe Ġfont Ġtype Ġand Ġfont Ġstyle Ġ( e . g ., Ġital ics , Ġbold Ġprint ) Ġof Ġa Ġtoken Ġand Ġits Ġposition Ġon Ġthe Ġphysical Ġpage Ġ( for ĠPDF s Ġonly ) Ġwas Ġspecified Ġas Ġattributes Ġto Ġthe Ġtoken Ġelements Ġof Ġthe Ġtokens Ġlayer Ġ( cf . ĠFigure ĠFIG REF 34 Ġfor Ġan Ġexample ) </s> Information Ġon Ġphysical Ġpage Ġsegment ation Ġ( for ĠPDF s Ġonly ), Ġparagraph Ġsegment ation , Ġand Ġline Ġsegment ation Ġwas Ġadded Ġas Ġpart Ġof Ġa Ġtext span Ġelement Ġin Ġthe Ġtext st ructure Ġlayer </s> A Ġseparate Ġimages Ġlayer Ġwas Ġintroduced Ġto Ġhold Ġimage Ġelements Ġthat Ġtake Ġas Ġattributes Ġthe Ġx Ġand Ġy Ġcoordinates Ġof Ġthe Ġimages , Ġtheir Ġdimensions Ġ( width Ġand Ġheight ), Ġand Ġthe Ġnumber Ġof Ġthe Ġpage Ġon Ġwhich Ġthey Ġoccur </s> A Ġseparate Ġfonts Ġlayer Ġwas Ġintroduced Ġto Ġpreserve Ġdetailed Ġinformation Ġon Ġthe Ġfont Ġconfigurations Ġreferenced Ġin Ġthe Ġtokens Ġlayer </s> L ingu istic Ġannotation Ġwas Ġadded Ġautomatically Ġusing Ġthe ĠPar Z u Ġdependency Ġparser Ġfor ĠGerman ĠB IB REF 19 Ġ( for Ġtokens Ġand Ġdependency Ġpars es ), Ġthe ĠNL T K Ġtool kit ĠB IB REF 20 Ġ( for Ġsentences ), Ġthe ĠTree T agger ĠB IB REF 21 Ġ( for Ġpart - of - speech Ġtags Ġand Ġle mm as ), Ġand ĠZ m orge ĠB IB REF 22 Ġ( for Ġmorph ological Ġunits ). ĠFigure ĠFIG REF 34 Ġshows Ġa Ġsample Ġcorpus Ġannotation . ĠTogether , Ġthe Ġmetadata Ġshown Ġin ĠFigure ĠFIG REF 11 Ġand Ġthe Ġannotations Ġpresented Ġin ĠFigure ĠFIG REF 34 Ġconstitute Ġa Ġcomplete ĠTC F Ġfile . </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠCorpus ĠProfile </s> The Ġresulting Ġcorpus Ġcontains Ġ6 , 217 Ġdocuments Ġ( 5 , 461 Ġmon oling ual Ġdocuments Ġplus Ġ378 Ġdocuments Ġfor Ġeach Ġside Ġof Ġthe Ġparallel Ġdata ). ĠTable ĠT AB REF 36 Ġshows Ġthe Ġcorpus Ġprofile . ĠThe Ġmon oling ual - only Ġdocuments Ġon Ġaverage Ġcontain Ġfewer Ġsentences Ġthan Ġthe Ġsimplified ĠGerman Ġside Ġof Ġthe Ġparallel Ġdata Ġ( average Ġdocument Ġlength Ġin Ġsentences Ġ31 . 64 Ġvs . Ġ55 . 75 ). ĠThe Ġaverage Ġsentence Ġlength Ġis Ġalmost Ġequal Ġ( app rox . Ġ11 Ġtokens ). ĠHence , Ġthe Ġmon oling ual - only Ġtexts Ġare Ġshorter Ġthan Ġthe Ġsimplified ĠGerman Ġtexts Ġin Ġthe Ġparallel Ġdata . ĠCompared Ġto Ġtheir ĠGerman Ġcounterparts , Ġthe Ġsimplified ĠGerman Ġtexts Ġin Ġthe Ġparallel Ġdata Ġhave Ġclearly Ġundergone Ġa Ġprocess Ġof Ġlex ical Ġsimpl ification : ĠThe Ġvocabulary Ġis Ġsmaller Ġby Ġ51 % Ġ( 33 , 384 Ġvs . Ġ16 , 352 Ġtypes ), Ġwhich Ġis Ġcomparable Ġto Ġthe Ġrate Ġof Ġreduction Ġreported Ġin ĠSection ĠSEC REF 2 Ġfor Ġthe ĠNew sel a ĠCorpus Ġ( 50 . 8 %). </s> Building Ġa ĠCorpus Ġfor ĠAutomatic ĠRead ability ĠAssessment Ġand ĠAutomatic ĠText ĠSimpl ification Ġof ĠGerman Ġ: :: ĠEm pir ical Ġvalidation Ġof Ġthe Ġcorpus </s> b att ist i - 2019 Ġapplied Ġun super vised Ġmachine Ġlearning Ġtechniques Ġto Ġthe Ġsimplified ĠGerman Ġtexts Ġof Ġthe Ġcorpus Ġpresented Ġin Ġthis Ġpaper Ġwith Ġthe Ġaim Ġof Ġinvestigating Ġevidence Ġof Ġmultiple Ġcomplexity Ġlevels . ĠWhile Ġthe Ġdetailed Ġresults Ġare Ġbeyond Ġthe Ġscope Ġof Ġthis Ġpaper , Ġthe Ġauthor Ġfound Ġfeatures Ġbased Ġon Ġthe Ġstructural Ġinformation Ġthat Ġis Ġa Ġunique Ġproperty Ġof Ġthis Ġcorpus Ġ( e . g ., Ġnumber Ġof Ġimages , Ġnumber Ġof Ġparagraphs , Ġnumber Ġof Ġlines , Ġnumber Ġof Ġwords Ġof Ġa Ġspecific Ġfont Ġtype , Ġand Ġadherence Ġto Ġa Ġone - sent ence - per - line Ġrule ) Ġto Ġbe Ġpredictive Ġof Ġthe Ġlevel Ġof Ġdifficulty Ġof Ġa Ġsimplified ĠGerman Ġtext . ĠTo Ġour Ġknowledge , Ġthis Ġis Ġthe Ġfirst Ġstudy Ġto Ġdeliver Ġempirical Ġproof Ġof Ġthe Ġrelevance Ġof Ġsuch Ġfeatures . </s> Conclusion Ġand ĠOutlook </s> We Ġhave Ġintroduced Ġa Ġcorpus Ġcompiled Ġfor Ġuse Ġin Ġautomatic Ġread ability Ġassessment Ġand Ġautomatic Ġtext Ġsimpl ification Ġof ĠGerman . ĠWhile Ġsuch Ġtasks Ġhave Ġbeen Ġaddressed Ġfor Ġother Ġlanguages , Ġresearch Ġon ĠGerman Ġis Ġstill Ġscarce . ĠThe Ġfeatures Ġexploited Ġas Ġpart Ġof Ġmachine Ġlearning Ġapproaches Ġto Ġread ability Ġassessment Ġso Ġfar Ġtypically Ġinclude Ġsurface Ġand / or Ġ( d eeper ) Ġlinguistic Ġfeatures . ĠThe Ġcorpus Ġpresented Ġin Ġthis Ġpaper Ġadditionally Ġcontains Ġinformation Ġon Ġtext Ġstructure , Ġtyp ography , Ġand Ġimages . ĠThese Ġfeatures Ġhave Ġbeen Ġshown Ġto Ġbe Ġindicative Ġof Ġsimple Ġvs . Ġcomplex Ġtexts Ġboth Ġtheoretically Ġand , Ġusing Ġthe Ġcorpus Ġdescribed Ġin Ġthis Ġpaper , Ġempir ically . </s> Information Ġon Ġtext Ġstructure , Ġtyp ography , Ġand Ġimages Ġcan Ġalso Ġbe Ġlever aged Ġas Ġpart Ġof Ġa Ġneural Ġmachine Ġtranslation Ġapproach Ġto Ġtext Ġsimpl ification . ĠA Ġset Ġof Ġparallel Ġdocuments Ġused Ġin Ġmachine Ġtranslation Ġadditionally Ġrequires Ġsentence Ġalign ments , Ġwhich Ġare Ġstill Ġmissing Ġfrom Ġour Ġcorpus . ĠHence , Ġas Ġa Ġnext Ġstep , Ġwe Ġwill Ġinclude Ġsuch Ġinformation Ġusing Ġthe ĠCustom ized ĠAl ignment Ġfor ĠText ĠSimpl ification Ġ( C ATS ) Ġtool ĠB IB REF 23 .