OpATT BIBREF6, Neural Content Planning with conditional copy (NCP+CC) BIBREF4
EXTRACTIVE
Unanswerable
NONE
Table TABREF23 displays the automatic evaluation results on both development and test set. We chose Conditional Copy (CC) model as our baseline, which is the best model in Wiseman. We included reported scores with updated IE model by Puduppully and our implementation's result on CC in this paper. Also, we compared our models with other existing works on this dataset including OpATT BIBREF6 and Neural Content Planning with conditional copy (NCP+CC) BIBREF4. In addition, we implemented three other hierarchical encoders that encoded tables' row dimension information in both record-level and row-level to compare with the hierarchical structure of encoder in our model. The decoder was equipped with dual attention BIBREF9. The one with LSTM cell is similar to the one in N18-2097 with 1 layer from {1,2,3}. The one with CNN cell BIBREF10 has kernel width 3 from {3, 5} and 10 layer from {5,10,15,20}. The one with transformer-style encoder (MHSA) BIBREF11 has 8 head from {8, 10} and 5 layer from {2,3,4,5,6}. The heads and layers mentioned above were for both record-level encoder and row-level encoder respectively. The self-attention (SA) cell we used, as described in Section SECREF3, achieved better overall performance in terms of F1% of CS, CO and BLEU among the hierarchical encoders. Also we implemented a template system same as the one used in Wiseman which outputted eight sentences: an introductory sentence (two teams' points and who win), six top players' statistics (ranked by their points) and a conclusion sentence. We refer the readers to Wiseman's paper for more detailed information on templates. The gold reference's result is also included in Table TABREF23. Overall, our model performs better than other neural models on both development and test set in terms of RG's P%, F1% score of CS, CO and BLEU, indicating our model's clear improvement on generating high-fidelity, informative and fluent texts. Also, our model with three dimension representations outperforms hierarchical encoders with only row dimension representation on development set. This indicates that cell and time dimension representation are important in representing the tables. Compared to reported baseline result in Wiseman, we achieved improvement of $22.27\%$ in terms of RG, $26.84\%$ in terms of CS F1%, $35.28\%$ in terms of CO and $18.75\%$ in terms of BLEU on test set. Unsurprisingly, template system achieves best on RG P% and CS R% due to the included domain knowledge. Also, the high RG # and low CS P% indicates that template will include vast information while many of them are deemed redundant. In addition, the low CO and low BLEU indicates that the rigid structure of the template will produce texts that aren't as adaptive to the given tables and natural as those produced by neural models. Also, we conducted ablation study on our model to evaluate each component's contribution on development set. Based on the results, the absence of row-level encoder hurts our model's performance across all metrics especially the content selection ability.
1909.02304
Introduction </s> Table - to - text Ġgeneration Ġis Ġan Ġimportant Ġand Ġchallenging Ġtask Ġin Ġnatural Ġlanguage Ġprocessing , Ġwhich Ġaims Ġto Ġproduce Ġthe Ġsummar ization Ġof Ġnumerical Ġtable ĠB IB REF 0 , ĠB IB REF 1 . ĠThe Ġrelated Ġmethods Ġcan Ġbe Ġempir ically Ġdivided Ġinto Ġtwo Ġcategories , Ġpipeline Ġmodel Ġand Ġend - to - end Ġmodel . ĠThe Ġformer Ġconsists Ġof Ġcontent Ġselection , Ġdocument Ġplanning Ġand Ġreal isation , Ġmainly Ġfor Ġearly Ġindustrial Ġapplications , Ġsuch Ġas Ġweather Ġforecasting Ġand Ġmedical Ġmonitoring , Ġetc . ĠThe Ġlatter Ġgenerates Ġtext Ġdirectly Ġfrom Ġthe Ġtable Ġthrough Ġa Ġstandard Ġneural Ġenc oder - dec oder Ġframework Ġto Ġavoid Ġerror Ġpropagation Ġand Ġhas Ġachieved Ġremarkable Ġprogress . ĠIn Ġthis Ġpaper , Ġwe Ġparticularly Ġfocus Ġon Ġexploring Ġhow Ġto Ġimprove Ġthe Ġperformance Ġof Ġneural Ġmethods Ġon Ġtable - to - text Ġgeneration . </s> Recently , ĠR OT OW IRE , Ġwhich Ġprovides Ġtables Ġof ĠNBA Ġplayers ' Ġand Ġteams ' Ġstatistics Ġwith Ġa Ġdescriptive Ġsummary , Ġhas Ġdrawn Ġincreasing Ġattention Ġfrom Ġacademic Ġcommunity . ĠFigure ĠFIG REF 1 Ġshows Ġan Ġexample Ġof Ġparts Ġof Ġa Ġgame 's Ġstatistics Ġand Ġits Ġcorresponding Ġcomputer Ġgenerated Ġsummary . ĠWe Ġcan Ġsee Ġthat Ġthe Ġtables Ġhas Ġa Ġformal Ġstructure Ġincluding Ġtable Ġrow Ġheader , Ġtable Ġcolumn Ġheader Ġand Ġtable Ġcells . ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġis Ġa Ġtable Ġrow Ġheader Ġthat Ġrepresents Ġa Ġplayer , ĠâĢ ľ P TS âĢ Ŀ Ġis Ġa Ġtable Ġcolumn Ġheader Ġindicating Ġthe Ġcolumn Ġcontains Ġplayer 's Ġscore Ġand ĠâĢ ľ 18 âĢ Ŀ Ġis Ġthe Ġvalue Ġof Ġthe Ġtable Ġcell , Ġthat Ġis , ĠAl ĠJefferson Ġscored Ġ18 Ġpoints . ĠSeveral Ġrelated Ġmodels Ġhave Ġbeen Ġproposed Ġ. ĠThey Ġtypically Ġencode Ġthe Ġtable 's Ġrecords Ġseparately Ġor Ġas Ġa Ġlong Ġsequence Ġand Ġgenerate Ġa Ġlong Ġdescriptive Ġsummary Ġby Ġa Ġstandard ĠSe q 2 Se q Ġdec oder Ġwith Ġsome Ġmodifications . ĠWis eman Ġexplored Ġtwo Ġtypes Ġof Ġcopy Ġmechanism Ġand Ġfound Ġconditional Ġcopy Ġmodel ĠB IB REF 3 Ġperform Ġbetter Ġ. ĠP ud upp ully Ġenhanced Ġcontent Ġselection Ġability Ġby Ġexplicitly Ġselecting Ġand Ġplanning Ġrelevant Ġrecords . ĠLi Ġimproved Ġthe Ġprecision Ġof Ġdescribing Ġdata Ġrecords Ġin Ġthe Ġgenerated Ġtexts Ġby Ġgenerating Ġa Ġtemplate Ġat Ġfirst Ġand Ġfilling Ġin Ġslots Ġvia Ġcopy Ġmechanism . ĠN ie Ġutilized Ġresults Ġfrom Ġpre - exec uted Ġoperations Ġto Ġimprove Ġthe Ġfidelity Ġof Ġgenerated Ġtexts . ĠHowever , Ġwe Ġclaim Ġthat Ġtheir Ġencoding Ġof Ġtables Ġas Ġsets Ġof Ġrecords Ġor Ġa Ġlong Ġsequence Ġis Ġnot Ġsuitable . ĠBecause Ġ( 1 ) Ġthe Ġtable Ġconsists Ġof Ġmultiple Ġplayers Ġand Ġdifferent Ġtypes Ġof Ġinformation Ġas Ġshown Ġin ĠFigure ĠFIG REF 1 . ĠThe Ġearlier Ġencoding Ġapproaches Ġonly Ġconsidered Ġthe Ġtable Ġas Ġsets Ġof Ġrecords Ġor Ġone Ġdimensional Ġsequence , Ġwhich Ġwould Ġlose Ġthe Ġinformation Ġof Ġother Ġ( column ) Ġdimension . Ġ( 2 ) Ġthe Ġtable Ġcell Ġconsists Ġof Ġtime - series Ġdata Ġwhich Ġchange Ġover Ġtime . ĠThat Ġis Ġto Ġsay , Ġsometimes Ġhistorical Ġdata Ġcan Ġhelp Ġthe Ġmodel Ġselect Ġcontent . ĠMoreover , Ġwhen Ġa Ġhuman Ġwrites Ġa Ġbasketball Ġreport , Ġhe Ġwill Ġnot Ġonly Ġfocus Ġon Ġthe Ġplayers ' Ġoutstanding Ġperformance Ġin Ġthe Ġcurrent Ġmatch , Ġbut Ġalso Ġsummarize Ġplayers ' Ġperformance Ġin Ġrecent Ġmatches . ĠLets Ġtake ĠFigure ĠFIG REF 1 Ġagain . ĠNot Ġonly Ġdo Ġthe Ġgold Ġtexts Ġmention ĠAl ĠJefferson 's Ġgreat Ġperformance Ġin Ġthis Ġmatch , Ġit Ġalso Ġstates Ġthat ĠâĢ ľ It Ġwas Ġthe Ġsecond Ġtime Ġin Ġthe Ġlast Ġthree Ġgames Ġhe 's Ġposted Ġa Ġdouble - double âĢ Ŀ . ĠAlso Ġgold Ġtexts Ġsummarize ĠJohn ĠWall 's ĠâĢ ľ double - double âĢ Ŀ Ġperformance Ġin Ġthe Ġsimilar Ġway . ĠSum mar izing Ġa Ġplayer 's Ġperformance Ġin Ġrecent Ġmatches Ġrequires Ġthe Ġmodeling Ġof Ġtable Ġcell Ġwith Ġrespect Ġto Ġits Ġhistorical Ġdata Ġ( time Ġdimension ) Ġwhich Ġis Ġabsent Ġin Ġbaseline Ġmodel . ĠAlthough Ġbaseline Ġmodel ĠCond itional ĠCopy Ġ( CC ) Ġtries Ġto Ġsummarize Ġit Ġfor ĠGerald ĠHenderson , Ġit Ġclearly Ġproduce Ġwrong Ġstatements Ġsince Ġhe Ġdidn 't Ġget ĠâĢ ľ double - double âĢ Ŀ Ġin Ġthis Ġmatch . </s> To Ġaddress Ġthe Ġaforementioned Ġproblems , Ġwe Ġpresent Ġa Ġhierarchical Ġenc oder Ġto Ġsimultaneously Ġmodel Ġrow , Ġcolumn Ġand Ġtime Ġdimension Ġinformation . ĠIn Ġdetail , Ġour Ġmodel Ġis Ġdivided Ġinto Ġthree Ġlayers . ĠThe Ġfirst Ġlayer Ġis Ġused Ġto Ġlearn Ġthe Ġrepresentation Ġof Ġthe Ġtable Ġcell . ĠSpecifically , Ġwe Ġemploy Ġthree Ġself - att ention Ġmodels Ġto Ġobtain Ġthree Ġrepresentations Ġof Ġthe Ġtable Ġcell Ġin Ġits Ġrow , Ġcolumn Ġand Ġtime Ġdimension . ĠThen , Ġin Ġthe Ġsecond Ġlayer , Ġwe Ġdesign Ġa Ġrecord Ġfusion Ġgate Ġto Ġidentify Ġthe Ġmore Ġimportant Ġrepresentation Ġfrom Ġthose Ġthree Ġdimension Ġand Ġcombine Ġthem Ġinto Ġa Ġdense Ġvector . ĠIn Ġthe Ġthird Ġlayer , Ġwe Ġuse Ġmean Ġpool ing Ġmethod Ġto Ġmerge Ġthe Ġpreviously Ġobtained Ġtable Ġcell Ġrepresentations Ġin Ġthe Ġsame Ġrow Ġinto Ġthe Ġrepresentation Ġof Ġthe Ġtable 's Ġrow . ĠThen , Ġwe Ġuse Ġself - att ention Ġwith Ġcontent Ġselection Ġgate ĠB IB REF 4 Ġto Ġfilter Ġun important Ġrows ' Ġinformation . ĠTo Ġthe Ġbest Ġof Ġour Ġknowledge , Ġthis Ġis Ġthe Ġfirst Ġwork Ġon Ġneural Ġtable - to - text Ġgeneration Ġvia Ġmodeling Ġcolumn Ġand Ġtime Ġdimension Ġinformation Ġso Ġfar . ĠWe Ġconducted Ġexperiments Ġon ĠR OT OW IRE . ĠResults Ġshow Ġthat Ġour Ġmodel Ġoutper forms Ġexisting Ġsystems , Ġimproving Ġbaseline ĠB LE U Ġfrom Ġ14 . 19 Ġto Ġ16 . 85 Ġ($ + 18 . 75 \ % $ ), ĠP % Ġof Ġrelation Ġgeneration Ġ( RG ) Ġfrom Ġ74 . 80 Ġto Ġ91 . 46 Ġ($ + 22 . 27 \ % $ ), ĠF 1 % Ġof Ġcontent Ġselection Ġ( CS ) Ġfrom Ġ32 . 49 Ġto Ġ41 . 21 Ġ($ + 26 . 84 \ % $ ) Ġand Ġcontent Ġordering Ġ( CO ) Ġfrom Ġ15 . 42 Ġto Ġ20 . 86 Ġ($ + 35 . 28 \ % $ ) Ġon Ġtest Ġset . ĠIt Ġalso Ġexceeds Ġthe Ġstate - of - the - art Ġmodel Ġin Ġterms Ġof Ġthose Ġmetrics . </s> P rel im in aries Ġ: :: ĠNot ations </s> The Ġinput Ġto Ġthe Ġmodel Ġare Ġtables Ġ$ S = \ l brace Ġs ^{ 1 }, Ġs ^{ 2 }, Ġs ^{ 3 }\ r brace Ġ$ . Ġ$ s ^{ 1 } $, Ġ$ s ^{ 2 } $, Ġand Ġ$ s ^{ 3 } $ Ġcontain Ġrecords Ġabout Ġplayers ' Ġperformance Ġin Ġhome Ġteam , Ġplayers ' Ġperformance Ġin Ġvisiting Ġteam Ġand Ġteam 's Ġoverall Ġperformance Ġrespectively . ĠWe Ġregard Ġeach Ġcell Ġin Ġthe Ġtable Ġas Ġrecord . ĠEach Ġrecord Ġ$ r $ Ġconsists Ġof Ġfour Ġtypes Ġof Ġinformation Ġincluding Ġvalue Ġ$ r . v $ Ġ( e . g . Ġ18 ), Ġentity Ġ$ r . e $ Ġ( e . g . ĠAl ĠJefferson ), Ġtype Ġ$ r . c $ Ġ( e . g . ĠPO IN TS ) Ġand Ġa Ġfeature Ġ$ r . f $ Ġ( e . g . Ġvisiting ) Ġwhich Ġindicate Ġwhether Ġa Ġplayer Ġor Ġa Ġteam Ġcompete Ġin Ġhome Ġcourt Ġor Ġnot . ĠEach Ġplayer Ġor Ġteam Ġtakes Ġone Ġrow Ġin Ġthe Ġtable Ġand Ġeach Ġcolumn Ġcontains Ġa Ġtype Ġof Ġrecord Ġsuch Ġas Ġpoints , Ġassists , Ġetc . ĠAlso , Ġtables Ġcontain Ġthe Ġdate Ġwhen Ġthe Ġmatch Ġhappened Ġand Ġwe Ġlet Ġ$ k $ Ġdenote Ġthe Ġdate Ġof Ġthe Ġrecord . ĠWe Ġalso Ġcreate Ġtimelines Ġfor Ġrecords . ĠThe Ġdetails Ġof Ġtimeline Ġconstruction Ġis Ġdescribed Ġin ĠSection ĠSEC REF 4 . ĠFor Ġsimplicity , Ġwe Ġomit Ġtable Ġid Ġ$ l $ Ġand Ġrecord Ġdate Ġ$ k $ Ġin Ġthe Ġfollowing Ġsections Ġand Ġlet Ġ$ r _{ i , j } $ Ġdenotes Ġa Ġrecord Ġof Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn Ġin Ġthe Ġtable . ĠWe Ġassume Ġthe Ġrecords Ġcome Ġfrom Ġthe Ġsame Ġtable Ġand Ġ$ k $ Ġis Ġthe Ġdate Ġof Ġthe Ġmentioned Ġrecord . ĠGiven Ġthose Ġinformation , Ġthe Ġmodel Ġis Ġexpected Ġto Ġgenerate Ġtext Ġ$ y =( y _{ 1 }, Ġ... , Ġy _{ t }, Ġ... , Ġy _{ T }) $ Ġdescribing Ġthese Ġtables . Ġ$ T $ Ġdenotes Ġthe Ġlength Ġof Ġthe Ġtext . </s> P rel im in aries Ġ: :: ĠRecord ĠTimeline ĠCon str c ution </s> In Ġthis Ġpaper , Ġwe Ġconstruct Ġtimelines Ġ$ tl = \ l brace Ġt l _{ e , c }\ r brace Ġ_ { e = 1 , c = 1 } ^{ E , C } $ Ġfor Ġrecords . Ġ$ E $ Ġdenotes Ġthe Ġnumber Ġof Ġdistinct Ġrecord Ġentities Ġand Ġ$ C $ Ġdenotes Ġthe Ġnumber Ġof Ġrecord Ġtypes . ĠFor Ġeach Ġtimeline Ġ$ tl _{ e , c } $, Ġwe Ġfirst Ġextract Ġrecords Ġwith Ġthe Ġsame Ġentity Ġ$ e $ Ġand Ġtype Ġ$ c $ Ġfrom Ġdataset . ĠThen Ġwe Ġsort Ġthem Ġinto Ġa Ġsequence Ġaccording Ġto Ġthe Ġrecord 's Ġdate Ġfrom Ġold Ġto Ġnew . ĠThis Ġsequence Ġis Ġconsidered Ġas Ġtimeline Ġ$ tl _{ e , c } $. ĠFor Ġexample , Ġin ĠFigure ĠFIG REF 3 , Ġthe ĠâĢ ľ Tim eline âĢ Ŀ Ġpart Ġin Ġthe Ġlower - left Ġcorner Ġrepresents Ġa Ġtimeline Ġfor Ġentity ĠAl ĠJefferson Ġand Ġtype ĠPTS Ġ( points ). </s> P rel im in aries Ġ: :: ĠBas eline ĠModel </s> We Ġuse ĠSe q 2 Se q Ġmodel Ġwith Ġattention ĠB IB REF 5 Ġand Ġconditional Ġcopy ĠB IB REF 3 Ġas Ġthe Ġbase Ġmodel . ĠDuring Ġtraining , Ġgiven Ġtables Ġ$ S $ Ġand Ġtheir Ġcorresponding Ġreference Ġtexts Ġ$ y $, Ġthe Ġmodel Ġmaxim ized Ġthe Ġconditional Ġprobability Ġ$ P ( y | S ) Ġ= Ġ\ pro d Ġ_ { t = 1 } ^{ T } P ( y _{ t } | y _{ < t }, S ) $ Ġ. Ġ$ t $ Ġis Ġthe Ġtim est ep Ġof Ġdec oder . ĠFirst , Ġfor Ġeach Ġrecord Ġof Ġthe Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn Ġin Ġthe Ġtable , Ġwe Ġutilize Ġ1 - layer ĠML P Ġto Ġencode Ġthe Ġembed d ings Ġof Ġeach Ġrecord 's Ġfour Ġtypes Ġof Ġinformation Ġinto Ġa Ġdense Ġvector Ġ${ r } _{ i , Ġj } $, Ġ$ r _{ i , Ġj } = Re LU ( W _{ a } [ r _{ i , Ġj }. e ; r _{ i , Ġj }. c ; r _{ i , Ġj }. v ; r _{ i , Ġj }. f ]+ b _{ a }) $. Ġ$ W _{ a } $ Ġand Ġ$ b _{ a } $ Ġare Ġtrain able Ġparameters . ĠThe Ġword Ġembed d ings Ġfor Ġeach Ġtype Ġof Ġinformation Ġare Ġtrain able Ġand Ġrandomly Ġinitialized Ġbefore Ġtraining Ġfollowing ĠWis eman . Ġ$ [ ; ] $ Ġdenotes Ġthe Ġvector Ġconc aten ation . ĠThen , Ġwe Ġuse Ġa ĠL ST M Ġdec oder Ġwith Ġattention Ġand Ġconditional Ġcopy Ġto Ġmodel Ġthe Ġconditional Ġprobability Ġ$ P ( y _{ t } | y _{ < t }, S ) $. ĠThe Ġbase Ġmodel Ġfirst Ġuse Ġattention Ġmechanism ĠB IB REF 5 Ġto Ġfind Ġrelevant Ġrecords Ġfrom Ġthe Ġinput Ġtables Ġand Ġrepresent Ġthem Ġas Ġcontext Ġvector . ĠPlease Ġnote Ġthat Ġthe Ġbase Ġmodel Ġdoesn 't Ġutilize Ġthe Ġstructure Ġof Ġthree Ġtables Ġand Ġnormal ize Ġthe Ġattention Ġweight Ġ$\ alpha Ġ_ { t , Ġi ^ {\ prime Ġ}, Ġj ^ {\ prime Ġ}} $ Ġacross Ġevery Ġrecords Ġin Ġevery Ġtables . ĠThen Ġit Ġcombines Ġthe Ġcontext Ġvector Ġwith Ġdec oder 's Ġhidden Ġstate Ġ$ d _{ t } $ Ġand Ġform Ġa Ġnew Ġattention al Ġhidden Ġstate Ġ$\ t ilde { d } _{ t } $ Ġwhich Ġis Ġused Ġto Ġgenerate Ġwords Ġfrom Ġvocabulary Ġ$ P _{ gen } ( y _{ t } | y _{ < t }, S )= soft max ( W _{ d }\ t ilde { d } _{ t } + b _{ d }) $ ĠAlso Ġthe Ġconditional Ġcopy Ġmechanism Ġis Ġadopted Ġin Ġbase Ġmodel . ĠIt Ġintroduces Ġa Ġvariable Ġ$ z _{ t } $ Ġto Ġdecide Ġwhether Ġto Ġcopy Ġfrom Ġtables Ġor Ġgenerate Ġfrom Ġvocabulary . ĠThe Ġprobability Ġto Ġcopy Ġfrom Ġtable Ġis Ġ$ P ( z _{ t } = 1 | y _{ < t }, ĠS )= s igm oid ( w _{ e } Ġ\ cd ot Ġd _{ t } + b _{ e }) $. ĠThen Ġit Ġdecom poses Ġthe Ġconditional Ġprobability Ġof Ġgenerating Ġ$ t ^{ th } $ Ġword Ġ$ P ( y _{ t } | y _{ < t }, S ) $, Ġgiven Ġthe Ġtables Ġ$ S $ Ġand Ġpreviously Ġgenerated Ġwords Ġ$ y _{ < t } $, Ġas Ġfollows . </s> App roach </s> In Ġthis Ġsection , Ġwe Ġpropose Ġan Ġeffective Ġhierarchical Ġenc oder Ġto Ġutilize Ġthree Ġdimensional Ġstructure Ġof Ġinput Ġdata Ġin Ġorder Ġto Ġimprove Ġtable Ġrepresentation . ĠThose Ġthree Ġdimensions Ġinclude Ġrow , Ġcolumn Ġand Ġtime . ĠAs Ġshown Ġin ĠFigure ĠFIG REF 3 , Ġduring Ġencoding , Ġour Ġmodel Ġconsists Ġof Ġthree Ġlayers Ġincluding Ġrecord Ġenc od ers , Ġrecord Ġfusion Ġgate Ġand Ġrow - level Ġenc oder . ĠGiven Ġtables Ġ$ S $ Ġas Ġdescribed Ġin ĠSection ĠSEC REF 2 , Ġwe Ġfirst Ġencode Ġeach Ġrecord Ġin Ġeach Ġdimension Ġrespectively . ĠThen Ġwe Ġuse Ġthe Ġrecord Ġfusion Ġgate Ġto Ġcombine Ġthem Ġinto Ġa Ġdense Ġrepresentation . ĠAfterwards , Ġwe Ġobtain Ġrow - level Ġrepresentation Ġvia Ġmean Ġpool ing Ġand Ġself - att ention Ġwith Ġcontent Ġselection Ġgate . ĠIn Ġdecoding Ġphase , Ġthe Ġdec oder Ġcan Ġfirst Ġfind Ġimportant Ġrow Ġthen Ġattend Ġto Ġimportant Ġrecord Ġwhen Ġgenerating Ġtexts . ĠWe Ġdescribe Ġmodel 's Ġdetails Ġin Ġfollowing Ġparts . </s> App roach Ġ: :: ĠLayer Ġ1 : ĠRecord ĠEnc od ers Ġ: :: ĠRow ĠDimension ĠEnc oder </s> Based Ġon Ġour Ġobservation , Ġwhen Ġsomeone 's Ġpoints Ġis Ġmentioned Ġin Ġtexts , Ġsome Ġrelated Ġrecords Ġsuch Ġas ĠâĢ ľ field Ġgoals Ġmade âĢ Ŀ Ġ( F GM ) Ġand ĠâĢ ľ field Ġgoals Ġattempted âĢ Ŀ Ġ( F GA ) Ġwill Ġalso Ġbe Ġincluded Ġin Ġtexts . ĠTaken Ġgold Ġtexts Ġin ĠFigure ĠFIG REF 1 Ġas Ġexample , Ġwhen ĠAl ĠJefferson 's Ġpoint Ġ18 Ġis Ġmentioned , Ġhis ĠF GM Ġ9 Ġand ĠF GA Ġ19 Ġare Ġalso Ġmentioned . ĠThus , Ġwhen Ġmodeling Ġa Ġrecord , Ġother Ġrecords Ġin Ġthe Ġsame Ġrow Ġcan Ġbe Ġuseful . ĠSince Ġthe Ġrecord Ġin Ġthe Ġrow Ġis Ġnot Ġsequential , Ġwe Ġuse Ġa Ġself - att ention Ġnetwork Ġwhich Ġis Ġsimilar Ġto ĠLiu 2018 Learning ST Ġto Ġmodel Ġrecords Ġin Ġthe Ġcontext Ġof Ġother Ġrecords Ġin Ġthe Ġsame Ġrow . ĠLet Ġ$ r _{ i , Ġj } ^{ row } $ Ġbe Ġthe Ġrow Ġdimension Ġrepresentation Ġof Ġthe Ġrecord Ġof Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn . ĠThen , Ġwe Ġobtain Ġthe Ġcontext Ġvector Ġin Ġrow Ġdimension Ġ$ c _{ i , Ġj } ^{ row } $ Ġby Ġattending Ġto Ġother Ġrecords Ġin Ġthe Ġsame Ġrow Ġas Ġfollows . ĠPlease Ġnote Ġthat Ġ$\ alpha Ġ_ { i , Ġj , Ġj ^ {\ prime Ġ}} ^{ row } Ġ\ pro pt o Ġexp ( r _{ i , Ġj } ^{ T } W _{ o }{ r } _{ i , Ġj ^ {\ prime Ġ} }) $ Ġis Ġnormalized Ġacross Ġrecords Ġin Ġthe Ġsame Ġrow Ġ$ i $. Ġ$ W _{ o } $ Ġis Ġa Ġtrain able Ġparameter . </s> Then , Ġwe Ġcombine Ġrecord 's Ġrepresentation Ġwith Ġ$ c _{ i , Ġj } $ Ġand Ġobtain Ġthe Ġrow Ġdimension Ġrecord Ġrepresentation Ġ$ r _{ i , Ġj } ^{ row } = tan h ( W _{ f } [ r _{ i , Ġj }; Ġc _{ i , Ġj } ^{ row } ]) $. Ġ$ W _{ f } $ Ġis Ġa Ġtrain able Ġparameter . </s> App roach Ġ: :: ĠLayer Ġ1 : ĠRecord ĠEnc od ers Ġ: :: ĠColumn ĠDimension ĠEnc oder </s> Each Ġinput Ġtable Ġconsists Ġof Ġmultiple Ġrows Ġand Ġcolumns . ĠEach Ġcolumn Ġin Ġthe Ġtable Ġcovers Ġone Ġtype Ġof Ġinformation Ġsuch Ġas Ġpoints . ĠOnly Ġfew Ġof Ġthe Ġrow Ġmay Ġhave Ġhigh Ġpoints Ġor Ġother Ġtype Ġof Ġinformation Ġand Ġthus Ġbecome Ġthe Ġimportant Ġone . ĠFor Ġexample , Ġin ĠâĢ ľ Column ĠDimension âĢ Ŀ Ġpart Ġof ĠFigure ĠFIG REF 3 , ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġis Ġmore Ġimportant Ġthan ĠâĢ ľ Gary ĠNeal âĢ Ŀ Ġbecause Ġthe Ġformer Ġone Ġhave Ġmore Ġimpressive Ġpoints . ĠTherefore , Ġwhen Ġencoding Ġa Ġrecord , Ġit Ġis Ġhelpful Ġto Ġcompare Ġit Ġwith Ġother Ġrecords Ġin Ġthe Ġsame Ġcolumn Ġin Ġorder Ġto Ġunderstand Ġthe Ġperformance Ġlevel Ġreflected Ġby Ġthe Ġrecord Ġamong Ġhis Ġteammates Ġ( rows ). ĠWe Ġemploy Ġself - att ention Ġsimilar Ġto Ġthe Ġone Ġused Ġin ĠSection ĠSEC REF 8 Ġin Ġcolumn Ġdimension Ġto Ġcompare Ġbetween Ġrecords . ĠWe Ġlet Ġ${ r _{ i , Ġj } ^{ col }} $ Ġbe Ġthe Ġcolumn Ġrepresentation Ġof Ġthe Ġrecord Ġof Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn . ĠWe Ġobtain Ġcontext Ġvector Ġin Ġcolumn Ġdimension Ġ$ c _{ i , Ġj } ^{ col } $ Ġas Ġfollows . ĠPlease Ġnote Ġthat Ġ$\ alpha Ġ_ { j , Ġi , Ġi ^ {\ prime Ġ}} $ Ġis Ġnormalized Ġacross Ġrecords Ġfrom Ġdifferent Ġrows Ġ$ i ^ {\ prime Ġ} $ Ġbut Ġof Ġthe Ġsame Ġcolumn Ġ$ j $. ĠThe Ġcolumn Ġdimension Ġrepresentation Ġ$ r _{ i , Ġj } ^{ col } $ Ġis Ġobtained Ġsimilar Ġto Ġrow Ġdimension . </s> App roach Ġ: :: ĠLayer Ġ1 : ĠRecord ĠEnc od ers Ġ: :: ĠTime ĠDimension ĠEnc oder </s> As Ġmentioned Ġin ĠSection ĠSEC REF 1 , Ġwe Ġfind Ġsome Ġexpressions Ġin Ġtexts Ġrequire Ġinformation Ġabout Ġplayers ' Ġhistorical Ġ( time Ġdimension ) Ġperformance . ĠSo Ġthe Ġhistory Ġinformation Ġof Ġrecord Ġ$ r _{ i , j } $ Ġis Ġimportant . ĠNote Ġthat Ġwe Ġhave Ġalready Ġconstructed Ġtimeline Ġfor Ġeach Ġrecord Ġentity Ġand Ġtype Ġas Ġdescribed Ġin ĠSection ĠSEC REF 4 . ĠGiven Ġthose Ġtimelines , ĠWe Ġcollect Ġrecords Ġwith Ġsame Ġentity Ġand Ġtype Ġin Ġthe Ġtimeline Ġwhich Ġhas Ġdate Ġbefore Ġdate Ġ$ k $ Ġof Ġthe Ġrecord Ġ$ r _{ i , j } $ Ġas Ġhistory Ġinformation . ĠSince Ġfor Ġsome Ġrecord , Ġthe Ġhistory Ġinformation Ġcan Ġbe Ġtoo Ġlong , Ġwe Ġset Ġa Ġhistory Ġwindow . ĠThus , Ġwe Ġkeep Ġmost Ġrecent Ġhistory Ġinformation Ġsequence Ġwithin Ġhistory Ġwindow Ġand Ġdenote Ġthem Ġas Ġ$ hist ( r _{ i , Ġj }) $. ĠWe Ġmodel Ġthis Ġkind Ġof Ġinformation Ġin Ġtime Ġdimension Ġvia Ġself - att ention . ĠHowever , Ġunlike Ġthe Ġun ordered Ġnature Ġof Ġrows Ġand Ġcolumns , Ġthe Ġhistory Ġinformation Ġis Ġsequential . ĠTherefore , Ġwe Ġintroduce Ġa Ġtrain able Ġposition Ġembed ding Ġ$ emb _{ pos } ( k ^ {\ prime Ġ}) $ Ġand Ġadd Ġit Ġto Ġthe Ġrecord 's Ġrepresentation Ġand Ġobtain Ġa Ġnew Ġrecord Ġrepresentation Ġ${ r p } _{ k ^ {\ prime Ġ}} $. ĠIt Ġdenotes Ġthe Ġrepresentation Ġof Ġa Ġrecord Ġwith Ġthe Ġsame Ġentity Ġand Ġtype Ġof Ġ$ r _{ i , j } $ Ġbut Ġof Ġthe Ġdate Ġ$ k ^ {\ prime Ġ} $ Ġbefore Ġ$ k $ Ġin Ġthe Ġcorresponding Ġhistory Ġwindow . ĠWe Ġuse Ġ$ r _{ i , Ġj } ^{ time } $ Ġto Ġdenote Ġthe Ġhistory Ġrepresentation Ġof Ġthe Ġrecord Ġof Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn . ĠThen Ġthe Ġhistory Ġdimension Ġcontext Ġvector Ġis Ġobtained Ġby Ġattending Ġto Ġhistory Ġrecords Ġin Ġthe Ġwindow . ĠPlease Ġnote Ġthat Ġwe Ġuse Ġ1 - layer ĠML P Ġas Ġscore Ġfunction Ġhere Ġand Ġ$\ alpha Ġ_ { k , Ġk ^ {\ prime Ġ}} ^{ time } $ Ġis Ġnormalized Ġwithin Ġthe Ġhistory Ġwindow . ĠWe Ġobtain Ġthe Ġtime Ġdimension Ġrepresentation Ġ$ r _{ i , Ġj } ^{ time } $ Ġsimilar Ġto Ġrow Ġdimension . </s> App roach Ġ: :: ĠLayer Ġ2 : ĠRecord ĠFusion ĠGate </s> After Ġobtaining Ġrecord Ġrepresentations Ġin Ġthree Ġdimension , Ġit Ġis Ġimportant Ġto Ġfigure Ġout Ġwhich Ġrepresentation Ġplays Ġa Ġmore Ġimportant Ġrole Ġin Ġreflecting Ġthe Ġrecord 's Ġinformation . ĠIf Ġa Ġrecord Ġstands Ġout Ġfrom Ġother Ġrow 's Ġrecords Ġof Ġsame Ġcolumn , Ġthe Ġcolumn Ġdimension Ġrepresentation Ġmay Ġhave Ġa Ġhigher Ġweight Ġin Ġforming Ġthe Ġoverall Ġrecord Ġrepresentation . ĠIf Ġa Ġrecord Ġdiffers Ġfrom Ġprevious Ġmatch Ġsignificantly , Ġthe Ġhistory Ġdimension Ġrepresentation Ġmay Ġhave Ġa Ġhigher Ġweight . ĠAlso , Ġsome Ġtypes Ġof Ġinformation Ġmay Ġappear Ġin Ġtexts Ġmore Ġfrequently Ġtogether Ġwhich Ġcan Ġbe Ġreflected Ġby Ġrow Ġdimension Ġrepresentation . ĠTherefore , Ġwe Ġpropose Ġa Ġrecord Ġfusion Ġgate Ġto Ġadapt ively Ġcombine Ġall Ġthree Ġdimension Ġrepresentations . ĠFirst , Ġwe Ġconc aten ate Ġ$ r _{ i , Ġj } ^{ row } $, Ġ$ r _{ i , Ġj } ^{ col } $ Ġand Ġ$ r _{ i , Ġj } ^{ time } $, Ġthen Ġadopt Ġa Ġ1 - layer ĠML P Ġto Ġobtain Ġa Ġgeneral Ġrepresentation Ġ$ r _{ i , Ġj } ^{ gen } $ Ġwhich Ġwe Ġconsider Ġas Ġa Ġbaseline Ġrepresentation Ġof Ġrecords ' Ġinformation . ĠThen , Ġwe Ġcompare Ġeach Ġdimension Ġrepresentation Ġwith Ġthe Ġbaseline Ġand Ġobtain Ġits Ġweight Ġin Ġthe Ġfinal Ġrecord Ġrepresentation . ĠWe Ġuse Ġ1 - layer ĠML P Ġas Ġthe Ġscore Ġfunction . ĠEqu ation ĠDIS PLAY _ FORM 15 Ġshows Ġan Ġexample Ġof Ġcalculating Ġcolumn Ġdimension Ġrepresentation 's Ġweight Ġin Ġthe Ġfinal Ġrecord Ġrepresentation . ĠThe Ġweight Ġof Ġrow Ġand Ġtime Ġdimension Ġrepresentation Ġis Ġobtained Ġsimilar Ġto Ġthe Ġweight Ġof Ġcolumn Ġdimension Ġrepresentation . </s> In Ġthe Ġend , Ġthe Ġfused Ġrecord Ġrepresentation Ġ$\ t ilde { r } _{ i , Ġj } $ Ġis Ġthe Ġweighted Ġsum Ġof Ġthe Ġthree Ġdimension Ġrepresentations . </s> App roach Ġ: :: ĠLayer Ġ3 : ĠRow - level ĠEnc oder </s> For Ġeach Ġrow , Ġwe Ġcombine Ġits Ġrecords Ġvia Ġmean Ġpool ing Ġ( Equ ation ĠDIS PLAY _ FORM 18 ) Ġin Ġorder Ġto Ġobtain Ġa Ġgeneral Ġrepresentation Ġof Ġthe Ġrow Ġwhich Ġmay Ġreflect Ġthe Ġrow Ġ( player Ġor Ġteam )' s Ġoverall Ġperformance . Ġ$ C $ Ġdenotes Ġthe Ġnumber Ġof Ġcolumns . </s> Then , Ġwe Ġadopt Ġcontent Ġselection Ġgate Ġ$ g _{ i } $, Ġwhich Ġis Ġproposed Ġby ĠP ud upp ully Ġon Ġrows ' Ġrepresentations Ġ$ row _{ i } $, Ġand Ġobtain Ġa Ġnew Ġrepresentation Ġ$\ t ilde { row } _{ i } = g _{ i }\ od ot Ġrow _{ i } $ Ġto Ġchoose Ġmore Ġimportant Ġinformation Ġbased Ġon Ġeach Ġrow 's Ġcontext . </s> App roach Ġ: :: ĠDecoder Ġwith ĠDual ĠAttention </s> Since Ġrecord Ġenc od ers Ġwith Ġrecord Ġfusion Ġgate Ġprovide Ġrecord - level Ġrepresentation Ġand Ġrow - level Ġenc oder Ġprovides Ġrow - level Ġrepresentation . ĠInspired Ġby ĠN 18 - 20 97 , Ġwe Ġcan Ġmodify Ġthe Ġdec oder Ġin Ġbase Ġmodel Ġto Ġfirst Ġchoose Ġimportant Ġrow Ġthen Ġattend Ġto Ġrecords Ġwhen Ġgenerating Ġeach Ġword . ĠFollowing Ġnot ations Ġin ĠSection ĠSEC REF 5 , Ġ$\ beta Ġ_ { t , Ġi } Ġ\ pro pt o Ġexp ( score ( d _{ t }, Ġrow _{ i } )) $ Ġob tains Ġthe Ġattention Ġweight Ġwith Ġrespect Ġto Ġeach Ġrow . ĠPlease Ġnote Ġthat Ġ$\ beta Ġ_ { t , Ġi } $ Ġis Ġnormalized Ġacross Ġall Ġrow - level Ġrepresentations Ġfrom Ġall Ġthree Ġtables . ĠThen , Ġ$\ gam ma Ġ_ { t , Ġi , Ġj } Ġ\ pro pt o Ġexp ( score ( d _{ t }, Ġ\ t ilde { r } _{ i , Ġj } )) $ Ġob tains Ġattention Ġweight Ġfor Ġrecords . ĠPlease Ġnote Ġthat Ġwe Ġnormal ize Ġ$\ gam ma Ġ_ { t , Ġi , Ġj } $ Ġamong Ġrecords Ġin Ġthe Ġsame Ġrow . </s> We Ġuse Ġthe Ġrow - level Ġattention Ġ$\ beta Ġ_ { t , Ġi } $ Ġas Ġguidance Ġfor Ġchoosing Ġrow Ġbased Ġon Ġrow 's Ġgeneral Ġrepresentation . ĠThen Ġwe Ġuse Ġit Ġto Ġre - weight Ġthe Ġrecord - level Ġattention Ġ$\ gam ma Ġ_ { t , Ġi , Ġj } $ Ġand Ġchange Ġthe Ġattention Ġweight Ġin Ġbase Ġmodel Ġto Ġ$\ t ilde {\ alpha Ġ} _{ t , Ġi , Ġj } $. ĠPlease Ġnote Ġthat Ġ${ \ t ilde {\ alpha Ġ} _{ t , Ġi , Ġj }} $ Ġsum Ġto Ġ1 Ġacross Ġall Ġrecords Ġin Ġall Ġtables . </s> App roach Ġ: :: ĠTraining </s> Given Ġa Ġbatch Ġof Ġinput Ġtables Ġ$\ l brace ĠS \ r brace Ġ_ { G } $ Ġand Ġreference Ġoutput Ġ$\ l brace ĠY \ r brace Ġ_ { G } $, Ġwe Ġuse Ġnegative Ġlog - like lihood Ġas Ġthe Ġloss Ġfunction Ġfor Ġour Ġmodel . ĠWe Ġtrain Ġthe Ġmodel Ġby Ġminimizing Ġ$ L $. Ġ$ G $ Ġis Ġthe Ġnumber Ġof Ġexamples Ġin Ġthe Ġbatch Ġand Ġ$ T _{ g } $ Ġrepresents Ġthe Ġlength Ġof Ġ$ g ^{ th } $ Ġreference 's Ġlength . </s> Exper iments Ġ: :: ĠDat as et Ġand ĠEvaluation ĠMet rics </s> We Ġconducted Ġexperiments Ġon ĠR OT OW IRE ĠB IB REF 2 . ĠFor Ġeach Ġexample , Ġit Ġprovides Ġthree Ġtables Ġas Ġdescribed Ġin ĠSection ĠSEC REF 2 Ġwhich Ġconsists Ġof Ġ6 28 Ġrecords Ġin Ġtotal Ġwith Ġa Ġlong Ġgame Ġsummary . ĠThe Ġaverage Ġlength Ġof Ġgame Ġsummary Ġis Ġ337 . 1 . ĠIn Ġthis Ġpaper , Ġwe Ġfollowed Ġthe Ġdata Ġsplit Ġintroduced Ġin ĠWis eman : Ġ3 398 Ġexamples Ġin Ġtraining Ġset , Ġ7 27 Ġexamples Ġin Ġdevelopment Ġset Ġand Ġ7 28 Ġexamples Ġin Ġtest Ġset . ĠWe Ġfollowed ĠWis eman 's Ġwork Ġand Ġuse ĠB LE U ĠB IB REF 7 Ġand Ġthree Ġextract ive Ġevaluation Ġmetrics ĠRG , ĠCS Ġand ĠCO ĠB IB REF 2 Ġfor Ġevaluation . ĠThe Ġmain Ġidea Ġof Ġthe Ġextract ive Ġevaluation Ġmetrics Ġis Ġto Ġuse Ġan ĠInformation ĠExt raction Ġ( IE ) Ġmodel Ġto Ġidentify Ġrecords Ġmentioned Ġin Ġtexts . ĠThen Ġcompare Ġthem Ġwith Ġtables Ġor Ġrecords Ġextracted Ġfrom Ġreference Ġto Ġevaluate Ġthe Ġmodel . ĠRG Ġ( Rel ation ĠGeneration ) Ġmeasures Ġcontent Ġfidelity Ġof Ġtexts . ĠCS Ġ( Content ĠSelection ) Ġmeasures Ġmodel 's Ġability Ġon Ġcontent Ġselection . ĠCO Ġ( Content ĠOrder ing ) Ġmeasures Ġmodel 's Ġability Ġon Ġordering Ġthe Ġchosen Ġrecords Ġin Ġtexts . ĠWe Ġrefer Ġthe Ġreaders Ġto ĠWis eman 's Ġpaper Ġfor Ġmore Ġdetails . </s> Exper iments Ġ: :: ĠImplementation ĠDetails </s> Following Ġconfigurations Ġin ĠP ud upp ully , Ġwe Ġset Ġword Ġembed ding Ġand ĠL ST M Ġdec oder Ġhidden Ġsize Ġas Ġ600 . ĠThe Ġdec oder 's Ġlayer Ġwas Ġset Ġto Ġbe Ġ2 . ĠInput Ġfeeding ĠB IB REF 5 Ġwas Ġalso Ġused Ġfor Ġdec oder . ĠWe Ġapplied Ġdrop out Ġat Ġa Ġrate Ġ0 . 3 . ĠFor Ġtraining , Ġwe Ġused ĠAd ag rad ĠB IB REF 8 Ġoptim izer Ġwith Ġlearning Ġrate Ġof Ġ0 . 15 , Ġtrunc ated ĠB PT T Ġ( block Ġlength Ġ100 ), Ġbatch Ġsize Ġof Ġ5 Ġand Ġlearning Ġrate Ġdecay Ġof Ġ0 . 97 . ĠFor Ġinfer ring , Ġwe Ġset Ġbeam Ġsize Ġas Ġ5 . ĠWe Ġalso Ġset Ġthe Ġhistory Ġwindows Ġsize Ġas Ġ3 Ġfrom Ġ{ 3 , 5 , 7 } Ġbased Ġon Ġthe Ġresults . ĠCode Ġof Ġour Ġmodel Ġcan Ġbe Ġfound Ġat Ġhttps :// github . com / ern est g ong / data 2 text - three - dim ensions /. </s> Exper iments Ġ: :: ĠResults Ġ: :: ĠAutomatic ĠEvaluation </s> Table ĠT AB REF 23 Ġdisplays Ġthe Ġautomatic Ġevaluation Ġresults Ġon Ġboth Ġdevelopment Ġand Ġtest Ġset . ĠWe Ġchose ĠCond itional ĠCopy Ġ( CC ) Ġmodel Ġas Ġour Ġbaseline , Ġwhich Ġis Ġthe Ġbest Ġmodel Ġin ĠWis eman . ĠWe Ġincluded Ġreported Ġscores Ġwith Ġupdated ĠIE Ġmodel Ġby ĠP ud upp ully Ġand Ġour Ġimplementation 's Ġresult Ġon ĠCC Ġin Ġthis Ġpaper . ĠAlso , Ġwe Ġcompared Ġour Ġmodels Ġwith Ġother Ġexisting Ġworks Ġon Ġthis Ġdataset Ġincluding ĠOp ATT ĠB IB REF 6 Ġand ĠNeural ĠContent ĠPlanning Ġwith Ġconditional Ġcopy Ġ( NC P + CC ) ĠB IB REF 4 . ĠIn Ġaddition , Ġwe Ġimplemented Ġthree Ġother Ġhierarchical Ġenc od ers Ġthat Ġencoded Ġtables ' Ġrow Ġdimension Ġinformation Ġin Ġboth Ġrecord - level Ġand Ġrow - level Ġto Ġcompare Ġwith Ġthe Ġhierarchical Ġstructure Ġof Ġenc oder Ġin Ġour Ġmodel . ĠThe Ġdec oder Ġwas Ġequipped Ġwith Ġdual Ġattention ĠB IB REF 9 . ĠThe Ġone Ġwith ĠL ST M Ġcell Ġis Ġsimilar Ġto Ġthe Ġone Ġin ĠN 18 - 20 97 Ġwith Ġ1 Ġlayer Ġfrom Ġ{ 1 , 2 , 3 }. ĠThe Ġone Ġwith ĠCNN Ġcell ĠB IB REF 10 Ġhas Ġkernel Ġwidth Ġ3 Ġfrom Ġ{ 3 , Ġ5 } Ġand Ġ10 Ġlayer Ġfrom Ġ{ 5 , 10 , 15 , 20 }. ĠThe Ġone Ġwith Ġtransformer - style Ġenc oder Ġ( MH SA ) ĠB IB REF 11 Ġhas Ġ8 Ġhead Ġfrom Ġ{ 8 , Ġ10 } Ġand Ġ5 Ġlayer Ġfrom Ġ{ 2 , 3 , 4 , 5 , 6 }. ĠThe Ġheads Ġand Ġlayers Ġmentioned Ġabove Ġwere Ġfor Ġboth Ġrecord - level Ġenc oder Ġand Ġrow - level Ġenc oder Ġrespectively . ĠThe Ġself - att ention Ġ( SA ) Ġcell Ġwe Ġused , Ġas Ġdescribed Ġin ĠSection ĠSEC REF 3 , Ġachieved Ġbetter Ġoverall Ġperformance Ġin Ġterms Ġof ĠF 1 % Ġof ĠCS , ĠCO Ġand ĠB LE U Ġamong Ġthe Ġhierarchical Ġenc od ers . ĠAlso Ġwe Ġimplemented Ġa Ġtemplate Ġsystem Ġsame Ġas Ġthe Ġone Ġused Ġin ĠWis eman Ġwhich Ġoutput ted Ġeight Ġsentences : Ġan Ġintroductory Ġsentence Ġ( two Ġteams ' Ġpoints Ġand Ġwho Ġwin ), Ġsix Ġtop Ġplayers ' Ġstatistics Ġ( ranked Ġby Ġtheir Ġpoints ) Ġand Ġa Ġconclusion Ġsentence . ĠWe Ġrefer Ġthe Ġreaders Ġto ĠWis eman 's Ġpaper Ġfor Ġmore Ġdetailed Ġinformation Ġon Ġtemplates . ĠThe Ġgold Ġreference 's Ġresult Ġis Ġalso Ġincluded Ġin ĠTable ĠT AB REF 23 . ĠOverall , Ġour Ġmodel Ġperforms Ġbetter Ġthan Ġother Ġneural Ġmodels Ġon Ġboth Ġdevelopment Ġand Ġtest Ġset Ġin Ġterms Ġof ĠRG 's ĠP %, ĠF 1 % Ġscore Ġof ĠCS , ĠCO Ġand ĠB LE U , Ġindicating Ġour Ġmodel 's Ġclear Ġimprovement Ġon Ġgenerating Ġhigh - f idelity , Ġinformative Ġand Ġfluent Ġtexts . ĠAlso , Ġour Ġmodel Ġwith Ġthree Ġdimension Ġrepresentations Ġoutper forms Ġhierarchical Ġenc od ers Ġwith Ġonly Ġrow Ġdimension Ġrepresentation Ġon Ġdevelopment Ġset . ĠThis Ġindicates Ġthat Ġcell Ġand Ġtime Ġdimension Ġrepresentation Ġare Ġimportant Ġin Ġrepresenting Ġthe Ġtables . ĠCompared Ġto Ġreported Ġbaseline Ġresult Ġin ĠWis eman , Ġwe Ġachieved Ġimprovement Ġof Ġ$ 22 . 27 \ % $ Ġin Ġterms Ġof ĠRG , Ġ$ 26 . 84 \ % $ Ġin Ġterms Ġof ĠCS ĠF 1 %, Ġ$ 35 . 28 \ % $ Ġin Ġterms Ġof ĠCO Ġand Ġ$ 18 . 75 \ % $ Ġin Ġterms Ġof ĠB LE U Ġon Ġtest Ġset . ĠUn surprisingly , Ġtemplate Ġsystem Ġachieves Ġbest Ġon ĠRG ĠP % Ġand ĠCS ĠR % Ġdue Ġto Ġthe Ġincluded Ġdomain Ġknowledge . ĠAlso , Ġthe Ġhigh ĠRG Ġ# Ġand Ġlow ĠCS ĠP % Ġindicates Ġthat Ġtemplate Ġwill Ġinclude Ġvast Ġinformation Ġwhile Ġmany Ġof Ġthem Ġare Ġdeemed Ġredundant . ĠIn Ġaddition , Ġthe Ġlow ĠCO Ġand Ġlow ĠB LE U Ġindicates Ġthat Ġthe Ġrigid Ġstructure Ġof Ġthe Ġtemplate Ġwill Ġproduce Ġtexts Ġthat Ġaren 't Ġas Ġadaptive Ġto Ġthe Ġgiven Ġtables Ġand Ġnatural Ġas Ġthose Ġproduced Ġby Ġneural Ġmodels . ĠAlso , Ġwe Ġconducted Ġab lation Ġstudy Ġon Ġour Ġmodel Ġto Ġevaluate Ġeach Ġcomponent 's Ġcontribution Ġon Ġdevelopment Ġset . ĠBased Ġon Ġthe Ġresults , Ġthe Ġabsence Ġof Ġrow - level Ġenc oder Ġhurts Ġour Ġmodel 's Ġperformance Ġacross Ġall Ġmetrics Ġespecially Ġthe Ġcontent Ġselection Ġability . </s> Row , Ġcolumn Ġand Ġtime Ġdimension Ġinformation Ġare Ġimportant Ġto Ġthe Ġmodeling Ġof Ġtables Ġbecause Ġsubtract ing Ġany Ġof Ġthem Ġwill Ġresult Ġin Ġperformance Ġdrop . ĠAlso , Ġposition Ġembed ding Ġis Ġcritical Ġwhen Ġmodeling Ġtime Ġdimension Ġinformation Ġaccording Ġto Ġthe Ġresults . ĠIn Ġaddition , Ġrecord Ġfusion Ġgate Ġplays Ġan Ġimportant Ġrole Ġbecause ĠB LE U , ĠCO , ĠRG ĠP % Ġand ĠCS ĠP % Ġdrop Ġsignificantly Ġafter Ġsubtract ing Ġit Ġfrom Ġfull Ġmodel . ĠResults Ġshow Ġthat Ġeach Ġcomponent Ġin Ġthe Ġmodel Ġcontributes Ġto Ġthe Ġoverall Ġperformance . ĠIn Ġaddition , Ġwe Ġcompare Ġour Ġmodel Ġwith Ġdelayed Ġcopy Ġmodel Ġ( D EL ) ĠB IB REF 12 Ġalong Ġwith Ġgold Ġtext , Ġtemplate Ġsystem Ġ( T EM ), Ġconditional Ġcopy Ġ( CC ) ĠB IB REF 2 Ġand ĠN CP + CC Ġ( NC P ) ĠB IB REF 4 . ĠLi 's Ġmodel Ġgenerate Ġa Ġtemplate Ġat Ġfirst Ġand Ġthen Ġfill Ġin Ġthe Ġslots Ġwith Ġdelayed Ġcopy Ġmechanism . ĠSince Ġits Ġresult Ġin ĠLi 's Ġpaper Ġwas Ġevaluated Ġby ĠIE Ġmodel Ġtrained Ġby ĠWis eman Ġand ĠâĢ ľ re lex ical ization âĢ Ŀ Ġby ĠLi , Ġwe Ġadopted Ġthe Ġcorresponding ĠIE Ġmodel Ġand Ġre - im plement ĠâĢ ľ re lex ical ization âĢ Ŀ Ġas Ġsuggested Ġby ĠLi Ġfor Ġfair Ġcomparison . ĠPlease Ġnote Ġthat ĠCC 's Ġevaluation Ġresults Ġvia Ġour Ġre - im ple mented ĠâĢ ľ re lex ical ization âĢ Ŀ Ġis Ġcomparable Ġto Ġthe Ġreported Ġresult Ġin ĠLi . ĠWe Ġapplied Ġthem Ġon Ġmodels Ġother Ġthan ĠDEL Ġas Ġshown Ġin ĠTable ĠT AB REF 28 Ġand Ġreport ĠDEL 's Ġresult Ġfrom ĠB IB REF 12 's Ġpaper . ĠIt Ġshows Ġthat Ġour Ġmodel Ġoutper form ĠLi 's Ġmodel Ġsignificantly Ġacross Ġall Ġautomatic Ġevaluation Ġmetrics Ġin ĠTable ĠT AB REF 28 . </s> Exper iments Ġ: :: ĠResults Ġ: :: ĠHuman ĠEvaluation </s> In Ġthis Ġsection , Ġwe Ġhired Ġthree Ġgraduates Ġwho Ġpassed Ġintermediate ĠEnglish Ġtest Ġ( College ĠEnglish ĠTest ĠBand Ġ6 ) Ġand Ġwere Ġfamiliar Ġwith ĠNBA Ġgames Ġto Ġperform Ġhuman Ġevaluation . </s> First , Ġin Ġorder Ġto Ġcheck Ġif Ġhistory Ġinformation Ġis Ġimportant , Ġwe Ġsampled Ġ100 Ġsumm aries Ġfrom Ġtraining Ġset Ġand Ġasked Ġrat ers Ġto Ġmanually Ġcheck Ġwhether Ġthe Ġsummary Ġcontained Ġexpressions Ġthat Ġneed Ġto Ġbe Ġinferred Ġfrom Ġhistory Ġinformation . ĠIt Ġturns Ġout Ġthat Ġ$ 56 . 7 \ % $ Ġsumm aries Ġof Ġthe Ġsampled Ġsumm aries Ġneed Ġhistory Ġinformation . </s> Following Ġhuman Ġevaluation Ġsettings Ġin ĠP ud upp ully , Ġwe Ġconducted Ġthe Ġfollowing Ġhuman Ġevaluation Ġexperiments Ġat Ġthe Ġsame Ġscale . ĠThe Ġsecond Ġexperiment Ġis Ġto Ġassess Ġwhether Ġthe Ġimprovement Ġon Ġrelation Ġgeneration Ġmetric Ġreported Ġin Ġautomatic Ġevaluation Ġis Ġsupported Ġby Ġhuman Ġevaluation . ĠWe Ġcompared Ġour Ġfull Ġmodel Ġwith Ġgold Ġtexts , Ġtemplate - based Ġsystem , ĠCC ĠB IB REF 2 Ġand ĠN CP + CC Ġ( NC P ) ĠB IB REF 4 . ĠWe Ġrandomly Ġsampled Ġ30 Ġexamples Ġfrom Ġtest Ġset . ĠThen , Ġwe Ġrandomly Ġsampled Ġ4 Ġsentences Ġfrom Ġeach Ġmodel 's Ġoutput Ġfor Ġeach Ġexample . ĠWe Ġprovided Ġthe Ġrat ers Ġof Ġthose Ġsampled Ġsentences Ġwith Ġthe Ġcorresponding ĠNBA Ġgame Ġstatistics . ĠThey Ġwere Ġasked Ġto Ġcount Ġthe Ġnumber Ġof Ġsupporting Ġand Ġcontradict ing Ġfacts Ġin Ġeach Ġsentence . ĠEach Ġsentence Ġis Ġrated Ġindependently . ĠWe Ġreport Ġthe Ġaverage Ġnumber Ġof Ġsupporting Ġfacts Ġ(# Sup ) Ġand Ġcontradict ing Ġfacts Ġ(# Cont ) Ġin ĠTable ĠT AB REF 30 . ĠUn surprisingly , Ġtemplate - based Ġsystem Ġincludes Ġmost Ġsupporting Ġfacts Ġand Ġleast Ġcontradict ing Ġfacts Ġin Ġits Ġtexts Ġbecause Ġthe Ġtemplate Ġconsists Ġof Ġa Ġlarge Ġnumber Ġof Ġfacts Ġand Ġall Ġof Ġthose Ġfacts Ġare Ġextracted Ġfrom Ġthe Ġtable . ĠAlso , Ġour Ġmodel Ġproduces Ġless Ġcontradict ing Ġfacts Ġthan Ġother Ġtwo Ġneural Ġmodels . ĠAlthough Ġour Ġmodel Ġproduces Ġless Ġsupporting Ġfacts Ġthan ĠN CP Ġand ĠCC , Ġit Ġstill Ġincludes Ġenough Ġsupporting Ġfacts Ġ( s lightly Ġmore Ġthan Ġgold Ġtexts ). ĠAlso , Ġcomparing Ġto ĠN CP + CC Ġ( NC P ) âĢ Ļ s Ġtendency Ġto Ġinclude Ġvast Ġinformation Ġthat Ġcontain Ġredundant Ġinformation , Ġour Ġmodel âĢ Ļ s Ġability Ġto Ġselect Ġand Ġaccurately Ġconvey Ġinformation Ġis Ġbetter . ĠAll Ġother Ġresults Ġ( Gold , ĠCC , ĠN CP Ġand Ġours ) Ġare Ġsignificantly Ġdifferent Ġfrom Ġtemplate - based Ġsystem 's Ġresults Ġin Ġterms Ġof Ġnumber Ġof Ġsupporting Ġfacts Ġaccording Ġto Ġone - way ĠAN OVA Ġwith Ġpost h oc ĠTu key ĠH SD Ġtests . ĠAll Ġsignificance Ġdifference Ġreported Ġin Ġthis Ġpaper Ġare Ġless Ġthan Ġ0 . 05 . ĠOur Ġmodel Ġis Ġalso Ġsignificantly Ġdifferent Ġfrom Ġthe ĠN CP Ġmodel . ĠAs Ġfor Ġaverage Ġnumber Ġof Ġcontradict ing Ġfacts , Ġour Ġmodel Ġis Ġsignificantly Ġdifferent Ġfrom Ġother Ġtwo Ġneural Ġmodels . ĠSurprisingly , Ġgold Ġtexts Ġwere Ġfound Ġcontaining Ġcontradict ing Ġfacts . ĠWe Ġchecked Ġthe Ġrat ers 's Ġresult Ġand Ġfound Ġthat Ġgold Ġtexts Ġoccasionally Ġinclude Ġwrong Ġfield - goal Ġor Ġthree - point Ġpercent Ġor Ġwrong Ġpoints Ġdifference Ġbetween Ġthe Ġwinner Ġand Ġthe Ġdefeated Ġteam . ĠWe Ġcan Ġtreat Ġthe Ġaverage Ġcontradict ing Ġfacts Ġnumber Ġof Ġgold Ġtexts Ġas Ġa Ġlower Ġbound . </s> In Ġthe Ġthird Ġexperiment , Ġfollowing ĠP ud upp ully , Ġwe Ġasked Ġrat ers Ġto Ġevaluate Ġthose Ġmodels Ġin Ġterms Ġof Ġgram matical ity Ġ( is Ġit Ġmore Ġfluent Ġand Ġgram matical ?), Ġco herence Ġ( is Ġit Ġeasier Ġto Ġread Ġor Ġfollows Ġmore Ġnatural Ġordering Ġof Ġfacts ? Ġ) Ġand Ġconc is eness Ġ( does Ġit Ġavoid Ġredundant Ġinformation Ġand Ġrepet itions ?). ĠWe Ġadopted Ġthe Ġsame Ġ30 Ġexamples Ġfrom Ġabove Ġand Ġarranged Ġevery Ġ5 - t uple Ġof Ġsumm aries Ġinto Ġ10 Ġpairs . ĠThen , Ġwe Ġasked Ġthe Ġrat ers Ġto Ġchoose Ġwhich Ġsystem Ġperforms Ġthe Ġbest Ġgiven Ġeach Ġpair . ĠScores Ġare Ġcomputed Ġas Ġthe Ġdifference Ġbetween Ġpercentage Ġof Ġtimes Ġwhen Ġthe Ġmodel Ġis Ġchosen Ġas Ġthe Ġbest Ġand Ġpercentage Ġof Ġtimes Ġwhen Ġthe Ġmodel Ġis Ġchosen Ġas Ġthe Ġworst . ĠGold Ġtexts Ġis Ġsignificantly Ġmore Ġgram matical Ġthan Ġothers Ġacross Ġall Ġthree Ġmetrics . ĠAlso , Ġour Ġmodel Ġperforms Ġsignificantly Ġbetter Ġthan Ġother Ġtwo Ġneural Ġmodels Ġ( CC , ĠN CP ) Ġin Ġall Ġthree Ġmetrics . ĠTemplate - based Ġsystem Ġgenerates Ġsignificantly Ġmore Ġgram matical Ġand Ġconcise Ġbut Ġsignificantly Ġless Ġcoherent Ġresults , Ġcompared Ġto Ġall Ġthree Ġneural Ġmodels . ĠBecause Ġthe Ġrigid Ġstructure Ġof Ġtexts Ġensures Ġthe Ġcorrect Ġgram matical ity Ġand Ġno Ġrepetition Ġin Ġtemplate - based Ġsystem 's Ġoutput . ĠHowever , Ġsince Ġthe Ġtemplates Ġare Ġst il ted Ġand Ġlack Ġvariability Ġcompared Ġto Ġothers , Ġit Ġwas Ġdeemed Ġless Ġcoherent Ġthan Ġthe Ġothers Ġby Ġthe Ġrat ers . </s> Exper iments Ġ: :: ĠResults Ġ: :: ĠQual itative ĠExample </s> Figure ĠFIG REF 32 Ġshows Ġan Ġexample Ġgenerated Ġby Ġour Ġmodel . ĠIt Ġevidently Ġhas Ġseveral Ġnice Ġproperties : Ġit Ġcan Ġaccurately Ġselect Ġimportant Ġplayer ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġfrom Ġthe Ġtables Ġwho Ġis Ġneglected Ġby Ġbaseline Ġmodel , Ġwhich Ġneed Ġthe Ġmodel Ġto Ġunderstand Ġperformance Ġdifference Ġof Ġa Ġtype Ġof Ġdata Ġ( column ) Ġbetween Ġeach Ġrows Ġ( players ). ĠAlso Ġit Ġcorrectly Ġsummarize Ġperformance Ġof ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġin Ġthis Ġmatch Ġas ĠâĢ ľ double - double âĢ Ŀ Ġwhich Ġrequires Ġability Ġto Ġcapture Ġdependency Ġfrom Ġdifferent Ġcolumns Ġ( different Ġtype Ġof Ġrecord ) Ġin Ġthe Ġsame Ġrow Ġ( player ). ĠIn Ġaddition , Ġit Ġmodels ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġhistory Ġperformance Ġand Ġcorrectly Ġstates Ġthat ĠâĢ ľ It Ġwas Ġhis Ġsecond Ġdouble - double Ġover Ġhis Ġlast Ġthree Ġgames âĢ Ŀ , Ġwhich Ġis Ġalso Ġmentioned Ġin Ġgold Ġtexts Ġincluded Ġin ĠFigure Ġ1 Ġin Ġa Ġsimilar Ġway . </s> Related ĠWork </s> In Ġrecent Ġyears , Ġneural Ġdata - to - text Ġsystems Ġmake Ġremarkable Ġprogress Ġon Ġgenerating Ġtexts Ġdirectly Ġfrom Ġdata . ĠMei 2016 What TT Ġproposes Ġan Ġenc oder - align er - dec oder Ġmodel Ġto Ġgenerate Ġweather Ġforecast , Ġwhile ĠN 18 - 20 98 Ġpropose Ġa Ġmixed Ġhierarchical Ġattention . ĠAAA I 18 16 203 Ġproposes Ġa Ġhybrid Ġcontent - Ġand Ġlinkage - based Ġattention Ġmechanism Ġto Ġmodel Ġthe Ġorder Ġof Ġcontent . ĠAAA I 18 165 99 Ġpropose Ġto Ġintegrate Ġfield Ġinformation Ġinto Ġtable Ġrepresentation Ġand Ġenhance Ġdec oder Ġwith Ġdual Ġattention . ĠAAA I 18 16 138 Ġdevelops Ġa Ġtable - aware Ġenc oder - dec oder Ġmodel . ĠWis eman Ġintroduced Ġa Ġdocument - scale Ġdata - to - text Ġdataset , Ġconsisting Ġof Ġlong Ġtext Ġwith Ġmore Ġredundant Ġrecords , Ġwhich Ġrequires Ġthe Ġmodel Ġto Ġselect Ġimportant Ġinformation Ġto Ġgenerate . ĠWe Ġdescribe Ġrecent Ġworks Ġin ĠSection ĠSEC REF 1 . ĠAlso , Ġsome Ġstudies Ġin Ġabstract ive Ġtext Ġsummar ization Ġencode Ġlong Ġtexts Ġin Ġa Ġhierarchical Ġmanner . ĠN 18 - 20 97 Ġuses Ġa Ġhierarchical Ġenc oder Ġto Ġencode Ġinput , Ġpaired Ġwith Ġa Ġdiscourse - aware Ġdec oder . ĠW 17 - 450 5 Ġencode Ġdocument Ġhierarch ically Ġand Ġpropose Ġcoarse - to - fine Ġattention Ġfor Ġdec oder . ĠRecently , Ġli u 2019 h ier arch ical Ġpropose Ġa Ġhierarchical Ġenc oder Ġfor Ġdata - to - text Ġgeneration Ġwhich Ġuses ĠL ST M Ġas Ġits Ġcell . Ġmur ak ami - et al - 2017 - learning Ġpropose Ġto Ġmodel Ġstock Ġmarket Ġtime - series Ġdata Ġand Ġgenerate Ġcomments . ĠAs Ġfor Ġincorporating Ġhistorical Ġbackground Ġin Ġgeneration , Ġrob in 1994 re vision Ġproposed Ġto Ġbuild Ġa Ġdraft Ġwith Ġessential Ġnew Ġfacts Ġat Ġfirst , Ġthen Ġincorporate Ġbackground Ġfacts Ġwhen Ġrev ising Ġthe Ġdraft Ġbased Ġon Ġfunctional Ġunification Ġgram m ars . ĠDifferent Ġfrom Ġthat , Ġwe Ġencode Ġthe Ġhistorical Ġ( time Ġdimension ) Ġinformation Ġin Ġthe Ġneural Ġdata - to - text Ġmodel Ġin Ġan Ġend - to - end Ġfashion . ĠEx isting Ġworks Ġon Ġdata - to - text Ġgeneration Ġneglect Ġthe Ġjoint Ġrepresentation Ġof Ġtables ' Ġrow , Ġcolumn Ġand Ġtime Ġdimension Ġinformation . ĠIn Ġthis Ġpaper , Ġwe Ġpropose Ġan Ġeffective Ġhierarchical Ġenc oder Ġwhich Ġmodels Ġinformation Ġfrom Ġrow , Ġcolumn Ġand Ġtime Ġdimension Ġsimultaneously . </s> Conclusion </s> In Ġthis Ġwork , Ġwe Ġpresent Ġan Ġeffective Ġhierarchical Ġenc oder Ġfor Ġtable - to - text Ġgeneration Ġthat Ġlearns Ġtable Ġrepresentations Ġfrom Ġrow , Ġcolumn Ġand Ġtime Ġdimension . ĠIn Ġdetail , Ġour Ġmodel Ġconsists Ġof Ġthree Ġlayers , Ġwhich Ġlearn Ġrecords ' Ġrepresentation Ġin Ġthree Ġdimension , Ġcombine Ġthose Ġrepresentations Ġvia Ġtheir Ġsail ency Ġand Ġobtain Ġrow - level Ġrepresentation Ġbased Ġon Ġrecords ' Ġrepresentation . ĠThen , Ġduring Ġdecoding , Ġit Ġwill Ġselect Ġimportant Ġtable Ġrow Ġbefore Ġattending Ġto Ġrecords . ĠExper iments Ġare Ġconducted Ġon ĠR OT OW IRE , Ġa Ġbenchmark Ġdataset Ġof ĠNBA Ġgames . ĠBoth Ġautomatic Ġand Ġhuman Ġevaluation Ġresults Ġshow Ġthat Ġour Ġmodel Ġachieves Ġthe Ġnew Ġstate - of - the - art Ġperformance . </s> Acknowled gements </s> We Ġwould Ġlike Ġto Ġthank Ġthe Ġanonymous Ġreviewers Ġfor Ġtheir Ġhelpful Ġcomments . ĠWe 'd Ġalso Ġlike Ġto Ġthank ĠXin wei ĠG eng , ĠY ibo ĠSun , ĠZheng p eng ĠXiang Ġand ĠY uy u ĠChen Ġfor Ġtheir Ġvaluable Ġinput . ĠThis Ġwork Ġwas Ġsupported Ġby Ġthe ĠNational ĠKey ĠR & D ĠProgram Ġof ĠChina Ġvia Ġgrant Ġ2018 Y FB 100 5 103 Ġand ĠNational ĠNatural ĠScience ĠFoundation Ġof ĠChina Ġ( NS FC ) Ġvia Ġgrant Ġ6 163 2011 Ġand Ġ6 17 72 156 .
Introduction Table-to-text generation is an important and challenging task in natural language processing, which aims to produce the summarization of numerical table BIBREF0, BIBREF1. The related methods can be empirically divided into two categories, pipeline model and end-to-end model. The former consists of content selection, document planning and realisation, mainly for early industrial applications, such as weather forecasting and medical monitoring, etc. The latter generates text directly from the table through a standard neural encoder-decoder framework to avoid error propagation and has achieved remarkable progress. In this paper, we particularly focus on exploring how to improve the performance of neural methods on table-to-text generation. Recently, ROTOWIRE, which provides tables of NBA players' and teams' statistics with a descriptive summary, has drawn increasing attention from academic community. Figure FIGREF1 shows an example of parts of a game's statistics and its corresponding computer generated summary. We can see that the tables has a formal structure including table row header, table column header and table cells. “Al Jefferson” is a table row header that represents a player, “PTS” is a table column header indicating the column contains player's score and “18” is the value of the table cell, that is, Al Jefferson scored 18 points. Several related models have been proposed . They typically encode the table's records separately or as a long sequence and generate a long descriptive summary by a standard Seq2Seq decoder with some modifications. Wiseman explored two types of copy mechanism and found conditional copy model BIBREF3 perform better . Puduppully enhanced content selection ability by explicitly selecting and planning relevant records. Li improved the precision of describing data records in the generated texts by generating a template at first and filling in slots via copy mechanism. Nie utilized results from pre-executed operations to improve the fidelity of generated texts. However, we claim that their encoding of tables as sets of records or a long sequence is not suitable. Because (1) the table consists of multiple players and different types of information as shown in Figure FIGREF1. The earlier encoding approaches only considered the table as sets of records or one dimensional sequence, which would lose the information of other (column) dimension. (2) the table cell consists of time-series data which change over time. That is to say, sometimes historical data can help the model select content. Moreover, when a human writes a basketball report, he will not only focus on the players' outstanding performance in the current match, but also summarize players' performance in recent matches. Lets take Figure FIGREF1 again. Not only do the gold texts mention Al Jefferson's great performance in this match, it also states that “It was the second time in the last three games he's posted a double-double”. Also gold texts summarize John Wall's “double-double” performance in the similar way. Summarizing a player's performance in recent matches requires the modeling of table cell with respect to its historical data (time dimension) which is absent in baseline model. Although baseline model Conditional Copy (CC) tries to summarize it for Gerald Henderson, it clearly produce wrong statements since he didn't get “double-double” in this match. To address the aforementioned problems, we present a hierarchical encoder to simultaneously model row, column and time dimension information. In detail, our model is divided into three layers. The first layer is used to learn the representation of the table cell. Specifically, we employ three self-attention models to obtain three representations of the table cell in its row, column and time dimension. Then, in the second layer, we design a record fusion gate to identify the more important representation from those three dimension and combine them into a dense vector. In the third layer, we use mean pooling method to merge the previously obtained table cell representations in the same row into the representation of the table's row. Then, we use self-attention with content selection gate BIBREF4 to filter unimportant rows' information. To the best of our knowledge, this is the first work on neural table-to-text generation via modeling column and time dimension information so far. We conducted experiments on ROTOWIRE. Results show that our model outperforms existing systems, improving baseline BLEU from 14.19 to 16.85 ($+18.75\%$), P% of relation generation (RG) from 74.80 to 91.46 ($+22.27\%$), F1% of content selection (CS) from 32.49 to 41.21 ($+26.84\%$) and content ordering (CO) from 15.42 to 20.86 ($+35.28\%$) on test set. It also exceeds the state-of-the-art model in terms of those metrics. Preliminaries ::: Notations The input to the model are tables $S=\lbrace s^{1}, s^{2}, s^{3}\rbrace $. $s^{1}$, $s^{2}$, and $s^{3}$ contain records about players' performance in home team, players' performance in visiting team and team's overall performance respectively. We regard each cell in the table as record. Each record $r$ consists of four types of information including value $r.v$ (e.g. 18), entity $r.e$ (e.g. Al Jefferson), type $r.c$ (e.g. POINTS) and a feature $r.f$ (e.g. visiting) which indicate whether a player or a team compete in home court or not. Each player or team takes one row in the table and each column contains a type of record such as points, assists, etc. Also, tables contain the date when the match happened and we let $k$ denote the date of the record. We also create timelines for records. The details of timeline construction is described in Section SECREF4. For simplicity, we omit table id $l$ and record date $k$ in the following sections and let $r_{i,j}$ denotes a record of $i^{th}$ row and $j^{th}$ column in the table. We assume the records come from the same table and $k$ is the date of the mentioned record. Given those information, the model is expected to generate text $y=(y_{1}, ..., y_{t}, ..., y_{T})$ describing these tables. $T$ denotes the length of the text. Preliminaries ::: Record Timeline Constrcution In this paper, we construct timelines $tl=\lbrace tl_{e,c}\rbrace _{e=1,c=1}^{E,C}$ for records. $E$ denotes the number of distinct record entities and $C$ denotes the number of record types. For each timeline $tl_{e,c}$, we first extract records with the same entity $e$ and type $c$ from dataset. Then we sort them into a sequence according to the record's date from old to new. This sequence is considered as timeline $tl_{e,c}$. For example, in Figure FIGREF3, the “Timeline” part in the lower-left corner represents a timeline for entity Al Jefferson and type PTS (points). Preliminaries ::: Baseline Model We use Seq2Seq model with attention BIBREF5 and conditional copy BIBREF3 as the base model. During training, given tables $S$ and their corresponding reference texts $y$, the model maximized the conditional probability $P(y|S) = \prod _{t=1}^{T}P(y_{t}|y_{<t},S)$ . $t$ is the timestep of decoder. First, for each record of the $i^{th}$ row and $j^{th}$ column in the table, we utilize 1-layer MLP to encode the embeddings of each record's four types of information into a dense vector ${r}_{i, j}$, $r_{i, j}=ReLU(W_{a}[r_{i, j}.e;r_{i, j}.c;r_{i, j}.v;r_{i, j}.f]+b_{a})$. $W_{a}$ and $b_{a}$ are trainable parameters. The word embeddings for each type of information are trainable and randomly initialized before training following Wiseman. $[;]$ denotes the vector concatenation. Then, we use a LSTM decoder with attention and conditional copy to model the conditional probability $P(y_{t}|y_{<t},S)$. The base model first use attention mechanism BIBREF5 to find relevant records from the input tables and represent them as context vector. Please note that the base model doesn't utilize the structure of three tables and normalize the attention weight $\alpha _{t, i^{\prime }, j^{\prime }}$ across every records in every tables. Then it combines the context vector with decoder's hidden state $d_{t}$ and form a new attentional hidden state $\tilde{d}_{t}$ which is used to generate words from vocabulary $P_{gen}(y_{t}|y_{<t},S)=softmax(W_{d}\tilde{d}_{t}+b_{d})$ Also the conditional copy mechanism is adopted in base model. It introduces a variable $z_{t}$ to decide whether to copy from tables or generate from vocabulary. The probability to copy from table is $P(z_{t}=1|y_{<t}, S)=sigmoid(w_{e} \cdot d_{t}+b_{e})$. Then it decomposes the conditional probability of generating $t^{th}$ word $P(y_{t}|y_{<t},S)$, given the tables $S$ and previously generated words $y_{<t}$, as follows. Approach In this section, we propose an effective hierarchical encoder to utilize three dimensional structure of input data in order to improve table representation. Those three dimensions include row, column and time. As shown in Figure FIGREF3, during encoding, our model consists of three layers including record encoders, record fusion gate and row-level encoder. Given tables $S$ as described in Section SECREF2, we first encode each record in each dimension respectively. Then we use the record fusion gate to combine them into a dense representation. Afterwards, we obtain row-level representation via mean pooling and self-attention with content selection gate. In decoding phase, the decoder can first find important row then attend to important record when generating texts. We describe model's details in following parts. Approach ::: Layer 1: Record Encoders ::: Row Dimension Encoder Based on our observation, when someone's points is mentioned in texts, some related records such as “field goals made” (FGM) and “field goals attempted” (FGA) will also be included in texts. Taken gold texts in Figure FIGREF1 as example, when Al Jefferson's point 18 is mentioned, his FGM 9 and FGA 19 are also mentioned. Thus, when modeling a record, other records in the same row can be useful. Since the record in the row is not sequential, we use a self-attention network which is similar to Liu2018LearningST to model records in the context of other records in the same row. Let $r_{i, j}^{row}$ be the row dimension representation of the record of $i^{th}$ row and $j^{th}$ column. Then, we obtain the context vector in row dimension $c_{i, j}^{row}$ by attending to other records in the same row as follows. Please note that $\alpha _{i, j, j^{\prime }}^{row} \propto exp(r_{i, j}^{T}W_{o}{r}_{i, j^{\prime }})$ is normalized across records in the same row $i$. $W_{o}$ is a trainable parameter. Then, we combine record's representation with $c_{i, j}$ and obtain the row dimension record representation $r_{i, j}^{row}=tanh(W_{f}[r_{i, j}; c_{i, j}^{row}])$. $W_{f}$ is a trainable parameter. Approach ::: Layer 1: Record Encoders ::: Column Dimension Encoder Each input table consists of multiple rows and columns. Each column in the table covers one type of information such as points. Only few of the row may have high points or other type of information and thus become the important one. For example, in “Column Dimension” part of Figure FIGREF3, “Al Jefferson” is more important than “Gary Neal” because the former one have more impressive points. Therefore, when encoding a record, it is helpful to compare it with other records in the same column in order to understand the performance level reflected by the record among his teammates (rows). We employ self-attention similar to the one used in Section SECREF8 in column dimension to compare between records. We let ${r_{i, j}^{col}}$ be the column representation of the record of $i^{th}$ row and $j^{th}$ column. We obtain context vector in column dimension $c_{i, j}^{col}$ as follows. Please note that $\alpha _{j, i, i^{\prime }}$ is normalized across records from different rows $i^{\prime }$ but of the same column $j$. The column dimension representation $r_{i, j}^{col}$ is obtained similar to row dimension. Approach ::: Layer 1: Record Encoders ::: Time Dimension Encoder As mentioned in Section SECREF1, we find some expressions in texts require information about players' historical (time dimension) performance. So the history information of record $r_{i,j}$ is important. Note that we have already constructed timeline for each record entity and type as described in Section SECREF4. Given those timelines, We collect records with same entity and type in the timeline which has date before date $k$ of the record $r_{i,j}$ as history information. Since for some record, the history information can be too long, we set a history window. Thus, we keep most recent history information sequence within history window and denote them as $hist(r_{i, j})$. We model this kind of information in time dimension via self-attention. However, unlike the unordered nature of rows and columns, the history information is sequential. Therefore, we introduce a trainable position embedding $emb_{pos}(k^{\prime })$ and add it to the record's representation and obtain a new record representation ${rp}_{k^{\prime }}$. It denotes the representation of a record with the same entity and type of $r_{i,j}$ but of the date $k^{\prime }$ before $k$ in the corresponding history window. We use $r_{i, j}^{time}$ to denote the history representation of the record of $i^{th}$ row and $j^{th}$ column. Then the history dimension context vector is obtained by attending to history records in the window. Please note that we use 1-layer MLP as score function here and $\alpha _{k, k^{\prime }}^{time}$ is normalized within the history window. We obtain the time dimension representation $r_{i, j}^{time}$ similar to row dimension. Approach ::: Layer 2: Record Fusion Gate After obtaining record representations in three dimension, it is important to figure out which representation plays a more important role in reflecting the record's information. If a record stands out from other row's records of same column, the column dimension representation may have a higher weight in forming the overall record representation. If a record differs from previous match significantly, the history dimension representation may have a higher weight. Also, some types of information may appear in texts more frequently together which can be reflected by row dimension representation. Therefore, we propose a record fusion gate to adaptively combine all three dimension representations. First, we concatenate $r_{i, j}^{row}$, $r_{i, j}^{col}$ and $r_{i, j}^{time}$, then adopt a 1-layer MLP to obtain a general representation $r_{i, j}^{gen}$ which we consider as a baseline representation of records' information. Then, we compare each dimension representation with the baseline and obtain its weight in the final record representation. We use 1-layer MLP as the score function. Equation DISPLAY_FORM15 shows an example of calculating column dimension representation's weight in the final record representation. The weight of row and time dimension representation is obtained similar to the weight of column dimension representation. In the end, the fused record representation $\tilde{r}_{i, j}$ is the weighted sum of the three dimension representations. Approach ::: Layer 3: Row-level Encoder For each row, we combine its records via mean pooling (Equation DISPLAY_FORM18) in order to obtain a general representation of the row which may reflect the row (player or team)'s overall performance. $C$ denotes the number of columns. Then, we adopt content selection gate $g_{i}$, which is proposed by Puduppully on rows' representations $row_{i}$, and obtain a new representation $\tilde{row}_{i}=g_{i}\odot row_{i}$ to choose more important information based on each row's context. Approach ::: Decoder with Dual Attention Since record encoders with record fusion gate provide record-level representation and row-level encoder provides row-level representation. Inspired by N18-2097, we can modify the decoder in base model to first choose important row then attend to records when generating each word. Following notations in Section SECREF5, $\beta _{t, i} \propto exp(score(d_{t}, row_{i}))$ obtains the attention weight with respect to each row. Please note that $\beta _{t, i}$ is normalized across all row-level representations from all three tables. Then, $\gamma _{t, i, j} \propto exp(score(d_{t}, \tilde{r}_{i, j}))$ obtains attention weight for records. Please note that we normalize $\gamma _{t, i, j}$ among records in the same row. We use the row-level attention $\beta _{t, i}$ as guidance for choosing row based on row's general representation. Then we use it to re-weight the record-level attention $\gamma _{t, i, j}$ and change the attention weight in base model to $\tilde{\alpha }_{t, i, j}$. Please note that ${\tilde{\alpha }_{t, i, j}}$ sum to 1 across all records in all tables. Approach ::: Training Given a batch of input tables $\lbrace S\rbrace _{G}$ and reference output $\lbrace Y\rbrace _{G}$, we use negative log-likelihood as the loss function for our model. We train the model by minimizing $L$. $G$ is the number of examples in the batch and $T_{g}$ represents the length of $g^{th}$ reference's length. Experiments ::: Dataset and Evaluation Metrics We conducted experiments on ROTOWIRE BIBREF2. For each example, it provides three tables as described in Section SECREF2 which consists of 628 records in total with a long game summary. The average length of game summary is 337.1. In this paper, we followed the data split introduced in Wiseman: 3398 examples in training set, 727 examples in development set and 728 examples in test set. We followed Wiseman's work and use BLEU BIBREF7 and three extractive evaluation metrics RG, CS and CO BIBREF2 for evaluation. The main idea of the extractive evaluation metrics is to use an Information Extraction (IE) model to identify records mentioned in texts. Then compare them with tables or records extracted from reference to evaluate the model. RG (Relation Generation) measures content fidelity of texts. CS (Content Selection) measures model's ability on content selection. CO (Content Ordering) measures model's ability on ordering the chosen records in texts. We refer the readers to Wiseman's paper for more details. Experiments ::: Implementation Details Following configurations in Puduppully, we set word embedding and LSTM decoder hidden size as 600. The decoder's layer was set to be 2. Input feeding BIBREF5 was also used for decoder. We applied dropout at a rate 0.3. For training, we used Adagrad BIBREF8 optimizer with learning rate of 0.15, truncated BPTT (block length 100), batch size of 5 and learning rate decay of 0.97. For inferring, we set beam size as 5. We also set the history windows size as 3 from {3,5,7} based on the results. Code of our model can be found at https://github.com/ernestgong/data2text-three-dimensions/. Experiments ::: Results ::: Automatic Evaluation Table TABREF23 displays the automatic evaluation results on both development and test set. We chose Conditional Copy (CC) model as our baseline, which is the best model in Wiseman. We included reported scores with updated IE model by Puduppully and our implementation's result on CC in this paper. Also, we compared our models with other existing works on this dataset including OpATT BIBREF6 and Neural Content Planning with conditional copy (NCP+CC) BIBREF4. In addition, we implemented three other hierarchical encoders that encoded tables' row dimension information in both record-level and row-level to compare with the hierarchical structure of encoder in our model. The decoder was equipped with dual attention BIBREF9. The one with LSTM cell is similar to the one in N18-2097 with 1 layer from {1,2,3}. The one with CNN cell BIBREF10 has kernel width 3 from {3, 5} and 10 layer from {5,10,15,20}. The one with transformer-style encoder (MHSA) BIBREF11 has 8 head from {8, 10} and 5 layer from {2,3,4,5,6}. The heads and layers mentioned above were for both record-level encoder and row-level encoder respectively. The self-attention (SA) cell we used, as described in Section SECREF3, achieved better overall performance in terms of F1% of CS, CO and BLEU among the hierarchical encoders. Also we implemented a template system same as the one used in Wiseman which outputted eight sentences: an introductory sentence (two teams' points and who win), six top players' statistics (ranked by their points) and a conclusion sentence. We refer the readers to Wiseman's paper for more detailed information on templates. The gold reference's result is also included in Table TABREF23. Overall, our model performs better than other neural models on both development and test set in terms of RG's P%, F1% score of CS, CO and BLEU, indicating our model's clear improvement on generating high-fidelity, informative and fluent texts. Also, our model with three dimension representations outperforms hierarchical encoders with only row dimension representation on development set. This indicates that cell and time dimension representation are important in representing the tables. Compared to reported baseline result in Wiseman, we achieved improvement of $22.27\%$ in terms of RG, $26.84\%$ in terms of CS F1%, $35.28\%$ in terms of CO and $18.75\%$ in terms of BLEU on test set. Unsurprisingly, template system achieves best on RG P% and CS R% due to the included domain knowledge. Also, the high RG # and low CS P% indicates that template will include vast information while many of them are deemed redundant. In addition, the low CO and low BLEU indicates that the rigid structure of the template will produce texts that aren't as adaptive to the given tables and natural as those produced by neural models. Also, we conducted ablation study on our model to evaluate each component's contribution on development set. Based on the results, the absence of row-level encoder hurts our model's performance across all metrics especially the content selection ability. Row, column and time dimension information are important to the modeling of tables because subtracting any of them will result in performance drop. Also, position embedding is critical when modeling time dimension information according to the results. In addition, record fusion gate plays an important role because BLEU, CO, RG P% and CS P% drop significantly after subtracting it from full model. Results show that each component in the model contributes to the overall performance. In addition, we compare our model with delayed copy model (DEL) BIBREF12 along with gold text, template system (TEM), conditional copy (CC) BIBREF2 and NCP+CC (NCP) BIBREF4. Li's model generate a template at first and then fill in the slots with delayed copy mechanism. Since its result in Li's paper was evaluated by IE model trained by Wiseman and “relexicalization” by Li, we adopted the corresponding IE model and re-implement “relexicalization” as suggested by Li for fair comparison. Please note that CC's evaluation results via our re-implemented “relexicalization” is comparable to the reported result in Li. We applied them on models other than DEL as shown in Table TABREF28 and report DEL's result from BIBREF12's paper. It shows that our model outperform Li's model significantly across all automatic evaluation metrics in Table TABREF28. Experiments ::: Results ::: Human Evaluation In this section, we hired three graduates who passed intermediate English test (College English Test Band 6) and were familiar with NBA games to perform human evaluation. First, in order to check if history information is important, we sampled 100 summaries from training set and asked raters to manually check whether the summary contained expressions that need to be inferred from history information. It turns out that $56.7\%$ summaries of the sampled summaries need history information. Following human evaluation settings in Puduppully, we conducted the following human evaluation experiments at the same scale. The second experiment is to assess whether the improvement on relation generation metric reported in automatic evaluation is supported by human evaluation. We compared our full model with gold texts, template-based system, CC BIBREF2 and NCP+CC (NCP) BIBREF4. We randomly sampled 30 examples from test set. Then, we randomly sampled 4 sentences from each model's output for each example. We provided the raters of those sampled sentences with the corresponding NBA game statistics. They were asked to count the number of supporting and contradicting facts in each sentence. Each sentence is rated independently. We report the average number of supporting facts (#Sup) and contradicting facts (#Cont) in Table TABREF30. Unsurprisingly, template-based system includes most supporting facts and least contradicting facts in its texts because the template consists of a large number of facts and all of those facts are extracted from the table. Also, our model produces less contradicting facts than other two neural models. Although our model produces less supporting facts than NCP and CC, it still includes enough supporting facts (slightly more than gold texts). Also, comparing to NCP+CC (NCP)’s tendency to include vast information that contain redundant information, our model’s ability to select and accurately convey information is better. All other results (Gold, CC, NCP and ours) are significantly different from template-based system's results in terms of number of supporting facts according to one-way ANOVA with posthoc Tukey HSD tests. All significance difference reported in this paper are less than 0.05. Our model is also significantly different from the NCP model. As for average number of contradicting facts, our model is significantly different from other two neural models. Surprisingly, gold texts were found containing contradicting facts. We checked the raters's result and found that gold texts occasionally include wrong field-goal or three-point percent or wrong points difference between the winner and the defeated team. We can treat the average contradicting facts number of gold texts as a lower bound. In the third experiment, following Puduppully, we asked raters to evaluate those models in terms of grammaticality (is it more fluent and grammatical?), coherence (is it easier to read or follows more natural ordering of facts? ) and conciseness (does it avoid redundant information and repetitions?). We adopted the same 30 examples from above and arranged every 5-tuple of summaries into 10 pairs. Then, we asked the raters to choose which system performs the best given each pair. Scores are computed as the difference between percentage of times when the model is chosen as the best and percentage of times when the model is chosen as the worst. Gold texts is significantly more grammatical than others across all three metrics. Also, our model performs significantly better than other two neural models (CC, NCP) in all three metrics. Template-based system generates significantly more grammatical and concise but significantly less coherent results, compared to all three neural models. Because the rigid structure of texts ensures the correct grammaticality and no repetition in template-based system's output. However, since the templates are stilted and lack variability compared to others, it was deemed less coherent than the others by the raters. Experiments ::: Results ::: Qualitative Example Figure FIGREF32 shows an example generated by our model. It evidently has several nice properties: it can accurately select important player “Al Jefferson” from the tables who is neglected by baseline model, which need the model to understand performance difference of a type of data (column) between each rows (players). Also it correctly summarize performance of “Al Jefferson” in this match as “double-double” which requires ability to capture dependency from different columns (different type of record) in the same row (player). In addition, it models “Al Jefferson” history performance and correctly states that “It was his second double-double over his last three games”, which is also mentioned in gold texts included in Figure 1 in a similar way. Related Work In recent years, neural data-to-text systems make remarkable progress on generating texts directly from data. Mei2016WhatTT proposes an encoder-aligner-decoder model to generate weather forecast, while N18-2098 propose a mixed hierarchical attention. AAAI1816203 proposes a hybrid content- and linkage-based attention mechanism to model the order of content. AAAI1816599 propose to integrate field information into table representation and enhance decoder with dual attention. AAAI1816138 develops a table-aware encoder-decoder model. Wiseman introduced a document-scale data-to-text dataset, consisting of long text with more redundant records, which requires the model to select important information to generate. We describe recent works in Section SECREF1. Also, some studies in abstractive text summarization encode long texts in a hierarchical manner. N18-2097 uses a hierarchical encoder to encode input, paired with a discourse-aware decoder. W17-4505 encode document hierarchically and propose coarse-to-fine attention for decoder. Recently, liu2019hierarchical propose a hierarchical encoder for data-to-text generation which uses LSTM as its cell. murakami-etal-2017-learning propose to model stock market time-series data and generate comments. As for incorporating historical background in generation, robin1994revision proposed to build a draft with essential new facts at first, then incorporate background facts when revising the draft based on functional unification grammars. Different from that, we encode the historical (time dimension) information in the neural data-to-text model in an end-to-end fashion. Existing works on data-to-text generation neglect the joint representation of tables' row, column and time dimension information. In this paper, we propose an effective hierarchical encoder which models information from row, column and time dimension simultaneously. Conclusion In this work, we present an effective hierarchical encoder for table-to-text generation that learns table representations from row, column and time dimension. In detail, our model consists of three layers, which learn records' representation in three dimension, combine those representations via their sailency and obtain row-level representation based on records' representation. Then, during decoding, it will select important table row before attending to records. Experiments are conducted on ROTOWIRE, a benchmark dataset of NBA games. Both automatic and human evaluation results show that our model achieves the new state-of-the-art performance. Acknowledgements We would like to thank the anonymous reviewers for their helpful comments. We'd also like to thank Xinwei Geng, Yibo Sun, Zhengpeng Xiang and Yuyu Chen for their valuable input. This work was supported by the National Key R&D Program of China via grant 2018YFB1005103 and National Natural Science Foundation of China (NSFC) via grant 61632011 and 61772156.
What is the state-of-the-art model for the task?
5a22293b055f5775081d6acdc0450f7bd5f5de04
What Ġis Ġthe Ġstate - of - the - art Ġmodel Ġfor Ġthe Ġtask ?
Op ATT ĠB IB REF 6 , ĠNeural ĠContent ĠPlanning Ġwith Ġconditional Ġcopy Ġ( NC P + CC ) ĠB IB REF 4
<s> What Ġis Ġthe Ġstate - of - the - art Ġmodel Ġfor Ġthe Ġtask ? </s> Introduction </s> Table - to - text Ġgeneration Ġis Ġan Ġimportant Ġand Ġchallenging Ġtask Ġin Ġnatural Ġlanguage Ġprocessing , Ġwhich Ġaims Ġto Ġproduce Ġthe Ġsummar ization Ġof Ġnumerical Ġtable ĠB IB REF 0 , ĠB IB REF 1 . ĠThe Ġrelated Ġmethods Ġcan Ġbe Ġempir ically Ġdivided Ġinto Ġtwo Ġcategories , Ġpipeline Ġmodel Ġand Ġend - to - end Ġmodel . ĠThe Ġformer Ġconsists Ġof Ġcontent Ġselection , Ġdocument Ġplanning Ġand Ġreal isation , Ġmainly Ġfor Ġearly Ġindustrial Ġapplications , Ġsuch Ġas Ġweather Ġforecasting Ġand Ġmedical Ġmonitoring , Ġetc . ĠThe Ġlatter Ġgenerates Ġtext Ġdirectly Ġfrom Ġthe Ġtable Ġthrough Ġa Ġstandard Ġneural Ġenc oder - dec oder Ġframework Ġto Ġavoid Ġerror Ġpropagation Ġand Ġhas Ġachieved Ġremarkable Ġprogress . ĠIn Ġthis Ġpaper , Ġwe Ġparticularly Ġfocus Ġon Ġexploring Ġhow Ġto Ġimprove Ġthe Ġperformance Ġof Ġneural Ġmethods Ġon Ġtable - to - text Ġgeneration . </s> Recently , ĠR OT OW IRE , Ġwhich Ġprovides Ġtables Ġof ĠNBA Ġplayers ' Ġand Ġteams ' Ġstatistics Ġwith Ġa Ġdescriptive Ġsummary , Ġhas Ġdrawn Ġincreasing Ġattention Ġfrom Ġacademic Ġcommunity . ĠFigure ĠFIG REF 1 Ġshows Ġan Ġexample Ġof Ġparts Ġof Ġa Ġgame 's Ġstatistics Ġand Ġits Ġcorresponding Ġcomputer Ġgenerated Ġsummary . ĠWe Ġcan Ġsee Ġthat Ġthe Ġtables Ġhas Ġa Ġformal Ġstructure Ġincluding Ġtable Ġrow Ġheader , Ġtable Ġcolumn Ġheader Ġand Ġtable Ġcells . ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġis Ġa Ġtable Ġrow Ġheader Ġthat Ġrepresents Ġa Ġplayer , ĠâĢ ľ P TS âĢ Ŀ Ġis Ġa Ġtable Ġcolumn Ġheader Ġindicating Ġthe Ġcolumn Ġcontains Ġplayer 's Ġscore Ġand ĠâĢ ľ 18 âĢ Ŀ Ġis Ġthe Ġvalue Ġof Ġthe Ġtable Ġcell , Ġthat Ġis , ĠAl ĠJefferson Ġscored Ġ18 Ġpoints . ĠSeveral Ġrelated Ġmodels Ġhave Ġbeen Ġproposed Ġ. ĠThey Ġtypically Ġencode Ġthe Ġtable 's Ġrecords Ġseparately Ġor Ġas Ġa Ġlong Ġsequence Ġand Ġgenerate Ġa Ġlong Ġdescriptive Ġsummary Ġby Ġa Ġstandard ĠSe q 2 Se q Ġdec oder Ġwith Ġsome Ġmodifications . ĠWis eman Ġexplored Ġtwo Ġtypes Ġof Ġcopy Ġmechanism Ġand Ġfound Ġconditional Ġcopy Ġmodel ĠB IB REF 3 Ġperform Ġbetter Ġ. ĠP ud upp ully Ġenhanced Ġcontent Ġselection Ġability Ġby Ġexplicitly Ġselecting Ġand Ġplanning Ġrelevant Ġrecords . ĠLi Ġimproved Ġthe Ġprecision Ġof Ġdescribing Ġdata Ġrecords Ġin Ġthe Ġgenerated Ġtexts Ġby Ġgenerating Ġa Ġtemplate Ġat Ġfirst Ġand Ġfilling Ġin Ġslots Ġvia Ġcopy Ġmechanism . ĠN ie Ġutilized Ġresults Ġfrom Ġpre - exec uted Ġoperations Ġto Ġimprove Ġthe Ġfidelity Ġof Ġgenerated Ġtexts . ĠHowever , Ġwe Ġclaim Ġthat Ġtheir Ġencoding Ġof Ġtables Ġas Ġsets Ġof Ġrecords Ġor Ġa Ġlong Ġsequence Ġis Ġnot Ġsuitable . ĠBecause Ġ( 1 ) Ġthe Ġtable Ġconsists Ġof Ġmultiple Ġplayers Ġand Ġdifferent Ġtypes Ġof Ġinformation Ġas Ġshown Ġin ĠFigure ĠFIG REF 1 . ĠThe Ġearlier Ġencoding Ġapproaches Ġonly Ġconsidered Ġthe Ġtable Ġas Ġsets Ġof Ġrecords Ġor Ġone Ġdimensional Ġsequence , Ġwhich Ġwould Ġlose Ġthe Ġinformation Ġof Ġother Ġ( column ) Ġdimension . Ġ( 2 ) Ġthe Ġtable Ġcell Ġconsists Ġof Ġtime - series Ġdata Ġwhich Ġchange Ġover Ġtime . ĠThat Ġis Ġto Ġsay , Ġsometimes Ġhistorical Ġdata Ġcan Ġhelp Ġthe Ġmodel Ġselect Ġcontent . ĠMoreover , Ġwhen Ġa Ġhuman Ġwrites Ġa Ġbasketball Ġreport , Ġhe Ġwill Ġnot Ġonly Ġfocus Ġon Ġthe Ġplayers ' Ġoutstanding Ġperformance Ġin Ġthe Ġcurrent Ġmatch , Ġbut Ġalso Ġsummarize Ġplayers ' Ġperformance Ġin Ġrecent Ġmatches . ĠLets Ġtake ĠFigure ĠFIG REF 1 Ġagain . ĠNot Ġonly Ġdo Ġthe Ġgold Ġtexts Ġmention ĠAl ĠJefferson 's Ġgreat Ġperformance Ġin Ġthis Ġmatch , Ġit Ġalso Ġstates Ġthat ĠâĢ ľ It Ġwas Ġthe Ġsecond Ġtime Ġin Ġthe Ġlast Ġthree Ġgames Ġhe 's Ġposted Ġa Ġdouble - double âĢ Ŀ . ĠAlso Ġgold Ġtexts Ġsummarize ĠJohn ĠWall 's ĠâĢ ľ double - double âĢ Ŀ Ġperformance Ġin Ġthe Ġsimilar Ġway . ĠSum mar izing Ġa Ġplayer 's Ġperformance Ġin Ġrecent Ġmatches Ġrequires Ġthe Ġmodeling Ġof Ġtable Ġcell Ġwith Ġrespect Ġto Ġits Ġhistorical Ġdata Ġ( time Ġdimension ) Ġwhich Ġis Ġabsent Ġin Ġbaseline Ġmodel . ĠAlthough Ġbaseline Ġmodel ĠCond itional ĠCopy Ġ( CC ) Ġtries Ġto Ġsummarize Ġit Ġfor ĠGerald ĠHenderson , Ġit Ġclearly Ġproduce Ġwrong Ġstatements Ġsince Ġhe Ġdidn 't Ġget ĠâĢ ľ double - double âĢ Ŀ Ġin Ġthis Ġmatch . </s> To Ġaddress Ġthe Ġaforementioned Ġproblems , Ġwe Ġpresent Ġa Ġhierarchical Ġenc oder Ġto Ġsimultaneously Ġmodel Ġrow , Ġcolumn Ġand Ġtime Ġdimension Ġinformation . ĠIn Ġdetail , Ġour Ġmodel Ġis Ġdivided Ġinto Ġthree Ġlayers . ĠThe Ġfirst Ġlayer Ġis Ġused Ġto Ġlearn Ġthe Ġrepresentation Ġof Ġthe Ġtable Ġcell . ĠSpecifically , Ġwe Ġemploy Ġthree Ġself - att ention Ġmodels Ġto Ġobtain Ġthree Ġrepresentations Ġof Ġthe Ġtable Ġcell Ġin Ġits Ġrow , Ġcolumn Ġand Ġtime Ġdimension . ĠThen , Ġin Ġthe Ġsecond Ġlayer , Ġwe Ġdesign Ġa Ġrecord Ġfusion Ġgate Ġto Ġidentify Ġthe Ġmore Ġimportant Ġrepresentation Ġfrom Ġthose Ġthree Ġdimension Ġand Ġcombine Ġthem Ġinto Ġa Ġdense Ġvector . ĠIn Ġthe Ġthird Ġlayer , Ġwe Ġuse Ġmean Ġpool ing Ġmethod Ġto Ġmerge Ġthe Ġpreviously Ġobtained Ġtable Ġcell Ġrepresentations Ġin Ġthe Ġsame Ġrow Ġinto Ġthe Ġrepresentation Ġof Ġthe Ġtable 's Ġrow . ĠThen , Ġwe Ġuse Ġself - att ention Ġwith Ġcontent Ġselection Ġgate ĠB IB REF 4 Ġto Ġfilter Ġun important Ġrows ' Ġinformation . ĠTo Ġthe Ġbest Ġof Ġour Ġknowledge , Ġthis Ġis Ġthe Ġfirst Ġwork Ġon Ġneural Ġtable - to - text Ġgeneration Ġvia Ġmodeling Ġcolumn Ġand Ġtime Ġdimension Ġinformation Ġso Ġfar . ĠWe Ġconducted Ġexperiments Ġon ĠR OT OW IRE . ĠResults Ġshow Ġthat Ġour Ġmodel Ġoutper forms Ġexisting Ġsystems , Ġimproving Ġbaseline ĠB LE U Ġfrom Ġ14 . 19 Ġto Ġ16 . 85 Ġ($ + 18 . 75 \ % $ ), ĠP % Ġof Ġrelation Ġgeneration Ġ( RG ) Ġfrom Ġ74 . 80 Ġto Ġ91 . 46 Ġ($ + 22 . 27 \ % $ ), ĠF 1 % Ġof Ġcontent Ġselection Ġ( CS ) Ġfrom Ġ32 . 49 Ġto Ġ41 . 21 Ġ($ + 26 . 84 \ % $ ) Ġand Ġcontent Ġordering Ġ( CO ) Ġfrom Ġ15 . 42 Ġto Ġ20 . 86 Ġ($ + 35 . 28 \ % $ ) Ġon Ġtest Ġset . ĠIt Ġalso Ġexceeds Ġthe Ġstate - of - the - art Ġmodel Ġin Ġterms Ġof Ġthose Ġmetrics . </s> P rel im in aries Ġ: :: ĠNot ations </s> The Ġinput Ġto Ġthe Ġmodel Ġare Ġtables Ġ$ S = \ l brace Ġs ^{ 1 }, Ġs ^{ 2 }, Ġs ^{ 3 }\ r brace Ġ$ . Ġ$ s ^{ 1 } $, Ġ$ s ^{ 2 } $, Ġand Ġ$ s ^{ 3 } $ Ġcontain Ġrecords Ġabout Ġplayers ' Ġperformance Ġin Ġhome Ġteam , Ġplayers ' Ġperformance Ġin Ġvisiting Ġteam Ġand Ġteam 's Ġoverall Ġperformance Ġrespectively . ĠWe Ġregard Ġeach Ġcell Ġin Ġthe Ġtable Ġas Ġrecord . ĠEach Ġrecord Ġ$ r $ Ġconsists Ġof Ġfour Ġtypes Ġof Ġinformation Ġincluding Ġvalue Ġ$ r . v $ Ġ( e . g . Ġ18 ), Ġentity Ġ$ r . e $ Ġ( e . g . ĠAl ĠJefferson ), Ġtype Ġ$ r . c $ Ġ( e . g . ĠPO IN TS ) Ġand Ġa Ġfeature Ġ$ r . f $ Ġ( e . g . Ġvisiting ) Ġwhich Ġindicate Ġwhether Ġa Ġplayer Ġor Ġa Ġteam Ġcompete Ġin Ġhome Ġcourt Ġor Ġnot . ĠEach Ġplayer Ġor Ġteam Ġtakes Ġone Ġrow Ġin Ġthe Ġtable Ġand Ġeach Ġcolumn Ġcontains Ġa Ġtype Ġof Ġrecord Ġsuch Ġas Ġpoints , Ġassists , Ġetc . ĠAlso , Ġtables Ġcontain Ġthe Ġdate Ġwhen Ġthe Ġmatch Ġhappened Ġand Ġwe Ġlet Ġ$ k $ Ġdenote Ġthe Ġdate Ġof Ġthe Ġrecord . ĠWe Ġalso Ġcreate Ġtimelines Ġfor Ġrecords . ĠThe Ġdetails Ġof Ġtimeline Ġconstruction Ġis Ġdescribed Ġin ĠSection ĠSEC REF 4 . ĠFor Ġsimplicity , Ġwe Ġomit Ġtable Ġid Ġ$ l $ Ġand Ġrecord Ġdate Ġ$ k $ Ġin Ġthe Ġfollowing Ġsections Ġand Ġlet Ġ$ r _{ i , j } $ Ġdenotes Ġa Ġrecord Ġof Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn Ġin Ġthe Ġtable . ĠWe Ġassume Ġthe Ġrecords Ġcome Ġfrom Ġthe Ġsame Ġtable Ġand Ġ$ k $ Ġis Ġthe Ġdate Ġof Ġthe Ġmentioned Ġrecord . ĠGiven Ġthose Ġinformation , Ġthe Ġmodel Ġis Ġexpected Ġto Ġgenerate Ġtext Ġ$ y =( y _{ 1 }, Ġ... , Ġy _{ t }, Ġ... , Ġy _{ T }) $ Ġdescribing Ġthese Ġtables . Ġ$ T $ Ġdenotes Ġthe Ġlength Ġof Ġthe Ġtext . </s> P rel im in aries Ġ: :: ĠRecord ĠTimeline ĠCon str c ution </s> In Ġthis Ġpaper , Ġwe Ġconstruct Ġtimelines Ġ$ tl = \ l brace Ġt l _{ e , c }\ r brace Ġ_ { e = 1 , c = 1 } ^{ E , C } $ Ġfor Ġrecords . Ġ$ E $ Ġdenotes Ġthe Ġnumber Ġof Ġdistinct Ġrecord Ġentities Ġand Ġ$ C $ Ġdenotes Ġthe Ġnumber Ġof Ġrecord Ġtypes . ĠFor Ġeach Ġtimeline Ġ$ tl _{ e , c } $, Ġwe Ġfirst Ġextract Ġrecords Ġwith Ġthe Ġsame Ġentity Ġ$ e $ Ġand Ġtype Ġ$ c $ Ġfrom Ġdataset . ĠThen Ġwe Ġsort Ġthem Ġinto Ġa Ġsequence Ġaccording Ġto Ġthe Ġrecord 's Ġdate Ġfrom Ġold Ġto Ġnew . ĠThis Ġsequence Ġis Ġconsidered Ġas Ġtimeline Ġ$ tl _{ e , c } $. ĠFor Ġexample , Ġin ĠFigure ĠFIG REF 3 , Ġthe ĠâĢ ľ Tim eline âĢ Ŀ Ġpart Ġin Ġthe Ġlower - left Ġcorner Ġrepresents Ġa Ġtimeline Ġfor Ġentity ĠAl ĠJefferson Ġand Ġtype ĠPTS Ġ( points ). </s> P rel im in aries Ġ: :: ĠBas eline ĠModel </s> We Ġuse ĠSe q 2 Se q Ġmodel Ġwith Ġattention ĠB IB REF 5 Ġand Ġconditional Ġcopy ĠB IB REF 3 Ġas Ġthe Ġbase Ġmodel . ĠDuring Ġtraining , Ġgiven Ġtables Ġ$ S $ Ġand Ġtheir Ġcorresponding Ġreference Ġtexts Ġ$ y $, Ġthe Ġmodel Ġmaxim ized Ġthe Ġconditional Ġprobability Ġ$ P ( y | S ) Ġ= Ġ\ pro d Ġ_ { t = 1 } ^{ T } P ( y _{ t } | y _{ < t }, S ) $ Ġ. Ġ$ t $ Ġis Ġthe Ġtim est ep Ġof Ġdec oder . ĠFirst , Ġfor Ġeach Ġrecord Ġof Ġthe Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn Ġin Ġthe Ġtable , Ġwe Ġutilize Ġ1 - layer ĠML P Ġto Ġencode Ġthe Ġembed d ings Ġof Ġeach Ġrecord 's Ġfour Ġtypes Ġof Ġinformation Ġinto Ġa Ġdense Ġvector Ġ${ r } _{ i , Ġj } $, Ġ$ r _{ i , Ġj } = Re LU ( W _{ a } [ r _{ i , Ġj }. e ; r _{ i , Ġj }. c ; r _{ i , Ġj }. v ; r _{ i , Ġj }. f ]+ b _{ a }) $. Ġ$ W _{ a } $ Ġand Ġ$ b _{ a } $ Ġare Ġtrain able Ġparameters . ĠThe Ġword Ġembed d ings Ġfor Ġeach Ġtype Ġof Ġinformation Ġare Ġtrain able Ġand Ġrandomly Ġinitialized Ġbefore Ġtraining Ġfollowing ĠWis eman . Ġ$ [ ; ] $ Ġdenotes Ġthe Ġvector Ġconc aten ation . ĠThen , Ġwe Ġuse Ġa ĠL ST M Ġdec oder Ġwith Ġattention Ġand Ġconditional Ġcopy Ġto Ġmodel Ġthe Ġconditional Ġprobability Ġ$ P ( y _{ t } | y _{ < t }, S ) $. ĠThe Ġbase Ġmodel Ġfirst Ġuse Ġattention Ġmechanism ĠB IB REF 5 Ġto Ġfind Ġrelevant Ġrecords Ġfrom Ġthe Ġinput Ġtables Ġand Ġrepresent Ġthem Ġas Ġcontext Ġvector . ĠPlease Ġnote Ġthat Ġthe Ġbase Ġmodel Ġdoesn 't Ġutilize Ġthe Ġstructure Ġof Ġthree Ġtables Ġand Ġnormal ize Ġthe Ġattention Ġweight Ġ$\ alpha Ġ_ { t , Ġi ^ {\ prime Ġ}, Ġj ^ {\ prime Ġ}} $ Ġacross Ġevery Ġrecords Ġin Ġevery Ġtables . ĠThen Ġit Ġcombines Ġthe Ġcontext Ġvector Ġwith Ġdec oder 's Ġhidden Ġstate Ġ$ d _{ t } $ Ġand Ġform Ġa Ġnew Ġattention al Ġhidden Ġstate Ġ$\ t ilde { d } _{ t } $ Ġwhich Ġis Ġused Ġto Ġgenerate Ġwords Ġfrom Ġvocabulary Ġ$ P _{ gen } ( y _{ t } | y _{ < t }, S )= soft max ( W _{ d }\ t ilde { d } _{ t } + b _{ d }) $ ĠAlso Ġthe Ġconditional Ġcopy Ġmechanism Ġis Ġadopted Ġin Ġbase Ġmodel . ĠIt Ġintroduces Ġa Ġvariable Ġ$ z _{ t } $ Ġto Ġdecide Ġwhether Ġto Ġcopy Ġfrom Ġtables Ġor Ġgenerate Ġfrom Ġvocabulary . ĠThe Ġprobability Ġto Ġcopy Ġfrom Ġtable Ġis Ġ$ P ( z _{ t } = 1 | y _{ < t }, ĠS )= s igm oid ( w _{ e } Ġ\ cd ot Ġd _{ t } + b _{ e }) $. ĠThen Ġit Ġdecom poses Ġthe Ġconditional Ġprobability Ġof Ġgenerating Ġ$ t ^{ th } $ Ġword Ġ$ P ( y _{ t } | y _{ < t }, S ) $, Ġgiven Ġthe Ġtables Ġ$ S $ Ġand Ġpreviously Ġgenerated Ġwords Ġ$ y _{ < t } $, Ġas Ġfollows . </s> App roach </s> In Ġthis Ġsection , Ġwe Ġpropose Ġan Ġeffective Ġhierarchical Ġenc oder Ġto Ġutilize Ġthree Ġdimensional Ġstructure Ġof Ġinput Ġdata Ġin Ġorder Ġto Ġimprove Ġtable Ġrepresentation . ĠThose Ġthree Ġdimensions Ġinclude Ġrow , Ġcolumn Ġand Ġtime . ĠAs Ġshown Ġin ĠFigure ĠFIG REF 3 , Ġduring Ġencoding , Ġour Ġmodel Ġconsists Ġof Ġthree Ġlayers Ġincluding Ġrecord Ġenc od ers , Ġrecord Ġfusion Ġgate Ġand Ġrow - level Ġenc oder . ĠGiven Ġtables Ġ$ S $ Ġas Ġdescribed Ġin ĠSection ĠSEC REF 2 , Ġwe Ġfirst Ġencode Ġeach Ġrecord Ġin Ġeach Ġdimension Ġrespectively . ĠThen Ġwe Ġuse Ġthe Ġrecord Ġfusion Ġgate Ġto Ġcombine Ġthem Ġinto Ġa Ġdense Ġrepresentation . ĠAfterwards , Ġwe Ġobtain Ġrow - level Ġrepresentation Ġvia Ġmean Ġpool ing Ġand Ġself - att ention Ġwith Ġcontent Ġselection Ġgate . ĠIn Ġdecoding Ġphase , Ġthe Ġdec oder Ġcan Ġfirst Ġfind Ġimportant Ġrow Ġthen Ġattend Ġto Ġimportant Ġrecord Ġwhen Ġgenerating Ġtexts . ĠWe Ġdescribe Ġmodel 's Ġdetails Ġin Ġfollowing Ġparts . </s> App roach Ġ: :: ĠLayer Ġ1 : ĠRecord ĠEnc od ers Ġ: :: ĠRow ĠDimension ĠEnc oder </s> Based Ġon Ġour Ġobservation , Ġwhen Ġsomeone 's Ġpoints Ġis Ġmentioned Ġin Ġtexts , Ġsome Ġrelated Ġrecords Ġsuch Ġas ĠâĢ ľ field Ġgoals Ġmade âĢ Ŀ Ġ( F GM ) Ġand ĠâĢ ľ field Ġgoals Ġattempted âĢ Ŀ Ġ( F GA ) Ġwill Ġalso Ġbe Ġincluded Ġin Ġtexts . ĠTaken Ġgold Ġtexts Ġin ĠFigure ĠFIG REF 1 Ġas Ġexample , Ġwhen ĠAl ĠJefferson 's Ġpoint Ġ18 Ġis Ġmentioned , Ġhis ĠF GM Ġ9 Ġand ĠF GA Ġ19 Ġare Ġalso Ġmentioned . ĠThus , Ġwhen Ġmodeling Ġa Ġrecord , Ġother Ġrecords Ġin Ġthe Ġsame Ġrow Ġcan Ġbe Ġuseful . ĠSince Ġthe Ġrecord Ġin Ġthe Ġrow Ġis Ġnot Ġsequential , Ġwe Ġuse Ġa Ġself - att ention Ġnetwork Ġwhich Ġis Ġsimilar Ġto ĠLiu 2018 Learning ST Ġto Ġmodel Ġrecords Ġin Ġthe Ġcontext Ġof Ġother Ġrecords Ġin Ġthe Ġsame Ġrow . ĠLet Ġ$ r _{ i , Ġj } ^{ row } $ Ġbe Ġthe Ġrow Ġdimension Ġrepresentation Ġof Ġthe Ġrecord Ġof Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn . ĠThen , Ġwe Ġobtain Ġthe Ġcontext Ġvector Ġin Ġrow Ġdimension Ġ$ c _{ i , Ġj } ^{ row } $ Ġby Ġattending Ġto Ġother Ġrecords Ġin Ġthe Ġsame Ġrow Ġas Ġfollows . ĠPlease Ġnote Ġthat Ġ$\ alpha Ġ_ { i , Ġj , Ġj ^ {\ prime Ġ}} ^{ row } Ġ\ pro pt o Ġexp ( r _{ i , Ġj } ^{ T } W _{ o }{ r } _{ i , Ġj ^ {\ prime Ġ} }) $ Ġis Ġnormalized Ġacross Ġrecords Ġin Ġthe Ġsame Ġrow Ġ$ i $. Ġ$ W _{ o } $ Ġis Ġa Ġtrain able Ġparameter . </s> Then , Ġwe Ġcombine Ġrecord 's Ġrepresentation Ġwith Ġ$ c _{ i , Ġj } $ Ġand Ġobtain Ġthe Ġrow Ġdimension Ġrecord Ġrepresentation Ġ$ r _{ i , Ġj } ^{ row } = tan h ( W _{ f } [ r _{ i , Ġj }; Ġc _{ i , Ġj } ^{ row } ]) $. Ġ$ W _{ f } $ Ġis Ġa Ġtrain able Ġparameter . </s> App roach Ġ: :: ĠLayer Ġ1 : ĠRecord ĠEnc od ers Ġ: :: ĠColumn ĠDimension ĠEnc oder </s> Each Ġinput Ġtable Ġconsists Ġof Ġmultiple Ġrows Ġand Ġcolumns . ĠEach Ġcolumn Ġin Ġthe Ġtable Ġcovers Ġone Ġtype Ġof Ġinformation Ġsuch Ġas Ġpoints . ĠOnly Ġfew Ġof Ġthe Ġrow Ġmay Ġhave Ġhigh Ġpoints Ġor Ġother Ġtype Ġof Ġinformation Ġand Ġthus Ġbecome Ġthe Ġimportant Ġone . ĠFor Ġexample , Ġin ĠâĢ ľ Column ĠDimension âĢ Ŀ Ġpart Ġof ĠFigure ĠFIG REF 3 , ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġis Ġmore Ġimportant Ġthan ĠâĢ ľ Gary ĠNeal âĢ Ŀ Ġbecause Ġthe Ġformer Ġone Ġhave Ġmore Ġimpressive Ġpoints . ĠTherefore , Ġwhen Ġencoding Ġa Ġrecord , Ġit Ġis Ġhelpful Ġto Ġcompare Ġit Ġwith Ġother Ġrecords Ġin Ġthe Ġsame Ġcolumn Ġin Ġorder Ġto Ġunderstand Ġthe Ġperformance Ġlevel Ġreflected Ġby Ġthe Ġrecord Ġamong Ġhis Ġteammates Ġ( rows ). ĠWe Ġemploy Ġself - att ention Ġsimilar Ġto Ġthe Ġone Ġused Ġin ĠSection ĠSEC REF 8 Ġin Ġcolumn Ġdimension Ġto Ġcompare Ġbetween Ġrecords . ĠWe Ġlet Ġ${ r _{ i , Ġj } ^{ col }} $ Ġbe Ġthe Ġcolumn Ġrepresentation Ġof Ġthe Ġrecord Ġof Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn . ĠWe Ġobtain Ġcontext Ġvector Ġin Ġcolumn Ġdimension Ġ$ c _{ i , Ġj } ^{ col } $ Ġas Ġfollows . ĠPlease Ġnote Ġthat Ġ$\ alpha Ġ_ { j , Ġi , Ġi ^ {\ prime Ġ}} $ Ġis Ġnormalized Ġacross Ġrecords Ġfrom Ġdifferent Ġrows Ġ$ i ^ {\ prime Ġ} $ Ġbut Ġof Ġthe Ġsame Ġcolumn Ġ$ j $. ĠThe Ġcolumn Ġdimension Ġrepresentation Ġ$ r _{ i , Ġj } ^{ col } $ Ġis Ġobtained Ġsimilar Ġto Ġrow Ġdimension . </s> App roach Ġ: :: ĠLayer Ġ1 : ĠRecord ĠEnc od ers Ġ: :: ĠTime ĠDimension ĠEnc oder </s> As Ġmentioned Ġin ĠSection ĠSEC REF 1 , Ġwe Ġfind Ġsome Ġexpressions Ġin Ġtexts Ġrequire Ġinformation Ġabout Ġplayers ' Ġhistorical Ġ( time Ġdimension ) Ġperformance . ĠSo Ġthe Ġhistory Ġinformation Ġof Ġrecord Ġ$ r _{ i , j } $ Ġis Ġimportant . ĠNote Ġthat Ġwe Ġhave Ġalready Ġconstructed Ġtimeline Ġfor Ġeach Ġrecord Ġentity Ġand Ġtype Ġas Ġdescribed Ġin ĠSection ĠSEC REF 4 . ĠGiven Ġthose Ġtimelines , ĠWe Ġcollect Ġrecords Ġwith Ġsame Ġentity Ġand Ġtype Ġin Ġthe Ġtimeline Ġwhich Ġhas Ġdate Ġbefore Ġdate Ġ$ k $ Ġof Ġthe Ġrecord Ġ$ r _{ i , j } $ Ġas Ġhistory Ġinformation . ĠSince Ġfor Ġsome Ġrecord , Ġthe Ġhistory Ġinformation Ġcan Ġbe Ġtoo Ġlong , Ġwe Ġset Ġa Ġhistory Ġwindow . ĠThus , Ġwe Ġkeep Ġmost Ġrecent Ġhistory Ġinformation Ġsequence Ġwithin Ġhistory Ġwindow Ġand Ġdenote Ġthem Ġas Ġ$ hist ( r _{ i , Ġj }) $. ĠWe Ġmodel Ġthis Ġkind Ġof Ġinformation Ġin Ġtime Ġdimension Ġvia Ġself - att ention . ĠHowever , Ġunlike Ġthe Ġun ordered Ġnature Ġof Ġrows Ġand Ġcolumns , Ġthe Ġhistory Ġinformation Ġis Ġsequential . ĠTherefore , Ġwe Ġintroduce Ġa Ġtrain able Ġposition Ġembed ding Ġ$ emb _{ pos } ( k ^ {\ prime Ġ}) $ Ġand Ġadd Ġit Ġto Ġthe Ġrecord 's Ġrepresentation Ġand Ġobtain Ġa Ġnew Ġrecord Ġrepresentation Ġ${ r p } _{ k ^ {\ prime Ġ}} $. ĠIt Ġdenotes Ġthe Ġrepresentation Ġof Ġa Ġrecord Ġwith Ġthe Ġsame Ġentity Ġand Ġtype Ġof Ġ$ r _{ i , j } $ Ġbut Ġof Ġthe Ġdate Ġ$ k ^ {\ prime Ġ} $ Ġbefore Ġ$ k $ Ġin Ġthe Ġcorresponding Ġhistory Ġwindow . ĠWe Ġuse Ġ$ r _{ i , Ġj } ^{ time } $ Ġto Ġdenote Ġthe Ġhistory Ġrepresentation Ġof Ġthe Ġrecord Ġof Ġ$ i ^{ th } $ Ġrow Ġand Ġ$ j ^{ th } $ Ġcolumn . ĠThen Ġthe Ġhistory Ġdimension Ġcontext Ġvector Ġis Ġobtained Ġby Ġattending Ġto Ġhistory Ġrecords Ġin Ġthe Ġwindow . ĠPlease Ġnote Ġthat Ġwe Ġuse Ġ1 - layer ĠML P Ġas Ġscore Ġfunction Ġhere Ġand Ġ$\ alpha Ġ_ { k , Ġk ^ {\ prime Ġ}} ^{ time } $ Ġis Ġnormalized Ġwithin Ġthe Ġhistory Ġwindow . ĠWe Ġobtain Ġthe Ġtime Ġdimension Ġrepresentation Ġ$ r _{ i , Ġj } ^{ time } $ Ġsimilar Ġto Ġrow Ġdimension . </s> App roach Ġ: :: ĠLayer Ġ2 : ĠRecord ĠFusion ĠGate </s> After Ġobtaining Ġrecord Ġrepresentations Ġin Ġthree Ġdimension , Ġit Ġis Ġimportant Ġto Ġfigure Ġout Ġwhich Ġrepresentation Ġplays Ġa Ġmore Ġimportant Ġrole Ġin Ġreflecting Ġthe Ġrecord 's Ġinformation . ĠIf Ġa Ġrecord Ġstands Ġout Ġfrom Ġother Ġrow 's Ġrecords Ġof Ġsame Ġcolumn , Ġthe Ġcolumn Ġdimension Ġrepresentation Ġmay Ġhave Ġa Ġhigher Ġweight Ġin Ġforming Ġthe Ġoverall Ġrecord Ġrepresentation . ĠIf Ġa Ġrecord Ġdiffers Ġfrom Ġprevious Ġmatch Ġsignificantly , Ġthe Ġhistory Ġdimension Ġrepresentation Ġmay Ġhave Ġa Ġhigher Ġweight . ĠAlso , Ġsome Ġtypes Ġof Ġinformation Ġmay Ġappear Ġin Ġtexts Ġmore Ġfrequently Ġtogether Ġwhich Ġcan Ġbe Ġreflected Ġby Ġrow Ġdimension Ġrepresentation . ĠTherefore , Ġwe Ġpropose Ġa Ġrecord Ġfusion Ġgate Ġto Ġadapt ively Ġcombine Ġall Ġthree Ġdimension Ġrepresentations . ĠFirst , Ġwe Ġconc aten ate Ġ$ r _{ i , Ġj } ^{ row } $, Ġ$ r _{ i , Ġj } ^{ col } $ Ġand Ġ$ r _{ i , Ġj } ^{ time } $, Ġthen Ġadopt Ġa Ġ1 - layer ĠML P Ġto Ġobtain Ġa Ġgeneral Ġrepresentation Ġ$ r _{ i , Ġj } ^{ gen } $ Ġwhich Ġwe Ġconsider Ġas Ġa Ġbaseline Ġrepresentation Ġof Ġrecords ' Ġinformation . ĠThen , Ġwe Ġcompare Ġeach Ġdimension Ġrepresentation Ġwith Ġthe Ġbaseline Ġand Ġobtain Ġits Ġweight Ġin Ġthe Ġfinal Ġrecord Ġrepresentation . ĠWe Ġuse Ġ1 - layer ĠML P Ġas Ġthe Ġscore Ġfunction . ĠEqu ation ĠDIS PLAY _ FORM 15 Ġshows Ġan Ġexample Ġof Ġcalculating Ġcolumn Ġdimension Ġrepresentation 's Ġweight Ġin Ġthe Ġfinal Ġrecord Ġrepresentation . ĠThe Ġweight Ġof Ġrow Ġand Ġtime Ġdimension Ġrepresentation Ġis Ġobtained Ġsimilar Ġto Ġthe Ġweight Ġof Ġcolumn Ġdimension Ġrepresentation . </s> In Ġthe Ġend , Ġthe Ġfused Ġrecord Ġrepresentation Ġ$\ t ilde { r } _{ i , Ġj } $ Ġis Ġthe Ġweighted Ġsum Ġof Ġthe Ġthree Ġdimension Ġrepresentations . </s> App roach Ġ: :: ĠLayer Ġ3 : ĠRow - level ĠEnc oder </s> For Ġeach Ġrow , Ġwe Ġcombine Ġits Ġrecords Ġvia Ġmean Ġpool ing Ġ( Equ ation ĠDIS PLAY _ FORM 18 ) Ġin Ġorder Ġto Ġobtain Ġa Ġgeneral Ġrepresentation Ġof Ġthe Ġrow Ġwhich Ġmay Ġreflect Ġthe Ġrow Ġ( player Ġor Ġteam )' s Ġoverall Ġperformance . Ġ$ C $ Ġdenotes Ġthe Ġnumber Ġof Ġcolumns . </s> Then , Ġwe Ġadopt Ġcontent Ġselection Ġgate Ġ$ g _{ i } $, Ġwhich Ġis Ġproposed Ġby ĠP ud upp ully Ġon Ġrows ' Ġrepresentations Ġ$ row _{ i } $, Ġand Ġobtain Ġa Ġnew Ġrepresentation Ġ$\ t ilde { row } _{ i } = g _{ i }\ od ot Ġrow _{ i } $ Ġto Ġchoose Ġmore Ġimportant Ġinformation Ġbased Ġon Ġeach Ġrow 's Ġcontext . </s> App roach Ġ: :: ĠDecoder Ġwith ĠDual ĠAttention </s> Since Ġrecord Ġenc od ers Ġwith Ġrecord Ġfusion Ġgate Ġprovide Ġrecord - level Ġrepresentation Ġand Ġrow - level Ġenc oder Ġprovides Ġrow - level Ġrepresentation . ĠInspired Ġby ĠN 18 - 20 97 , Ġwe Ġcan Ġmodify Ġthe Ġdec oder Ġin Ġbase Ġmodel Ġto Ġfirst Ġchoose Ġimportant Ġrow Ġthen Ġattend Ġto Ġrecords Ġwhen Ġgenerating Ġeach Ġword . ĠFollowing Ġnot ations Ġin ĠSection ĠSEC REF 5 , Ġ$\ beta Ġ_ { t , Ġi } Ġ\ pro pt o Ġexp ( score ( d _{ t }, Ġrow _{ i } )) $ Ġob tains Ġthe Ġattention Ġweight Ġwith Ġrespect Ġto Ġeach Ġrow . ĠPlease Ġnote Ġthat Ġ$\ beta Ġ_ { t , Ġi } $ Ġis Ġnormalized Ġacross Ġall Ġrow - level Ġrepresentations Ġfrom Ġall Ġthree Ġtables . ĠThen , Ġ$\ gam ma Ġ_ { t , Ġi , Ġj } Ġ\ pro pt o Ġexp ( score ( d _{ t }, Ġ\ t ilde { r } _{ i , Ġj } )) $ Ġob tains Ġattention Ġweight Ġfor Ġrecords . ĠPlease Ġnote Ġthat Ġwe Ġnormal ize Ġ$\ gam ma Ġ_ { t , Ġi , Ġj } $ Ġamong Ġrecords Ġin Ġthe Ġsame Ġrow . </s> We Ġuse Ġthe Ġrow - level Ġattention Ġ$\ beta Ġ_ { t , Ġi } $ Ġas Ġguidance Ġfor Ġchoosing Ġrow Ġbased Ġon Ġrow 's Ġgeneral Ġrepresentation . ĠThen Ġwe Ġuse Ġit Ġto Ġre - weight Ġthe Ġrecord - level Ġattention Ġ$\ gam ma Ġ_ { t , Ġi , Ġj } $ Ġand Ġchange Ġthe Ġattention Ġweight Ġin Ġbase Ġmodel Ġto Ġ$\ t ilde {\ alpha Ġ} _{ t , Ġi , Ġj } $. ĠPlease Ġnote Ġthat Ġ${ \ t ilde {\ alpha Ġ} _{ t , Ġi , Ġj }} $ Ġsum Ġto Ġ1 Ġacross Ġall Ġrecords Ġin Ġall Ġtables . </s> App roach Ġ: :: ĠTraining </s> Given Ġa Ġbatch Ġof Ġinput Ġtables Ġ$\ l brace ĠS \ r brace Ġ_ { G } $ Ġand Ġreference Ġoutput Ġ$\ l brace ĠY \ r brace Ġ_ { G } $, Ġwe Ġuse Ġnegative Ġlog - like lihood Ġas Ġthe Ġloss Ġfunction Ġfor Ġour Ġmodel . ĠWe Ġtrain Ġthe Ġmodel Ġby Ġminimizing Ġ$ L $. Ġ$ G $ Ġis Ġthe Ġnumber Ġof Ġexamples Ġin Ġthe Ġbatch Ġand Ġ$ T _{ g } $ Ġrepresents Ġthe Ġlength Ġof Ġ$ g ^{ th } $ Ġreference 's Ġlength . </s> Exper iments Ġ: :: ĠDat as et Ġand ĠEvaluation ĠMet rics </s> We Ġconducted Ġexperiments Ġon ĠR OT OW IRE ĠB IB REF 2 . ĠFor Ġeach Ġexample , Ġit Ġprovides Ġthree Ġtables Ġas Ġdescribed Ġin ĠSection ĠSEC REF 2 Ġwhich Ġconsists Ġof Ġ6 28 Ġrecords Ġin Ġtotal Ġwith Ġa Ġlong Ġgame Ġsummary . ĠThe Ġaverage Ġlength Ġof Ġgame Ġsummary Ġis Ġ337 . 1 . ĠIn Ġthis Ġpaper , Ġwe Ġfollowed Ġthe Ġdata Ġsplit Ġintroduced Ġin ĠWis eman : Ġ3 398 Ġexamples Ġin Ġtraining Ġset , Ġ7 27 Ġexamples Ġin Ġdevelopment Ġset Ġand Ġ7 28 Ġexamples Ġin Ġtest Ġset . ĠWe Ġfollowed ĠWis eman 's Ġwork Ġand Ġuse ĠB LE U ĠB IB REF 7 Ġand Ġthree Ġextract ive Ġevaluation Ġmetrics ĠRG , ĠCS Ġand ĠCO ĠB IB REF 2 Ġfor Ġevaluation . ĠThe Ġmain Ġidea Ġof Ġthe Ġextract ive Ġevaluation Ġmetrics Ġis Ġto Ġuse Ġan ĠInformation ĠExt raction Ġ( IE ) Ġmodel Ġto Ġidentify Ġrecords Ġmentioned Ġin Ġtexts . ĠThen Ġcompare Ġthem Ġwith Ġtables Ġor Ġrecords Ġextracted Ġfrom Ġreference Ġto Ġevaluate Ġthe Ġmodel . ĠRG Ġ( Rel ation ĠGeneration ) Ġmeasures Ġcontent Ġfidelity Ġof Ġtexts . ĠCS Ġ( Content ĠSelection ) Ġmeasures Ġmodel 's Ġability Ġon Ġcontent Ġselection . ĠCO Ġ( Content ĠOrder ing ) Ġmeasures Ġmodel 's Ġability Ġon Ġordering Ġthe Ġchosen Ġrecords Ġin Ġtexts . ĠWe Ġrefer Ġthe Ġreaders Ġto ĠWis eman 's Ġpaper Ġfor Ġmore Ġdetails . </s> Exper iments Ġ: :: ĠImplementation ĠDetails </s> Following Ġconfigurations Ġin ĠP ud upp ully , Ġwe Ġset Ġword Ġembed ding Ġand ĠL ST M Ġdec oder Ġhidden Ġsize Ġas Ġ600 . ĠThe Ġdec oder 's Ġlayer Ġwas Ġset Ġto Ġbe Ġ2 . ĠInput Ġfeeding ĠB IB REF 5 Ġwas Ġalso Ġused Ġfor Ġdec oder . ĠWe Ġapplied Ġdrop out Ġat Ġa Ġrate Ġ0 . 3 . ĠFor Ġtraining , Ġwe Ġused ĠAd ag rad ĠB IB REF 8 Ġoptim izer Ġwith Ġlearning Ġrate Ġof Ġ0 . 15 , Ġtrunc ated ĠB PT T Ġ( block Ġlength Ġ100 ), Ġbatch Ġsize Ġof Ġ5 Ġand Ġlearning Ġrate Ġdecay Ġof Ġ0 . 97 . ĠFor Ġinfer ring , Ġwe Ġset Ġbeam Ġsize Ġas Ġ5 . ĠWe Ġalso Ġset Ġthe Ġhistory Ġwindows Ġsize Ġas Ġ3 Ġfrom Ġ{ 3 , 5 , 7 } Ġbased Ġon Ġthe Ġresults . ĠCode Ġof Ġour Ġmodel Ġcan Ġbe Ġfound Ġat Ġhttps :// github . com / ern est g ong / data 2 text - three - dim ensions /. </s> Exper iments Ġ: :: ĠResults Ġ: :: ĠAutomatic ĠEvaluation </s> Table ĠT AB REF 23 Ġdisplays Ġthe Ġautomatic Ġevaluation Ġresults Ġon Ġboth Ġdevelopment Ġand Ġtest Ġset . ĠWe Ġchose ĠCond itional ĠCopy Ġ( CC ) Ġmodel Ġas Ġour Ġbaseline , Ġwhich Ġis Ġthe Ġbest Ġmodel Ġin ĠWis eman . ĠWe Ġincluded Ġreported Ġscores Ġwith Ġupdated ĠIE Ġmodel Ġby ĠP ud upp ully Ġand Ġour Ġimplementation 's Ġresult Ġon ĠCC Ġin Ġthis Ġpaper . ĠAlso , Ġwe Ġcompared Ġour Ġmodels Ġwith Ġother Ġexisting Ġworks Ġon Ġthis Ġdataset Ġincluding ĠOp ATT ĠB IB REF 6 Ġand ĠNeural ĠContent ĠPlanning Ġwith Ġconditional Ġcopy Ġ( NC P + CC ) ĠB IB REF 4 . ĠIn Ġaddition , Ġwe Ġimplemented Ġthree Ġother Ġhierarchical Ġenc od ers Ġthat Ġencoded Ġtables ' Ġrow Ġdimension Ġinformation Ġin Ġboth Ġrecord - level Ġand Ġrow - level Ġto Ġcompare Ġwith Ġthe Ġhierarchical Ġstructure Ġof Ġenc oder Ġin Ġour Ġmodel . ĠThe Ġdec oder Ġwas Ġequipped Ġwith Ġdual Ġattention ĠB IB REF 9 . ĠThe Ġone Ġwith ĠL ST M Ġcell Ġis Ġsimilar Ġto Ġthe Ġone Ġin ĠN 18 - 20 97 Ġwith Ġ1 Ġlayer Ġfrom Ġ{ 1 , 2 , 3 }. ĠThe Ġone Ġwith ĠCNN Ġcell ĠB IB REF 10 Ġhas Ġkernel Ġwidth Ġ3 Ġfrom Ġ{ 3 , Ġ5 } Ġand Ġ10 Ġlayer Ġfrom Ġ{ 5 , 10 , 15 , 20 }. ĠThe Ġone Ġwith Ġtransformer - style Ġenc oder Ġ( MH SA ) ĠB IB REF 11 Ġhas Ġ8 Ġhead Ġfrom Ġ{ 8 , Ġ10 } Ġand Ġ5 Ġlayer Ġfrom Ġ{ 2 , 3 , 4 , 5 , 6 }. ĠThe Ġheads Ġand Ġlayers Ġmentioned Ġabove Ġwere Ġfor Ġboth Ġrecord - level Ġenc oder Ġand Ġrow - level Ġenc oder Ġrespectively . ĠThe Ġself - att ention Ġ( SA ) Ġcell Ġwe Ġused , Ġas Ġdescribed Ġin ĠSection ĠSEC REF 3 , Ġachieved Ġbetter Ġoverall Ġperformance Ġin Ġterms Ġof ĠF 1 % Ġof ĠCS , ĠCO Ġand ĠB LE U Ġamong Ġthe Ġhierarchical Ġenc od ers . ĠAlso Ġwe Ġimplemented Ġa Ġtemplate Ġsystem Ġsame Ġas Ġthe Ġone Ġused Ġin ĠWis eman Ġwhich Ġoutput ted Ġeight Ġsentences : Ġan Ġintroductory Ġsentence Ġ( two Ġteams ' Ġpoints Ġand Ġwho Ġwin ), Ġsix Ġtop Ġplayers ' Ġstatistics Ġ( ranked Ġby Ġtheir Ġpoints ) Ġand Ġa Ġconclusion Ġsentence . ĠWe Ġrefer Ġthe Ġreaders Ġto ĠWis eman 's Ġpaper Ġfor Ġmore Ġdetailed Ġinformation Ġon Ġtemplates . ĠThe Ġgold Ġreference 's Ġresult Ġis Ġalso Ġincluded Ġin ĠTable ĠT AB REF 23 . ĠOverall , Ġour Ġmodel Ġperforms Ġbetter Ġthan Ġother Ġneural Ġmodels Ġon Ġboth Ġdevelopment Ġand Ġtest Ġset Ġin Ġterms Ġof ĠRG 's ĠP %, ĠF 1 % Ġscore Ġof ĠCS , ĠCO Ġand ĠB LE U , Ġindicating Ġour Ġmodel 's Ġclear Ġimprovement Ġon Ġgenerating Ġhigh - f idelity , Ġinformative Ġand Ġfluent Ġtexts . ĠAlso , Ġour Ġmodel Ġwith Ġthree Ġdimension Ġrepresentations Ġoutper forms Ġhierarchical Ġenc od ers Ġwith Ġonly Ġrow Ġdimension Ġrepresentation Ġon Ġdevelopment Ġset . ĠThis Ġindicates Ġthat Ġcell Ġand Ġtime Ġdimension Ġrepresentation Ġare Ġimportant Ġin Ġrepresenting Ġthe Ġtables . ĠCompared Ġto Ġreported Ġbaseline Ġresult Ġin ĠWis eman , Ġwe Ġachieved Ġimprovement Ġof Ġ$ 22 . 27 \ % $ Ġin Ġterms Ġof ĠRG , Ġ$ 26 . 84 \ % $ Ġin Ġterms Ġof ĠCS ĠF 1 %, Ġ$ 35 . 28 \ % $ Ġin Ġterms Ġof ĠCO Ġand Ġ$ 18 . 75 \ % $ Ġin Ġterms Ġof ĠB LE U Ġon Ġtest Ġset . ĠUn surprisingly , Ġtemplate Ġsystem Ġachieves Ġbest Ġon ĠRG ĠP % Ġand ĠCS ĠR % Ġdue Ġto Ġthe Ġincluded Ġdomain Ġknowledge . ĠAlso , Ġthe Ġhigh ĠRG Ġ# Ġand Ġlow ĠCS ĠP % Ġindicates Ġthat Ġtemplate Ġwill Ġinclude Ġvast Ġinformation Ġwhile Ġmany Ġof Ġthem Ġare Ġdeemed Ġredundant . ĠIn Ġaddition , Ġthe Ġlow ĠCO Ġand Ġlow ĠB LE U Ġindicates Ġthat Ġthe Ġrigid Ġstructure Ġof Ġthe Ġtemplate Ġwill Ġproduce Ġtexts Ġthat Ġaren 't Ġas Ġadaptive Ġto Ġthe Ġgiven Ġtables Ġand Ġnatural Ġas Ġthose Ġproduced Ġby Ġneural Ġmodels . ĠAlso , Ġwe Ġconducted Ġab lation Ġstudy Ġon Ġour Ġmodel Ġto Ġevaluate Ġeach Ġcomponent 's Ġcontribution Ġon Ġdevelopment Ġset . ĠBased Ġon Ġthe Ġresults , Ġthe Ġabsence Ġof Ġrow - level Ġenc oder Ġhurts Ġour Ġmodel 's Ġperformance Ġacross Ġall Ġmetrics Ġespecially Ġthe Ġcontent Ġselection Ġability . </s> Row , Ġcolumn Ġand Ġtime Ġdimension Ġinformation Ġare Ġimportant Ġto Ġthe Ġmodeling Ġof Ġtables Ġbecause Ġsubtract ing Ġany Ġof Ġthem Ġwill Ġresult Ġin Ġperformance Ġdrop . ĠAlso , Ġposition Ġembed ding Ġis Ġcritical Ġwhen Ġmodeling Ġtime Ġdimension Ġinformation Ġaccording Ġto Ġthe Ġresults . ĠIn Ġaddition , Ġrecord Ġfusion Ġgate Ġplays Ġan Ġimportant Ġrole Ġbecause ĠB LE U , ĠCO , ĠRG ĠP % Ġand ĠCS ĠP % Ġdrop Ġsignificantly Ġafter Ġsubtract ing Ġit Ġfrom Ġfull Ġmodel . ĠResults Ġshow Ġthat Ġeach Ġcomponent Ġin Ġthe Ġmodel Ġcontributes Ġto Ġthe Ġoverall Ġperformance . ĠIn Ġaddition , Ġwe Ġcompare Ġour Ġmodel Ġwith Ġdelayed Ġcopy Ġmodel Ġ( D EL ) ĠB IB REF 12 Ġalong Ġwith Ġgold Ġtext , Ġtemplate Ġsystem Ġ( T EM ), Ġconditional Ġcopy Ġ( CC ) ĠB IB REF 2 Ġand ĠN CP + CC Ġ( NC P ) ĠB IB REF 4 . ĠLi 's Ġmodel Ġgenerate Ġa Ġtemplate Ġat Ġfirst Ġand Ġthen Ġfill Ġin Ġthe Ġslots Ġwith Ġdelayed Ġcopy Ġmechanism . ĠSince Ġits Ġresult Ġin ĠLi 's Ġpaper Ġwas Ġevaluated Ġby ĠIE Ġmodel Ġtrained Ġby ĠWis eman Ġand ĠâĢ ľ re lex ical ization âĢ Ŀ Ġby ĠLi , Ġwe Ġadopted Ġthe Ġcorresponding ĠIE Ġmodel Ġand Ġre - im plement ĠâĢ ľ re lex ical ization âĢ Ŀ Ġas Ġsuggested Ġby ĠLi Ġfor Ġfair Ġcomparison . ĠPlease Ġnote Ġthat ĠCC 's Ġevaluation Ġresults Ġvia Ġour Ġre - im ple mented ĠâĢ ľ re lex ical ization âĢ Ŀ Ġis Ġcomparable Ġto Ġthe Ġreported Ġresult Ġin ĠLi . ĠWe Ġapplied Ġthem Ġon Ġmodels Ġother Ġthan ĠDEL Ġas Ġshown Ġin ĠTable ĠT AB REF 28 Ġand Ġreport ĠDEL 's Ġresult Ġfrom ĠB IB REF 12 's Ġpaper . ĠIt Ġshows Ġthat Ġour Ġmodel Ġoutper form ĠLi 's Ġmodel Ġsignificantly Ġacross Ġall Ġautomatic Ġevaluation Ġmetrics Ġin ĠTable ĠT AB REF 28 . </s> Exper iments Ġ: :: ĠResults Ġ: :: ĠHuman ĠEvaluation </s> In Ġthis Ġsection , Ġwe Ġhired Ġthree Ġgraduates Ġwho Ġpassed Ġintermediate ĠEnglish Ġtest Ġ( College ĠEnglish ĠTest ĠBand Ġ6 ) Ġand Ġwere Ġfamiliar Ġwith ĠNBA Ġgames Ġto Ġperform Ġhuman Ġevaluation . </s> First , Ġin Ġorder Ġto Ġcheck Ġif Ġhistory Ġinformation Ġis Ġimportant , Ġwe Ġsampled Ġ100 Ġsumm aries Ġfrom Ġtraining Ġset Ġand Ġasked Ġrat ers Ġto Ġmanually Ġcheck Ġwhether Ġthe Ġsummary Ġcontained Ġexpressions Ġthat Ġneed Ġto Ġbe Ġinferred Ġfrom Ġhistory Ġinformation . ĠIt Ġturns Ġout Ġthat Ġ$ 56 . 7 \ % $ Ġsumm aries Ġof Ġthe Ġsampled Ġsumm aries Ġneed Ġhistory Ġinformation . </s> Following Ġhuman Ġevaluation Ġsettings Ġin ĠP ud upp ully , Ġwe Ġconducted Ġthe Ġfollowing Ġhuman Ġevaluation Ġexperiments Ġat Ġthe Ġsame Ġscale . ĠThe Ġsecond Ġexperiment Ġis Ġto Ġassess Ġwhether Ġthe Ġimprovement Ġon Ġrelation Ġgeneration Ġmetric Ġreported Ġin Ġautomatic Ġevaluation Ġis Ġsupported Ġby Ġhuman Ġevaluation . ĠWe Ġcompared Ġour Ġfull Ġmodel Ġwith Ġgold Ġtexts , Ġtemplate - based Ġsystem , ĠCC ĠB IB REF 2 Ġand ĠN CP + CC Ġ( NC P ) ĠB IB REF 4 . ĠWe Ġrandomly Ġsampled Ġ30 Ġexamples Ġfrom Ġtest Ġset . ĠThen , Ġwe Ġrandomly Ġsampled Ġ4 Ġsentences Ġfrom Ġeach Ġmodel 's Ġoutput Ġfor Ġeach Ġexample . ĠWe Ġprovided Ġthe Ġrat ers Ġof Ġthose Ġsampled Ġsentences Ġwith Ġthe Ġcorresponding ĠNBA Ġgame Ġstatistics . ĠThey Ġwere Ġasked Ġto Ġcount Ġthe Ġnumber Ġof Ġsupporting Ġand Ġcontradict ing Ġfacts Ġin Ġeach Ġsentence . ĠEach Ġsentence Ġis Ġrated Ġindependently . ĠWe Ġreport Ġthe Ġaverage Ġnumber Ġof Ġsupporting Ġfacts Ġ(# Sup ) Ġand Ġcontradict ing Ġfacts Ġ(# Cont ) Ġin ĠTable ĠT AB REF 30 . ĠUn surprisingly , Ġtemplate - based Ġsystem Ġincludes Ġmost Ġsupporting Ġfacts Ġand Ġleast Ġcontradict ing Ġfacts Ġin Ġits Ġtexts Ġbecause Ġthe Ġtemplate Ġconsists Ġof Ġa Ġlarge Ġnumber Ġof Ġfacts Ġand Ġall Ġof Ġthose Ġfacts Ġare Ġextracted Ġfrom Ġthe Ġtable . ĠAlso , Ġour Ġmodel Ġproduces Ġless Ġcontradict ing Ġfacts Ġthan Ġother Ġtwo Ġneural Ġmodels . ĠAlthough Ġour Ġmodel Ġproduces Ġless Ġsupporting Ġfacts Ġthan ĠN CP Ġand ĠCC , Ġit Ġstill Ġincludes Ġenough Ġsupporting Ġfacts Ġ( s lightly Ġmore Ġthan Ġgold Ġtexts ). ĠAlso , Ġcomparing Ġto ĠN CP + CC Ġ( NC P ) âĢ Ļ s Ġtendency Ġto Ġinclude Ġvast Ġinformation Ġthat Ġcontain Ġredundant Ġinformation , Ġour Ġmodel âĢ Ļ s Ġability Ġto Ġselect Ġand Ġaccurately Ġconvey Ġinformation Ġis Ġbetter . ĠAll Ġother Ġresults Ġ( Gold , ĠCC , ĠN CP Ġand Ġours ) Ġare Ġsignificantly Ġdifferent Ġfrom Ġtemplate - based Ġsystem 's Ġresults Ġin Ġterms Ġof Ġnumber Ġof Ġsupporting Ġfacts Ġaccording Ġto Ġone - way ĠAN OVA Ġwith Ġpost h oc ĠTu key ĠH SD Ġtests . ĠAll Ġsignificance Ġdifference Ġreported Ġin Ġthis Ġpaper Ġare Ġless Ġthan Ġ0 . 05 . ĠOur Ġmodel Ġis Ġalso Ġsignificantly Ġdifferent Ġfrom Ġthe ĠN CP Ġmodel . ĠAs Ġfor Ġaverage Ġnumber Ġof Ġcontradict ing Ġfacts , Ġour Ġmodel Ġis Ġsignificantly Ġdifferent Ġfrom Ġother Ġtwo Ġneural Ġmodels . ĠSurprisingly , Ġgold Ġtexts Ġwere Ġfound Ġcontaining Ġcontradict ing Ġfacts . ĠWe Ġchecked Ġthe Ġrat ers 's Ġresult Ġand Ġfound Ġthat Ġgold Ġtexts Ġoccasionally Ġinclude Ġwrong Ġfield - goal Ġor Ġthree - point Ġpercent Ġor Ġwrong Ġpoints Ġdifference Ġbetween Ġthe Ġwinner Ġand Ġthe Ġdefeated Ġteam . ĠWe Ġcan Ġtreat Ġthe Ġaverage Ġcontradict ing Ġfacts Ġnumber Ġof Ġgold Ġtexts Ġas Ġa Ġlower Ġbound . </s> In Ġthe Ġthird Ġexperiment , Ġfollowing ĠP ud upp ully , Ġwe Ġasked Ġrat ers Ġto Ġevaluate Ġthose Ġmodels Ġin Ġterms Ġof Ġgram matical ity Ġ( is Ġit Ġmore Ġfluent Ġand Ġgram matical ?), Ġco herence Ġ( is Ġit Ġeasier Ġto Ġread Ġor Ġfollows Ġmore Ġnatural Ġordering Ġof Ġfacts ? Ġ) Ġand Ġconc is eness Ġ( does Ġit Ġavoid Ġredundant Ġinformation Ġand Ġrepet itions ?). ĠWe Ġadopted Ġthe Ġsame Ġ30 Ġexamples Ġfrom Ġabove Ġand Ġarranged Ġevery Ġ5 - t uple Ġof Ġsumm aries Ġinto Ġ10 Ġpairs . ĠThen , Ġwe Ġasked Ġthe Ġrat ers Ġto Ġchoose Ġwhich Ġsystem Ġperforms Ġthe Ġbest Ġgiven Ġeach Ġpair . ĠScores Ġare Ġcomputed Ġas Ġthe Ġdifference Ġbetween Ġpercentage Ġof Ġtimes Ġwhen Ġthe Ġmodel Ġis Ġchosen Ġas Ġthe Ġbest Ġand Ġpercentage Ġof Ġtimes Ġwhen Ġthe Ġmodel Ġis Ġchosen Ġas Ġthe Ġworst . ĠGold Ġtexts Ġis Ġsignificantly Ġmore Ġgram matical Ġthan Ġothers Ġacross Ġall Ġthree Ġmetrics . ĠAlso , Ġour Ġmodel Ġperforms Ġsignificantly Ġbetter Ġthan Ġother Ġtwo Ġneural Ġmodels Ġ( CC , ĠN CP ) Ġin Ġall Ġthree Ġmetrics . ĠTemplate - based Ġsystem Ġgenerates Ġsignificantly Ġmore Ġgram matical Ġand Ġconcise Ġbut Ġsignificantly Ġless Ġcoherent Ġresults , Ġcompared Ġto Ġall Ġthree Ġneural Ġmodels . ĠBecause Ġthe Ġrigid Ġstructure Ġof Ġtexts Ġensures Ġthe Ġcorrect Ġgram matical ity Ġand Ġno Ġrepetition Ġin Ġtemplate - based Ġsystem 's Ġoutput . ĠHowever , Ġsince Ġthe Ġtemplates Ġare Ġst il ted Ġand Ġlack Ġvariability Ġcompared Ġto Ġothers , Ġit Ġwas Ġdeemed Ġless Ġcoherent Ġthan Ġthe Ġothers Ġby Ġthe Ġrat ers . </s> Exper iments Ġ: :: ĠResults Ġ: :: ĠQual itative ĠExample </s> Figure ĠFIG REF 32 Ġshows Ġan Ġexample Ġgenerated Ġby Ġour Ġmodel . ĠIt Ġevidently Ġhas Ġseveral Ġnice Ġproperties : Ġit Ġcan Ġaccurately Ġselect Ġimportant Ġplayer ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġfrom Ġthe Ġtables Ġwho Ġis Ġneglected Ġby Ġbaseline Ġmodel , Ġwhich Ġneed Ġthe Ġmodel Ġto Ġunderstand Ġperformance Ġdifference Ġof Ġa Ġtype Ġof Ġdata Ġ( column ) Ġbetween Ġeach Ġrows Ġ( players ). ĠAlso Ġit Ġcorrectly Ġsummarize Ġperformance Ġof ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġin Ġthis Ġmatch Ġas ĠâĢ ľ double - double âĢ Ŀ Ġwhich Ġrequires Ġability Ġto Ġcapture Ġdependency Ġfrom Ġdifferent Ġcolumns Ġ( different Ġtype Ġof Ġrecord ) Ġin Ġthe Ġsame Ġrow Ġ( player ). ĠIn Ġaddition , Ġit Ġmodels ĠâĢ ľ Al ĠJefferson âĢ Ŀ Ġhistory Ġperformance Ġand Ġcorrectly Ġstates Ġthat ĠâĢ ľ It Ġwas Ġhis Ġsecond Ġdouble - double Ġover Ġhis Ġlast Ġthree Ġgames âĢ Ŀ , Ġwhich Ġis Ġalso Ġmentioned Ġin Ġgold Ġtexts Ġincluded Ġin ĠFigure Ġ1 Ġin Ġa Ġsimilar Ġway . </s> Related ĠWork </s> In Ġrecent Ġyears , Ġneural Ġdata - to - text Ġsystems Ġmake Ġremarkable Ġprogress Ġon Ġgenerating Ġtexts Ġdirectly Ġfrom Ġdata . ĠMei 2016 What TT Ġproposes Ġan Ġenc oder - align er - dec oder Ġmodel Ġto Ġgenerate Ġweather Ġforecast , Ġwhile ĠN 18 - 20 98 Ġpropose Ġa Ġmixed Ġhierarchical Ġattention . ĠAAA I 18 16 203 Ġproposes Ġa Ġhybrid Ġcontent - Ġand Ġlinkage - based Ġattention Ġmechanism Ġto Ġmodel Ġthe Ġorder Ġof Ġcontent . ĠAAA I 18 165 99 Ġpropose Ġto Ġintegrate Ġfield Ġinformation Ġinto Ġtable Ġrepresentation Ġand Ġenhance Ġdec oder Ġwith Ġdual Ġattention . ĠAAA I 18 16 138 Ġdevelops Ġa Ġtable - aware Ġenc oder - dec oder Ġmodel . ĠWis eman Ġintroduced Ġa Ġdocument - scale Ġdata - to - text Ġdataset , Ġconsisting Ġof Ġlong Ġtext Ġwith Ġmore Ġredundant Ġrecords , Ġwhich Ġrequires Ġthe Ġmodel Ġto Ġselect Ġimportant Ġinformation Ġto Ġgenerate . ĠWe Ġdescribe Ġrecent Ġworks Ġin ĠSection ĠSEC REF 1 . ĠAlso , Ġsome Ġstudies Ġin Ġabstract ive Ġtext Ġsummar ization Ġencode Ġlong Ġtexts Ġin Ġa Ġhierarchical Ġmanner . ĠN 18 - 20 97 Ġuses Ġa Ġhierarchical Ġenc oder Ġto Ġencode Ġinput , Ġpaired Ġwith Ġa Ġdiscourse - aware Ġdec oder . ĠW 17 - 450 5 Ġencode Ġdocument Ġhierarch ically Ġand Ġpropose Ġcoarse - to - fine Ġattention Ġfor Ġdec oder . ĠRecently , Ġli u 2019 h ier arch ical Ġpropose Ġa Ġhierarchical Ġenc oder Ġfor Ġdata - to - text Ġgeneration Ġwhich Ġuses ĠL ST M Ġas Ġits Ġcell . Ġmur ak ami - et al - 2017 - learning Ġpropose Ġto Ġmodel Ġstock Ġmarket Ġtime - series Ġdata Ġand Ġgenerate Ġcomments . ĠAs Ġfor Ġincorporating Ġhistorical Ġbackground Ġin Ġgeneration , Ġrob in 1994 re vision Ġproposed Ġto Ġbuild Ġa Ġdraft Ġwith Ġessential Ġnew Ġfacts Ġat Ġfirst , Ġthen Ġincorporate Ġbackground Ġfacts Ġwhen Ġrev ising Ġthe Ġdraft Ġbased Ġon Ġfunctional Ġunification Ġgram m ars . ĠDifferent Ġfrom Ġthat , Ġwe Ġencode Ġthe Ġhistorical Ġ( time Ġdimension ) Ġinformation Ġin Ġthe Ġneural Ġdata - to - text Ġmodel Ġin Ġan Ġend - to - end Ġfashion . ĠEx isting Ġworks Ġon Ġdata - to - text Ġgeneration Ġneglect Ġthe Ġjoint Ġrepresentation Ġof Ġtables ' Ġrow , Ġcolumn Ġand Ġtime Ġdimension Ġinformation . ĠIn Ġthis Ġpaper , Ġwe Ġpropose Ġan Ġeffective Ġhierarchical Ġenc oder Ġwhich Ġmodels Ġinformation Ġfrom Ġrow , Ġcolumn Ġand Ġtime Ġdimension Ġsimultaneously . </s> Conclusion </s> In Ġthis Ġwork , Ġwe Ġpresent Ġan Ġeffective Ġhierarchical Ġenc oder Ġfor Ġtable - to - text Ġgeneration Ġthat Ġlearns Ġtable Ġrepresentations Ġfrom Ġrow , Ġcolumn Ġand Ġtime Ġdimension . ĠIn Ġdetail , Ġour Ġmodel Ġconsists Ġof Ġthree Ġlayers , Ġwhich Ġlearn Ġrecords ' Ġrepresentation Ġin Ġthree Ġdimension , Ġcombine Ġthose Ġrepresentations Ġvia Ġtheir Ġsail ency Ġand Ġobtain Ġrow - level Ġrepresentation Ġbased Ġon Ġrecords ' Ġrepresentation . ĠThen , Ġduring Ġdecoding , Ġit Ġwill Ġselect Ġimportant Ġtable Ġrow Ġbefore Ġattending Ġto Ġrecords . ĠExper iments Ġare Ġconducted Ġon ĠR OT OW IRE , Ġa Ġbenchmark Ġdataset Ġof ĠNBA Ġgames . ĠBoth Ġautomatic Ġand Ġhuman Ġevaluation Ġresults Ġshow Ġthat Ġour Ġmodel Ġachieves Ġthe Ġnew Ġstate - of - the - art Ġperformance . </s> Acknowled gements </s> We Ġwould Ġlike Ġto Ġthank Ġthe Ġanonymous Ġreviewers Ġfor Ġtheir Ġhelpful Ġcomments . ĠWe 'd Ġalso Ġlike Ġto Ġthank ĠXin wei ĠG eng , ĠY ibo ĠSun , ĠZheng p eng ĠXiang Ġand ĠY uy u ĠChen Ġfor Ġtheir Ġvaluable Ġinput . ĠThis Ġwork Ġwas Ġsupported Ġby Ġthe ĠNational ĠKey ĠR & D ĠProgram Ġof ĠChina Ġvia Ġgrant Ġ2018 Y FB 100 5 103 Ġand ĠNational ĠNatural ĠScience ĠFoundation Ġof ĠChina Ġ( NS FC ) Ġvia Ġgrant Ġ6 163 2011 Ġand Ġ6 17 72 156 .