from Food.com
EXTRACTIVE
We collect a novel dataset of 230K+ recipe texts and 1M+ user interactions (reviews) over 18 years (2000-2018) from Food.com. Here, we restrict to recipes with at least 3 steps, and at least 4 and no more than 20 ingredients. We discard users with fewer than 4 reviews, giving 180K+ recipes and 700K+ reviews, with splits as in tab:recipeixnstats.
1909.00105
Introduction </s> In Ġthe Ġkitchen , Ġwe Ġincreasingly Ġrely Ġon Ġinstructions Ġfrom Ġcooking Ġwebsites : Ġrecipes . ĠA Ġcook Ġwith Ġa Ġpred ile ction Ġfor ĠAsian Ġcuisine Ġmay Ġwish Ġto Ġprepare Ġchicken Ġcurry , Ġbut Ġmay Ġnot Ġknow Ġall Ġnecessary Ġingredients Ġapart Ġfrom Ġa Ġfew Ġbasics . ĠThese Ġusers Ġwith Ġlimited Ġknowledge Ġcannot Ġrely Ġon Ġexisting Ġrecipe Ġgeneration Ġapproaches Ġthat Ġfocus Ġon Ġcreating Ġcoherent Ġrecipes Ġgiven Ġall Ġingredients Ġand Ġa Ġrecipe Ġname ĠB IB REF 0 . ĠSuch Ġmodels Ġdo Ġnot Ġaddress Ġissues Ġof Ġpersonal Ġpreference Ġ( e . g . Ġculinary Ġtastes , Ġgarn ish Ġchoices ) Ġand Ġincomplete Ġrecipe Ġdetails . ĠWe Ġpropose Ġto Ġapproach Ġboth Ġproblems Ġvia Ġpersonalized Ġgeneration Ġof Ġplausible , Ġuser - specific Ġrecipes Ġusing Ġuser Ġpreferences Ġextracted Ġfrom Ġpreviously Ġconsumed Ġrecipes . </s> Our Ġwork Ġcombines Ġtwo Ġimportant Ġtasks Ġfrom Ġnatural Ġlanguage Ġprocessing Ġand Ġrecomm ender Ġsystems : Ġdata - to - text Ġgeneration ĠB IB REF 1 Ġand Ġpersonalized Ġrecommendation ĠB IB REF 2 . ĠOur Ġmodel Ġtakes Ġas Ġuser Ġinput Ġthe Ġname Ġof Ġa Ġspecific Ġdish , Ġa Ġfew Ġkey Ġingredients , Ġand Ġa Ġcalorie Ġlevel . ĠWe Ġpass Ġthese Ġloose Ġinput Ġspecifications Ġto Ġan Ġenc oder - dec oder Ġframework Ġand Ġattend Ġon Ġuser Ġprofiles âĢĶ learn ed Ġlatent Ġrepresentations Ġof Ġrecipes Ġpreviously Ġconsumed Ġby Ġa Ġuser âĢĶ to Ġgenerate Ġa Ġrecipe Ġpersonalized Ġto Ġthe Ġuser 's Ġtastes . ĠWe Ġfuse Ġthese Ġ` user - aware ' Ġrepresentations Ġwith Ġdec oder Ġoutput Ġin Ġan Ġattention Ġfusion Ġlayer Ġto Ġjointly Ġdetermine Ġtext Ġgeneration . ĠQuant itative Ġ( per plex ity , Ġuser - ranking ) Ġand Ġqualitative Ġanalysis Ġon Ġuser - aware Ġmodel Ġoutputs Ġconfirm Ġthat Ġpersonal ization Ġindeed Ġassists Ġin Ġgenerating Ġplausible Ġrecipes Ġfrom Ġincomplete Ġingredients . </s> While Ġpersonalized Ġtext Ġgeneration Ġhas Ġseen Ġsuccess Ġin Ġconve ying Ġuser Ġwriting Ġstyles Ġin Ġthe Ġproduct Ġreview ĠB IB REF 3 , ĠB IB REF 4 Ġand Ġdialogue ĠB IB REF 5 Ġspaces , Ġwe Ġare Ġthe Ġfirst Ġto Ġconsider Ġit Ġfor Ġthe Ġproblem Ġof Ġrecipe Ġgeneration , Ġwhere Ġoutput Ġquality Ġis Ġheavily Ġdependent Ġon Ġthe Ġcontent Ġof Ġthe Ġinstructions âĢĶ such Ġas Ġingredients Ġand Ġcooking Ġtechniques . </s> To Ġsummarize , Ġour Ġmain Ġcontributions Ġare Ġas Ġfollows : </s> We Ġexplore Ġa Ġnew Ġtask Ġof Ġgenerating Ġplausible Ġand Ġpersonalized Ġrecipes Ġfrom Ġincomplete Ġinput Ġspecifications Ġby Ġleveraging Ġhistorical Ġuser Ġpreferences ; </s> We Ġrelease Ġa Ġnew Ġdataset Ġof Ġ180 K + Ġrecipes Ġand Ġ700 K + Ġuser Ġreviews Ġfor Ġthis Ġtask ; </s> We Ġintroduce Ġnew Ġevaluation Ġstrategies Ġfor Ġgeneration Ġquality Ġin Ġinstructional Ġtexts , Ġcent ering Ġon Ġquantitative Ġmeasures Ġof Ġco herence . ĠWe Ġalso Ġshow Ġqual itatively Ġand Ġquant itatively Ġthat Ġpersonalized Ġmodels Ġgenerate Ġhigh - quality Ġand Ġspecific Ġrecipes Ġthat Ġalign Ġwith Ġhistorical Ġuser Ġpreferences . </s> Related ĠWork </s> Large - scale Ġtransformer - based Ġlanguage Ġmodels Ġhave Ġshown Ġsurprising Ġexpress ivity Ġand Ġflu ency Ġin Ġcreative Ġand Ġconditional Ġlong - text Ġgeneration ĠB IB REF 6 , ĠB IB REF 7 . ĠRecent Ġworks Ġhave Ġproposed Ġhierarchical Ġmethods Ġthat Ġcondition Ġon Ġnarrative Ġframeworks Ġto Ġgenerate Ġinternally Ġconsistent Ġlong Ġtexts ĠB IB REF 8 , ĠB IB REF 9 , ĠB IB REF 10 . ĠHere , Ġwe Ġgenerate Ġproced urally Ġstructured Ġrecipes Ġinstead Ġof Ġfree - form Ġnarratives . </s> Recipe Ġgeneration Ġbelongs Ġto Ġthe Ġfield Ġof Ġdata - to - text Ġnatural Ġlanguage Ġgeneration ĠB IB REF 1 , Ġwhich Ġsees Ġother Ġapplications Ġin Ġautomated Ġjournalism ĠB IB REF 11 , Ġquestion - ans w ering ĠB IB REF 12 , Ġand Ġabstract ive Ġsummar ization ĠB IB REF 13 , Ġamong Ġothers . ĠB IB REF 14 , ĠB IB REF 15 Ġmodel Ġrecipes Ġas Ġa Ġstructured Ġcollection Ġof Ġingredient Ġentities Ġacted Ġupon Ġby Ġcooking Ġactions . ĠB IB REF 0 Ġimposes Ġa Ġ` check list ' Ġattention Ġconstraint Ġemphasizing Ġhitherto Ġunused Ġingredients Ġduring Ġgeneration . ĠB IB REF 16 Ġattend Ġover Ġexplicit Ġingredient Ġreferences Ġin Ġthe Ġprior Ġrecipe Ġstep . ĠSimilar Ġhierarchical Ġapproaches Ġthat Ġinfer Ġa Ġfull Ġingredient Ġlist Ġto Ġconst rain Ġgeneration Ġwill Ġnot Ġhelp Ġpersonal ize Ġrecipes , Ġand Ġwould Ġbe Ġinf eas ible Ġin Ġour Ġsetting Ġdue Ġto Ġthe Ġpotentially Ġuncon str ained Ġnumber Ġof Ġingredients Ġ( from Ġa Ġspace Ġof Ġ10 K +) Ġin Ġa Ġrecipe . ĠWe Ġinstead Ġlearn Ġhistorical Ġpreferences Ġto Ġguide Ġfull Ġrecipe Ġgeneration . </s> A Ġrecent Ġline Ġof Ġwork Ġhas Ġexplored Ġuser - Ġand Ġitem - dependent Ġaspect - aware Ġreview Ġgeneration ĠB IB REF 3 , ĠB IB REF 4 . ĠThis Ġwork Ġis Ġrelated Ġto Ġours Ġin Ġthat Ġit Ġcombines Ġcontextual Ġlanguage Ġgeneration Ġwith Ġpersonal ization . ĠHere , Ġwe Ġattend Ġover Ġhistorical Ġuser Ġpreferences Ġfrom Ġpreviously Ġconsumed Ġrecipes Ġto Ġgenerate Ġrecipe Ġcontent , Ġrather Ġthan Ġwriting Ġstyles . </s> App roach </s> Our Ġmodel 's Ġinput Ġspecification Ġconsists Ġof : Ġthe Ġrecipe Ġname Ġas Ġa Ġsequence Ġof Ġtokens , Ġa Ġpartial Ġlist Ġof Ġingredients , Ġand Ġa Ġcaloric Ġlevel Ġ( high , Ġmedium , Ġlow ). ĠIt Ġoutputs Ġthe Ġrecipe Ġinstructions Ġas Ġa Ġtoken Ġsequence : Ġ$\ math cal Ġ{ W } _ r = \ l brace Ġw _{ r , 0 }, Ġ\ d ots Ġ, Ġw _{ r , T }\ r brace Ġ$ Ġfor Ġa Ġrecipe Ġ$ r $ Ġof Ġlength Ġ$ T $. ĠTo Ġpersonal ize Ġoutput , Ġwe Ġuse Ġhistorical Ġrecipe Ġinteractions Ġof Ġa Ġuser Ġ$ u Ġ\ in Ġ\ math cal Ġ{ U } $. </s> Enc oder : ĠOur Ġenc oder Ġhas Ġthree Ġembed ding Ġlayers : Ġvocabulary Ġembed ding Ġ$\ math cal Ġ{ V } $, Ġingredient Ġembed ding Ġ$\ math cal Ġ{ I } $, Ġand Ġcaloric - level Ġembed ding Ġ$\ math cal Ġ{ C } $. ĠEach Ġtoken Ġin Ġthe Ġ( length Ġ$ L _ n $ ) Ġrecipe Ġname Ġis Ġembedded Ġvia Ġ$\ math cal Ġ{ V } $ ; Ġthe Ġembedded Ġtoken Ġsequence Ġis Ġpassed Ġto Ġa Ġtwo - lay ered Ġbid irection al ĠGR U Ġ( Bi GR U ) ĠB IB REF 17 , Ġwhich Ġoutputs Ġhidden Ġstates Ġfor Ġnames Ġ$\ l brace Ġ\ math bf Ġ{ n } _{ \ text { enc }, j } Ġ\ in Ġ\ math bb Ġ{ R } ^{ 2 d _ h }\ r brace Ġ$ , Ġwith Ġhidden Ġsize Ġ$ d _ h $. ĠSimilarly Ġeach Ġof Ġthe Ġ$ L _ i $ Ġinput Ġingredients Ġis Ġembedded Ġvia Ġ$\ math cal Ġ{ I } $, Ġand Ġthe Ġembedded Ġingredient Ġsequence Ġis Ġpassed Ġto Ġanother Ġtwo - lay ered ĠBi GR U Ġto Ġoutput Ġingredient Ġhidden Ġstates Ġas Ġ$\ l brace Ġ\ math bf Ġ{ i } _{ \ text { enc }, j } Ġ\ in Ġ\ math bb Ġ{ R } ^{ 2 d _ h }\ r brace Ġ$ . ĠThe Ġcaloric Ġlevel Ġis Ġembedded Ġvia Ġ$\ math cal Ġ{ C } $ Ġand Ġpassed Ġthrough Ġa Ġprojection Ġlayer Ġwith Ġweights Ġ$ W _ c $ Ġto Ġgenerate Ġcalorie Ġhidden Ġrepresentation Ġ$\ math bf Ġ{ c } _{ \ text { enc }} Ġ\ in Ġ\ math bb Ġ{ R } ^{ 2 d _ h } $. </s> Ing red ient ĠAttention : ĠWe Ġapply Ġattention ĠB IB REF 18 Ġover Ġthe Ġencoded Ġingredients Ġto Ġuse Ġenc oder Ġoutputs Ġat Ġeach Ġdecoding Ġtime Ġstep . ĠWe Ġdefine Ġan Ġattention - score Ġfunction Ġ$\ alpha Ġ$ Ġwith Ġkey Ġ$ K $ Ġand Ġquery Ġ$ Q $ : </s> with Ġtrain able Ġweights Ġ$ W _{ \ alpha Ġ} $, Ġbias Ġ$\ math bf Ġ{ b } _{ \ alpha Ġ} $, Ġand Ġnormal ization Ġterm Ġ$ Z $. ĠAt Ġdecoding Ġtime Ġ$ t $, Ġwe Ġcalculate Ġthe Ġingredient Ġcontext Ġ$\ math bf Ġ{ a } _{ t } ^{ i } Ġ\ in Ġ\ math bb Ġ{ R } ^{ d _ h } $ Ġas : </s> Dec oder : ĠThe Ġdec oder Ġis Ġa Ġtwo - layer ĠGR U Ġwith Ġhidden Ġstate Ġ$ h _ t $ Ġconditioned Ġon Ġprevious Ġhidden Ġstate Ġ$ h _{ t - 1 } $ Ġand Ġinput Ġtoken Ġ$ w _{ r , Ġt } $ Ġfrom Ġthe Ġoriginal Ġrecipe Ġtext . ĠWe Ġproject Ġthe Ġconc aten ated Ġenc oder Ġoutputs Ġas Ġthe Ġinitial Ġdec oder Ġhidden Ġstate : </s> To Ġbias Ġgeneration Ġtoward Ġuser Ġpreferences , Ġwe Ġattend Ġover Ġa Ġuser 's Ġpreviously Ġreviewed Ġrecipes Ġto Ġjointly Ġdetermine Ġthe Ġfinal Ġoutput Ġtoken Ġdistribution . ĠWe Ġconsider Ġtwo Ġdifferent Ġschemes Ġto Ġmodel Ġpreferences Ġfrom Ġuser Ġhistories : Ġ( 1 ) Ġrecipe Ġinteractions , Ġand Ġ( 2 ) Ġtechniques Ġseen Ġtherein Ġ( defined Ġin Ġdata ). ĠB IB REF 19 , ĠB IB REF 20 , ĠB IB REF 21 Ġexplore Ġsimilar Ġschemes Ġfor Ġpersonalized Ġrecommendation . </s> Prior ĠRecipe ĠAttention : ĠWe Ġobtain Ġthe Ġset Ġof Ġprior Ġrecipes Ġfor Ġa Ġuser Ġ$ u $ : Ġ$ R ^ + _ u $, Ġwhere Ġeach Ġrecipe Ġcan Ġbe Ġrepresented Ġby Ġan Ġembed ding Ġfrom Ġa Ġrecipe Ġembed ding Ġlayer Ġ$\ math cal Ġ{ R } $ Ġor Ġan Ġaverage Ġof Ġthe Ġname Ġtokens Ġembedded Ġby Ġ$\ math cal Ġ{ V } $. ĠWe Ġattend Ġover Ġthe Ġ$ k $ - most Ġrecent Ġprior Ġrecipes , Ġ$ R ^{ k + } _ u $, Ġto Ġaccount Ġfor Ġtemporal Ġdrift Ġof Ġuser Ġpreferences ĠB IB REF 22 . ĠThese Ġembed d ings Ġare Ġused Ġin Ġthe Ġ` Prior ĠRecipe ' Ġand Ġ` Prior ĠName ' Ġmodels , Ġrespectively . </s> Given Ġa Ġrecipe Ġrepresentation Ġ$\ math bf Ġ{ r } Ġ\ in Ġ\ math bb Ġ{ R } ^{ d _ r } $ Ġ( where Ġ$ d _ r $ Ġis Ġrecipe - Ġor Ġvocabulary - embed ding Ġsize Ġdepending Ġon Ġthe Ġrecipe Ġrepresentation ) Ġthe Ġprior Ġrecipe Ġattention Ġcontext Ġ$\ math bf Ġ{ a } _{ t } ^{ r _ u } $ Ġis Ġcalculated Ġas </s> Prior ĠTechnique ĠAttention : ĠWe Ġcalculate Ġprior Ġtechnique Ġpreference Ġ( used Ġin Ġthe Ġ` Prior ĠTech ` Ġmodel ) Ġby Ġnormal izing Ġco - occ urrence Ġbetween Ġusers Ġand Ġtechniques Ġseen Ġin Ġ$ R ^ + _ u $, Ġto Ġobtain Ġa Ġpreference Ġvector Ġ$\ r ho Ġ_ { u } $. ĠEach Ġtechnique Ġ$ x $ Ġis Ġembedded Ġvia Ġa Ġtechnique Ġembed ding Ġlayer Ġ$\ math cal Ġ{ X } $ Ġto Ġ$\ math bf Ġ{ x }\ in Ġ\ math bb Ġ{ R } ^{ d _ x } $. ĠPrior Ġtechnique Ġattention Ġis Ġcalculated Ġas </s> where , Ġinspired Ġby Ġcopy Ġmechanisms ĠB IB REF 23 , ĠB IB REF 24 , Ġwe Ġadd Ġ$\ r ho Ġ_ { u , x } $ Ġfor Ġtechnique Ġ$ x $ Ġto Ġemphasize Ġthe Ġattention Ġby Ġthe Ġuser 's Ġprior Ġtechnique Ġpreference . </s> Att ention ĠFusion ĠLayer : ĠWe Ġfuse Ġall Ġcontexts Ġcalculated Ġat Ġtime Ġ$ t $, Ġconc aten ating Ġthem Ġwith Ġdec oder ĠGR U Ġoutput Ġand Ġprevious Ġtoken Ġembed ding : </s> We Ġthen Ġcalculate Ġthe Ġtoken Ġprobability : </s> and Ġmaximize Ġthe Ġlog - like lihood Ġof Ġthe Ġgenerated Ġsequence Ġconditioned Ġon Ġinput Ġspecifications Ġand Ġuser Ġpreferences . Ġfig : ex Ġshows Ġa Ġcase Ġwhere Ġthe ĠPrior ĠName Ġmodel Ġattends Ġstrongly Ġon Ġpreviously Ġconsumed Ġsav ory Ġrecipes Ġto Ġsuggest Ġthe Ġusage Ġof Ġan Ġadditional Ġingredient Ġ( ` cil ant ro '). </s> Recipe ĠDat as et : ĠFood . com </s> We Ġcollect Ġa Ġnovel Ġdataset Ġof Ġ230 K + Ġrecipe Ġtexts Ġand Ġ1 M + Ġuser Ġinteractions Ġ( review s ) Ġover Ġ18 Ġyears Ġ( 2000 - 2018 ) Ġfrom ĠFood . com . ĠHere , Ġwe Ġrestrict Ġto Ġrecipes Ġwith Ġat Ġleast Ġ3 Ġsteps , Ġand Ġat Ġleast Ġ4 Ġand Ġno Ġmore Ġthan Ġ20 Ġingredients . ĠWe Ġdiscard Ġusers Ġwith Ġfewer Ġthan Ġ4 Ġreviews , Ġgiving Ġ180 K + Ġrecipes Ġand Ġ700 K + Ġreviews , Ġwith Ġsplits Ġas Ġin Ġtab : reci pe ix n stats . </s> Our Ġmodel Ġmust Ġlearn Ġto Ġgenerate Ġfrom Ġa Ġdiverse Ġrecipe Ġspace : Ġin Ġour Ġtraining Ġdata , Ġthe Ġaverage Ġrecipe Ġlength Ġis Ġ117 Ġtokens Ġwith Ġa Ġmaximum Ġof Ġ256 . ĠThere Ġare Ġ13 K Ġunique Ġingredients Ġacross Ġall Ġrecipes . ĠRare Ġwords Ġdominate Ġthe Ġvocabulary : Ġ95 % Ġof Ġwords Ġappear Ġ$ < $ 100 Ġtimes , Ġaccounting Ġfor Ġonly Ġ1 . 65 % Ġof Ġall Ġword Ġusage . ĠAs Ġsuch , Ġwe Ġperform ĠByte - P air ĠEnc oding Ġ( B PE ) Ġtoken ization ĠB IB REF 25 , ĠB IB REF 26 , Ġgiving Ġa Ġtraining Ġvocabulary Ġof Ġ15 K Ġtokens Ġacross Ġ19 M Ġtotal Ġmentions . ĠUser Ġprofiles Ġare Ġsimilarly Ġdiverse : Ġ50 % Ġof Ġusers Ġhave Ġconsumed Ġ$\ le Ġ$ 6 Ġrecipes , Ġwhile Ġ10 % Ġof Ġusers Ġhave Ġconsumed Ġ$ > $ 45 Ġrecipes . </s> We Ġorder Ġreviews Ġby Ġtimestamp , Ġkeeping Ġthe Ġmost Ġrecent Ġreview Ġfor Ġeach Ġuser Ġas Ġthe Ġtest Ġset , Ġthe Ġsecond Ġmost Ġrecent Ġfor Ġvalidation , Ġand Ġthe Ġremainder Ġfor Ġtraining Ġ( sequ ential Ġleave - one - out Ġevaluation ĠB IB REF 27 ). ĠWe Ġevaluate Ġonly Ġon Ġrecipes Ġnot Ġin Ġthe Ġtraining Ġset . </s> We Ġmanually Ġconstruct Ġa Ġlist Ġof Ġ58 Ġcooking Ġtechniques Ġfrom Ġ384 Ġcooking Ġactions Ġcollected Ġby ĠB IB REF 15 ; Ġthe Ġmost Ġcommon Ġtechniques Ġ( b ake , Ġcombine , Ġpour , Ġboil ) Ġaccount Ġfor Ġ36 . 5 % Ġof Ġtechnique Ġmentions . ĠWe Ġapproximate Ġtechnique Ġadherence Ġvia Ġstring Ġmatch Ġbetween Ġthe Ġrecipe Ġtext Ġand Ġtechnique Ġlist . </s> Exper iments Ġand ĠResults </s> For Ġtraining Ġand Ġevaluation , Ġwe Ġprovide Ġour Ġmodel Ġwith Ġthe Ġfirst Ġ3 - 5 Ġingredients Ġlisted Ġin Ġeach Ġrecipe . ĠWe Ġdecode Ġrecipe Ġtext Ġvia Ġtop -$ k $ Ġsampling ĠB IB REF 7 , Ġfinding Ġ$ k = 3 $ Ġto Ġproduce Ġsatisfactory Ġresults . ĠWe Ġuse Ġa Ġhidden Ġsize Ġ$ d _ h = 256 $ Ġfor Ġboth Ġthe Ġenc oder Ġand Ġdec oder . ĠEmb edd ing Ġdimensions Ġfor Ġvocabulary , Ġingredient , Ġrecipe , Ġtechniques , Ġand Ġcaloric Ġlevel Ġare Ġ300 , Ġ10 , Ġ50 , Ġ50 , Ġand Ġ5 Ġ( respect ively ). ĠFor Ġprior Ġrecipe Ġattention , Ġwe Ġset Ġ$ k = 20 $, Ġthe Ġ80 th Ġ% - ile Ġfor Ġthe Ġnumber Ġof Ġuser Ġinteractions . ĠWe Ġuse Ġthe ĠAdam Ġoptim izer ĠB IB REF 28 Ġwith Ġa Ġlearning Ġrate Ġof Ġ$ 10 ^{ - 3 } $, Ġan ne aled Ġwith Ġa Ġdecay Ġrate Ġof Ġ0 . 9 ĠB IB REF 29 . ĠWe Ġalso Ġuse Ġteacher - forcing ĠB IB REF 30 Ġin Ġall Ġtraining Ġepoch s . </s> In Ġthis Ġwork , Ġwe Ġinvestigate Ġhow Ġleveraging Ġhistorical Ġuser Ġpreferences Ġcan Ġimprove Ġgeneration Ġquality Ġover Ġstrong Ġbas elines Ġin Ġour Ġsetting . ĠWe Ġcompare Ġour Ġpersonalized Ġmodels Ġagainst Ġtwo Ġbas elines . ĠThe Ġfirst Ġis Ġa Ġname - based ĠNe arest - Neigh bor Ġmodel Ġ( NN ). ĠWe Ġinitially Ġadapted Ġthe ĠNeural ĠCheck list ĠModel Ġof ĠB IB REF 0 Ġas Ġa Ġbaseline ; Ġhowever , Ġwe Ġultimately Ġuse Ġa Ġsimple ĠEnc oder - Dec oder Ġbaseline Ġwith Ġingredient Ġattention Ġ( Enc - Dec ), Ġwhich Ġprovides Ġcomparable Ġperformance Ġand Ġlower Ġcomplexity . ĠAll Ġpersonalized Ġmodels Ġoutper form Ġbaseline Ġin ĠB PE Ġperplex ity Ġ( tab : met rics ont est ) Ġwith ĠPrior ĠName Ġperforming Ġthe Ġbest . ĠWhile Ġour Ġmodels Ġexhibit Ġcomparable Ġperformance Ġto Ġbaseline Ġin ĠB LE U - 1 / 4 Ġand ĠR OU GE - L , Ġwe Ġgenerate Ġmore Ġdiverse Ġ( Dist inct - 1 / 2 : Ġpercentage Ġof Ġdistinct Ġun ig rams Ġand Ġbig rams ) Ġand Ġacceptable Ġrecipes . ĠB LE U Ġand ĠR OU GE Ġare Ġnot Ġthe Ġmost Ġappropriate Ġmetrics Ġfor Ġgeneration Ġquality . ĠA Ġ` correct ' Ġrecipe Ġcan Ġbe Ġwritten Ġin Ġmany Ġways Ġwith Ġthe Ġsame Ġmain Ġentities Ġ( ing redients ). ĠAs ĠB LE U - 1 / 4 Ġcapture Ġstructural Ġinformation Ġvia Ġn - gram Ġmatching , Ġthey Ġare Ġnot Ġcorrelated Ġwith Ġsubjective Ġrecipe Ġquality . ĠThis Ġmirrors Ġobservations Ġfrom ĠB IB REF 31 , ĠB IB REF 8 . </s> We Ġobserve Ġthat Ġpersonalized Ġmodels Ġmake Ġmore Ġdiverse Ġrecipes Ġthan Ġbaseline . ĠThey Ġthus Ġperform Ġbetter Ġin ĠB LE U - 1 Ġwith Ġmore Ġkey Ġentities Ġ( ing red ient Ġmentions ) Ġpresent , Ġbut Ġworse Ġin ĠB LE U - 4 , Ġas Ġthese Ġrecipes Ġare Ġwritten Ġin Ġa Ġpersonalized Ġway Ġand Ġdev iate Ġfrom Ġgold Ġon Ġthe Ġphr as al Ġlevel . ĠSimilarly , Ġthe Ġ` Prior ĠName ' Ġmodel Ġgenerates Ġmore Ġun ig ram - d iverse Ġrecipes Ġthan Ġother Ġpersonalized Ġmodels Ġand Ġob tains Ġa Ġcorrespond ingly Ġlower ĠB LE U - 1 Ġscore . </s> Qual itative ĠAnalysis : ĠWe Ġpresent Ġsample Ġoutputs Ġfor Ġa Ġcocktail Ġrecipe Ġin Ġtab : sample rec ipes , Ġand Ġadditional Ġrecipes Ġin Ġthe Ġappendix . ĠGeneration Ġquality Ġprogressively Ġimproves Ġfrom Ġgeneric Ġbaseline Ġoutput Ġto Ġa Ġblended Ġcocktail Ġproduced Ġby Ġour Ġbest Ġperforming Ġmodel . ĠModels Ġattending Ġover Ġprior Ġrecipes Ġexplicitly Ġreference Ġingredients . ĠThe ĠPrior ĠName Ġmodel Ġfurther Ġsuggests Ġthe Ġaddition Ġof Ġlemon Ġand Ġmint , Ġwhich Ġare Ġreasonably Ġassociated Ġwith Ġpreviously Ġconsumed Ġrecipes Ġlike Ġcoconut Ġm ousse Ġand Ġpork Ġskew ers . </s> Personal ization : ĠTo Ġmeasure Ġpersonal ization , Ġwe Ġevaluate Ġhow Ġclosely Ġthe Ġgenerated Ġtext Ġcorresponds Ġto Ġa Ġparticular Ġuser Ġprofile . ĠWe Ġcompute Ġthe Ġlikelihood Ġof Ġgenerated Ġrecipes Ġusing Ġidentical Ġinput Ġspecifications Ġbut Ġconditioned Ġon Ġten Ġdifferent Ġuser Ġprofiles âĢĶ one Ġ` gold ' Ġuser Ġwho Ġconsumed Ġthe Ġoriginal Ġrecipe , Ġand Ġnine Ġrandomly Ġgenerated Ġuser Ġprofiles . ĠFollowing ĠB IB REF 8 , Ġwe Ġexpect Ġthe Ġhighest Ġlikelihood Ġfor Ġthe Ġrecipe Ġconditioned Ġon Ġthe Ġgold Ġuser . ĠWe Ġmeasure Ġuser Ġmatching Ġaccuracy Ġ( U MA )âĢĶ the Ġproportion Ġwhere Ġthe Ġgold Ġuser Ġis Ġranked Ġhighest âĢĶ and ĠMean ĠRec ip rocal ĠRank Ġ( MR R ) ĠB IB REF 32 Ġof Ġthe Ġgold Ġuser . ĠAll Ġpersonalized Ġmodels Ġbeat Ġbas elines Ġin Ġboth Ġmetrics , Ġshowing Ġour Ġmodels Ġpersonal ize Ġgenerated Ġrecipes Ġto Ġthe Ġgiven Ġuser Ġprofiles . ĠThe ĠPrior ĠName Ġmodel Ġachieves Ġthe Ġbest ĠU MA Ġand ĠMR R Ġby Ġa Ġlarge Ġmargin , Ġrevealing Ġthat Ġprior Ġrecipe Ġnames Ġare Ġstrong Ġsignals Ġfor Ġpersonal ization . ĠMoreover , Ġthe Ġaddition Ġof Ġattention Ġmechanisms Ġto Ġcapture Ġthese Ġsignals Ġimproves Ġlanguage Ġmodeling Ġperformance Ġover Ġa Ġstrong Ġnon - personal ized Ġbaseline . </s> Recipe ĠLevel ĠCo herence : ĠA Ġplausible Ġrecipe Ġshould Ġpossess Ġa Ġcoherent Ġstep Ġorder , Ġand Ġwe Ġevaluate Ġthis Ġvia Ġa Ġmetric Ġfor Ġrecipe - level Ġco herence . ĠWe Ġuse Ġthe Ġneural Ġscoring Ġmodel Ġfrom ĠB IB REF 33 Ġto Ġmeasure Ġrecipe - level Ġco herence Ġfor Ġeach Ġgenerated Ġrecipe . ĠEach Ġrecipe Ġstep Ġis Ġencoded Ġby ĠB ERT ĠB IB REF 34 . ĠOur Ġscoring Ġmodel Ġis Ġa ĠGR U Ġnetwork Ġthat Ġlearns Ġthe Ġoverall Ġrecipe Ġstep Ġordering Ġstructure Ġby Ġminimizing Ġthe Ġcos ine Ġsimilarity Ġof Ġrecipe Ġstep Ġhidden Ġrepresentations Ġpresented Ġin Ġthe Ġcorrect Ġand Ġreverse Ġorders . ĠOnce Ġpret rained , Ġour Ġscorer Ġcalculates Ġthe Ġsimilarity Ġof Ġa Ġgenerated Ġrecipe Ġto Ġthe Ġforward Ġand Ġbackwards Ġordering Ġof Ġits Ġcorresponding Ġgold Ġlabel , Ġgiving Ġa Ġscore Ġequal Ġto Ġthe Ġdifference Ġbetween Ġthe Ġformer Ġand Ġlatter . ĠA Ġhigher Ġscore Ġindicates Ġbetter Ġstep Ġordering Ġ( with Ġa Ġmaximum Ġscore Ġof Ġ2 ). Ġtab : co herence met rics Ġshows Ġthat Ġour Ġpersonalized Ġmodels Ġachieve Ġaverage Ġrecipe - level Ġco herence Ġscores Ġof Ġ1 . 78 - 1 . 82 , Ġsurpass ing Ġthe Ġbaseline Ġat Ġ1 . 77 . </s> Recipe ĠStep ĠEnt ail ment : ĠLocal Ġco herence Ġis Ġalso Ġcrucial Ġto Ġa Ġuser Ġfollowing Ġa Ġrecipe : Ġit Ġis Ġcrucial Ġthat Ġsubsequent Ġsteps Ġare Ġlogically Ġconsistent Ġwith Ġprior Ġones . ĠWe Ġmodel Ġlocal Ġco herence Ġas Ġan Ġentail ment Ġtask : Ġpredicting Ġthe Ġlikelihood Ġthat Ġa Ġrecipe Ġstep Ġfollows Ġthe Ġpreceding . ĠWe Ġsample Ġseveral Ġconsecutive Ġ( positive ) Ġand Ġnon - con secut ive Ġ( negative ) Ġpairs Ġof Ġsteps Ġfrom Ġeach Ġrecipe . ĠWe Ġtrain Ġa ĠB ERT ĠB IB REF 34 Ġmodel Ġto Ġpredict Ġthe Ġentail ment Ġscore Ġof Ġa Ġpair Ġof Ġsteps Ġseparated Ġby Ġa Ġ[ SE P ] Ġtoken , Ġusing Ġthe Ġfinal Ġrepresentation Ġof Ġthe Ġ[ CL S ] Ġtoken . ĠThe Ġstep Ġentail ment Ġscore Ġis Ġcomputed Ġas Ġthe Ġaverage Ġof Ġscores Ġfor Ġeach Ġset Ġof Ġconsecutive Ġsteps Ġin Ġeach Ġrecipe , Ġaveraged Ġover Ġevery Ġgenerated Ġrecipe Ġfor Ġa Ġmodel , Ġas Ġshown Ġin Ġtab : co herence met rics . </s> Human ĠEvaluation : ĠWe Ġpresented Ġ310 Ġpairs Ġof Ġrecipes Ġfor Ġpair wise Ġcomparison ĠB IB REF 8 Ġ( details Ġin Ġappendix ) Ġbetween Ġbaseline Ġand Ġeach Ġpersonalized Ġmodel , Ġwith Ġresults Ġshown Ġin Ġtab : met rics ont est . ĠOn Ġaverage , Ġhuman Ġeval u ators Ġpreferred Ġpersonalized Ġmodel Ġoutputs Ġto Ġbaseline Ġ63 % Ġof Ġthe Ġtime , Ġconfirming Ġthat Ġpersonalized Ġattention Ġimproves Ġthe Ġsemantic Ġplaus ibility Ġof Ġgenerated Ġrecipes . ĠWe Ġalso Ġperformed Ġa Ġsmall - scale Ġhuman Ġco herence Ġsurvey Ġover Ġ90 Ġrecipes , Ġin Ġwhich Ġ60 % Ġof Ġusers Ġfound Ġrecipes Ġgenerated Ġby Ġpersonalized Ġmodels Ġto Ġbe Ġmore Ġcoherent Ġand Ġpreferable Ġto Ġthose Ġgenerated Ġby Ġbaseline Ġmodels . </s> Conclusion </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġnovel Ġtask : Ġto Ġgenerate Ġpersonalized Ġrecipes Ġfrom Ġincomplete Ġinput Ġspecifications Ġand Ġuser Ġhistories . ĠOn Ġa Ġlarge Ġnovel Ġdataset Ġof Ġ180 K Ġrecipes Ġand Ġ700 K Ġreviews , Ġwe Ġshow Ġthat Ġour Ġpersonalized Ġgener ative Ġmodels Ġcan Ġgenerate Ġplausible , Ġpersonalized , Ġand Ġcoherent Ġrecipes Ġpreferred Ġby Ġhuman Ġeval u ators Ġfor Ġconsumption . ĠWe Ġalso Ġintroduce Ġa Ġset Ġof Ġautomatic Ġco herence Ġmeasures Ġfor Ġinstructional Ġtexts Ġas Ġwell Ġas Ġpersonal ization Ġmetrics Ġto Ġsupport Ġour Ġclaims . ĠOur Ġfuture Ġwork Ġincludes Ġgenerating Ġstructured Ġrepresentations Ġof Ġrecipes Ġto Ġhandle Ġingredient Ġproperties , Ġas Ġwell Ġas Ġaccounting Ġfor Ġreferences Ġto Ġcollections Ġof Ġingredients Ġ( e . g . ĠâĢ ľ dry Ġmix "). </s> Acknowled gements . ĠThis Ġwork Ġis Ġpartly Ġsupported Ġby ĠNS F Ġ# 17 500 63 . ĠWe Ġthank Ġall Ġreviewers Ġfor Ġtheir Ġconstructive Ġsuggestions , Ġas Ġwell Ġas ĠRei ĠM ., ĠSu joy ĠP ., ĠAlicia ĠL ., ĠEric ĠH ., ĠTim ĠS ., ĠKathy ĠC ., ĠAllen ĠC ., Ġand ĠMic ah ĠI . Ġfor Ġtheir Ġfeedback . </s> App endix Ġ: :: ĠFood . com : ĠDat as et ĠDetails </s> Our Ġraw Ġdata Ġconsists Ġof Ġ270 K Ġrecipes Ġand Ġ1 . 4 M Ġuser - reci pe Ġinteractions Ġ( review s ) Ġscrap ed Ġfrom ĠFood . com , Ġcovering Ġa Ġperiod Ġof Ġ18 Ġyears Ġ( January Ġ2000 Ġto ĠDecember Ġ2018 ). ĠSee Ġtab : int - stats Ġfor Ġdataset Ġsummary Ġstatistics , Ġand Ġtab : sample g k Ġfor Ġsample Ġinformation Ġabout Ġone Ġuser - reci pe Ġinteraction Ġand Ġthe Ġrecipe Ġinvolved . </s> App endix Ġ: :: ĠGener ated ĠExamples </s> See Ġtab : sample ch x Ġfor Ġa Ġsample Ġrecipe Ġfor Ġchicken Ġchili Ġand Ġtab : sample w affle Ġfor Ġa Ġsample Ġrecipe Ġfor Ġsweet Ġw affles . </s> Human ĠEvaluation </s> We Ġprepared Ġa Ġset Ġof Ġ15 Ġpair wise Ġcomparisons Ġper Ġevaluation Ġsession , Ġand Ġcollected Ġ9 30 Ġpair wise Ġevaluations Ġ( 310 Ġper Ġpersonalized Ġmodel ) Ġover Ġ62 Ġsessions . ĠFor Ġeach Ġpair , Ġusers Ġwere Ġgiven Ġa Ġpartial Ġrecipe Ġspecification Ġ( name Ġand Ġ3 - 5 Ġkey Ġingredients ), Ġas Ġwell Ġas Ġtwo Ġgenerated Ġrecipes Ġlabeled Ġ` A ' Ġand Ġ` B '. ĠOne Ġrecipe Ġis Ġgenerated Ġfrom Ġour Ġbaseline Ġenc oder - dec oder Ġmodel Ġand Ġone Ġrecipe Ġis Ġgenerated Ġby Ġone Ġof Ġour Ġthree Ġpersonalized Ġmodels Ġ( Prior ĠTech , ĠPrior ĠName , ĠPrior ĠRecipe ). ĠThe Ġorder Ġof Ġrecipe Ġpresentation Ġ( A / B ) Ġis Ġrandomly Ġselected Ġfor Ġeach Ġquestion . ĠA Ġscreenshot Ġof Ġthe Ġuser Ġevaluation Ġinterface Ġis Ġgiven Ġin Ġfig : ex eval . ĠWe Ġask Ġthe Ġuser Ġto Ġindicate Ġwhich Ġrecipe Ġthey Ġfind Ġmore Ġcoherent , Ġand Ġwhich Ġrecipe Ġbest Ġaccompl ishes Ġthe Ġgoal Ġindicated Ġby Ġthe Ġrecipe Ġname . ĠA Ġscreenshot Ġof Ġthis Ġsurvey Ġinterface Ġis Ġgiven Ġin Ġfig : ex eval 2 .
Introduction In the kitchen, we increasingly rely on instructions from cooking websites: recipes. A cook with a predilection for Asian cuisine may wish to prepare chicken curry, but may not know all necessary ingredients apart from a few basics. These users with limited knowledge cannot rely on existing recipe generation approaches that focus on creating coherent recipes given all ingredients and a recipe name BIBREF0. Such models do not address issues of personal preference (e.g. culinary tastes, garnish choices) and incomplete recipe details. We propose to approach both problems via personalized generation of plausible, user-specific recipes using user preferences extracted from previously consumed recipes. Our work combines two important tasks from natural language processing and recommender systems: data-to-text generation BIBREF1 and personalized recommendation BIBREF2. Our model takes as user input the name of a specific dish, a few key ingredients, and a calorie level. We pass these loose input specifications to an encoder-decoder framework and attend on user profiles—learned latent representations of recipes previously consumed by a user—to generate a recipe personalized to the user's tastes. We fuse these `user-aware' representations with decoder output in an attention fusion layer to jointly determine text generation. Quantitative (perplexity, user-ranking) and qualitative analysis on user-aware model outputs confirm that personalization indeed assists in generating plausible recipes from incomplete ingredients. While personalized text generation has seen success in conveying user writing styles in the product review BIBREF3, BIBREF4 and dialogue BIBREF5 spaces, we are the first to consider it for the problem of recipe generation, where output quality is heavily dependent on the content of the instructions—such as ingredients and cooking techniques. To summarize, our main contributions are as follows: We explore a new task of generating plausible and personalized recipes from incomplete input specifications by leveraging historical user preferences; We release a new dataset of 180K+ recipes and 700K+ user reviews for this task; We introduce new evaluation strategies for generation quality in instructional texts, centering on quantitative measures of coherence. We also show qualitatively and quantitatively that personalized models generate high-quality and specific recipes that align with historical user preferences. Related Work Large-scale transformer-based language models have shown surprising expressivity and fluency in creative and conditional long-text generation BIBREF6, BIBREF7. Recent works have proposed hierarchical methods that condition on narrative frameworks to generate internally consistent long texts BIBREF8, BIBREF9, BIBREF10. Here, we generate procedurally structured recipes instead of free-form narratives. Recipe generation belongs to the field of data-to-text natural language generation BIBREF1, which sees other applications in automated journalism BIBREF11, question-answering BIBREF12, and abstractive summarization BIBREF13, among others. BIBREF14, BIBREF15 model recipes as a structured collection of ingredient entities acted upon by cooking actions. BIBREF0 imposes a `checklist' attention constraint emphasizing hitherto unused ingredients during generation. BIBREF16 attend over explicit ingredient references in the prior recipe step. Similar hierarchical approaches that infer a full ingredient list to constrain generation will not help personalize recipes, and would be infeasible in our setting due to the potentially unconstrained number of ingredients (from a space of 10K+) in a recipe. We instead learn historical preferences to guide full recipe generation. A recent line of work has explored user- and item-dependent aspect-aware review generation BIBREF3, BIBREF4. This work is related to ours in that it combines contextual language generation with personalization. Here, we attend over historical user preferences from previously consumed recipes to generate recipe content, rather than writing styles. Approach Our model's input specification consists of: the recipe name as a sequence of tokens, a partial list of ingredients, and a caloric level (high, medium, low). It outputs the recipe instructions as a token sequence: $\mathcal {W}_r=\lbrace w_{r,0}, \dots , w_{r,T}\rbrace $ for a recipe $r$ of length $T$. To personalize output, we use historical recipe interactions of a user $u \in \mathcal {U}$. Encoder: Our encoder has three embedding layers: vocabulary embedding $\mathcal {V}$, ingredient embedding $\mathcal {I}$, and caloric-level embedding $\mathcal {C}$. Each token in the (length $L_n$) recipe name is embedded via $\mathcal {V}$; the embedded token sequence is passed to a two-layered bidirectional GRU (BiGRU) BIBREF17, which outputs hidden states for names $\lbrace \mathbf {n}_{\text{enc},j} \in \mathbb {R}^{2d_h}\rbrace $, with hidden size $d_h$. Similarly each of the $L_i$ input ingredients is embedded via $\mathcal {I}$, and the embedded ingredient sequence is passed to another two-layered BiGRU to output ingredient hidden states as $\lbrace \mathbf {i}_{\text{enc},j} \in \mathbb {R}^{2d_h}\rbrace $. The caloric level is embedded via $\mathcal {C}$ and passed through a projection layer with weights $W_c$ to generate calorie hidden representation $\mathbf {c}_{\text{enc}} \in \mathbb {R}^{2d_h}$. Ingredient Attention: We apply attention BIBREF18 over the encoded ingredients to use encoder outputs at each decoding time step. We define an attention-score function $\alpha $ with key $K$ and query $Q$: with trainable weights $W_{\alpha }$, bias $\mathbf {b}_{\alpha }$, and normalization term $Z$. At decoding time $t$, we calculate the ingredient context $\mathbf {a}_{t}^{i} \in \mathbb {R}^{d_h}$ as: Decoder: The decoder is a two-layer GRU with hidden state $h_t$ conditioned on previous hidden state $h_{t-1}$ and input token $w_{r, t}$ from the original recipe text. We project the concatenated encoder outputs as the initial decoder hidden state: To bias generation toward user preferences, we attend over a user's previously reviewed recipes to jointly determine the final output token distribution. We consider two different schemes to model preferences from user histories: (1) recipe interactions, and (2) techniques seen therein (defined in data). BIBREF19, BIBREF20, BIBREF21 explore similar schemes for personalized recommendation. Prior Recipe Attention: We obtain the set of prior recipes for a user $u$: $R^+_u$, where each recipe can be represented by an embedding from a recipe embedding layer $\mathcal {R}$ or an average of the name tokens embedded by $\mathcal {V}$. We attend over the $k$-most recent prior recipes, $R^{k+}_u$, to account for temporal drift of user preferences BIBREF22. These embeddings are used in the `Prior Recipe' and `Prior Name' models, respectively. Given a recipe representation $\mathbf {r} \in \mathbb {R}^{d_r}$ (where $d_r$ is recipe- or vocabulary-embedding size depending on the recipe representation) the prior recipe attention context $\mathbf {a}_{t}^{r_u}$ is calculated as Prior Technique Attention: We calculate prior technique preference (used in the `Prior Tech` model) by normalizing co-occurrence between users and techniques seen in $R^+_u$, to obtain a preference vector $\rho _{u}$. Each technique $x$ is embedded via a technique embedding layer $\mathcal {X}$ to $\mathbf {x}\in \mathbb {R}^{d_x}$. Prior technique attention is calculated as where, inspired by copy mechanisms BIBREF23, BIBREF24, we add $\rho _{u,x}$ for technique $x$ to emphasize the attention by the user's prior technique preference. Attention Fusion Layer: We fuse all contexts calculated at time $t$, concatenating them with decoder GRU output and previous token embedding: We then calculate the token probability: and maximize the log-likelihood of the generated sequence conditioned on input specifications and user preferences. fig:ex shows a case where the Prior Name model attends strongly on previously consumed savory recipes to suggest the usage of an additional ingredient (`cilantro'). Recipe Dataset: Food.com We collect a novel dataset of 230K+ recipe texts and 1M+ user interactions (reviews) over 18 years (2000-2018) from Food.com. Here, we restrict to recipes with at least 3 steps, and at least 4 and no more than 20 ingredients. We discard users with fewer than 4 reviews, giving 180K+ recipes and 700K+ reviews, with splits as in tab:recipeixnstats. Our model must learn to generate from a diverse recipe space: in our training data, the average recipe length is 117 tokens with a maximum of 256. There are 13K unique ingredients across all recipes. Rare words dominate the vocabulary: 95% of words appear $<$100 times, accounting for only 1.65% of all word usage. As such, we perform Byte-Pair Encoding (BPE) tokenization BIBREF25, BIBREF26, giving a training vocabulary of 15K tokens across 19M total mentions. User profiles are similarly diverse: 50% of users have consumed $\le $6 recipes, while 10% of users have consumed $>$45 recipes. We order reviews by timestamp, keeping the most recent review for each user as the test set, the second most recent for validation, and the remainder for training (sequential leave-one-out evaluation BIBREF27). We evaluate only on recipes not in the training set. We manually construct a list of 58 cooking techniques from 384 cooking actions collected by BIBREF15; the most common techniques (bake, combine, pour, boil) account for 36.5% of technique mentions. We approximate technique adherence via string match between the recipe text and technique list. Experiments and Results For training and evaluation, we provide our model with the first 3-5 ingredients listed in each recipe. We decode recipe text via top-$k$ sampling BIBREF7, finding $k=3$ to produce satisfactory results. We use a hidden size $d_h=256$ for both the encoder and decoder. Embedding dimensions for vocabulary, ingredient, recipe, techniques, and caloric level are 300, 10, 50, 50, and 5 (respectively). For prior recipe attention, we set $k=20$, the 80th %-ile for the number of user interactions. We use the Adam optimizer BIBREF28 with a learning rate of $10^{-3}$, annealed with a decay rate of 0.9 BIBREF29. We also use teacher-forcing BIBREF30 in all training epochs. In this work, we investigate how leveraging historical user preferences can improve generation quality over strong baselines in our setting. We compare our personalized models against two baselines. The first is a name-based Nearest-Neighbor model (NN). We initially adapted the Neural Checklist Model of BIBREF0 as a baseline; however, we ultimately use a simple Encoder-Decoder baseline with ingredient attention (Enc-Dec), which provides comparable performance and lower complexity. All personalized models outperform baseline in BPE perplexity (tab:metricsontest) with Prior Name performing the best. While our models exhibit comparable performance to baseline in BLEU-1/4 and ROUGE-L, we generate more diverse (Distinct-1/2: percentage of distinct unigrams and bigrams) and acceptable recipes. BLEU and ROUGE are not the most appropriate metrics for generation quality. A `correct' recipe can be written in many ways with the same main entities (ingredients). As BLEU-1/4 capture structural information via n-gram matching, they are not correlated with subjective recipe quality. This mirrors observations from BIBREF31, BIBREF8. We observe that personalized models make more diverse recipes than baseline. They thus perform better in BLEU-1 with more key entities (ingredient mentions) present, but worse in BLEU-4, as these recipes are written in a personalized way and deviate from gold on the phrasal level. Similarly, the `Prior Name' model generates more unigram-diverse recipes than other personalized models and obtains a correspondingly lower BLEU-1 score. Qualitative Analysis: We present sample outputs for a cocktail recipe in tab:samplerecipes, and additional recipes in the appendix. Generation quality progressively improves from generic baseline output to a blended cocktail produced by our best performing model. Models attending over prior recipes explicitly reference ingredients. The Prior Name model further suggests the addition of lemon and mint, which are reasonably associated with previously consumed recipes like coconut mousse and pork skewers. Personalization: To measure personalization, we evaluate how closely the generated text corresponds to a particular user profile. We compute the likelihood of generated recipes using identical input specifications but conditioned on ten different user profiles—one `gold' user who consumed the original recipe, and nine randomly generated user profiles. Following BIBREF8, we expect the highest likelihood for the recipe conditioned on the gold user. We measure user matching accuracy (UMA)—the proportion where the gold user is ranked highest—and Mean Reciprocal Rank (MRR) BIBREF32 of the gold user. All personalized models beat baselines in both metrics, showing our models personalize generated recipes to the given user profiles. The Prior Name model achieves the best UMA and MRR by a large margin, revealing that prior recipe names are strong signals for personalization. Moreover, the addition of attention mechanisms to capture these signals improves language modeling performance over a strong non-personalized baseline. Recipe Level Coherence: A plausible recipe should possess a coherent step order, and we evaluate this via a metric for recipe-level coherence. We use the neural scoring model from BIBREF33 to measure recipe-level coherence for each generated recipe. Each recipe step is encoded by BERT BIBREF34. Our scoring model is a GRU network that learns the overall recipe step ordering structure by minimizing the cosine similarity of recipe step hidden representations presented in the correct and reverse orders. Once pretrained, our scorer calculates the similarity of a generated recipe to the forward and backwards ordering of its corresponding gold label, giving a score equal to the difference between the former and latter. A higher score indicates better step ordering (with a maximum score of 2). tab:coherencemetrics shows that our personalized models achieve average recipe-level coherence scores of 1.78-1.82, surpassing the baseline at 1.77. Recipe Step Entailment: Local coherence is also crucial to a user following a recipe: it is crucial that subsequent steps are logically consistent with prior ones. We model local coherence as an entailment task: predicting the likelihood that a recipe step follows the preceding. We sample several consecutive (positive) and non-consecutive (negative) pairs of steps from each recipe. We train a BERT BIBREF34 model to predict the entailment score of a pair of steps separated by a [SEP] token, using the final representation of the [CLS] token. The step entailment score is computed as the average of scores for each set of consecutive steps in each recipe, averaged over every generated recipe for a model, as shown in tab:coherencemetrics. Human Evaluation: We presented 310 pairs of recipes for pairwise comparison BIBREF8 (details in appendix) between baseline and each personalized model, with results shown in tab:metricsontest. On average, human evaluators preferred personalized model outputs to baseline 63% of the time, confirming that personalized attention improves the semantic plausibility of generated recipes. We also performed a small-scale human coherence survey over 90 recipes, in which 60% of users found recipes generated by personalized models to be more coherent and preferable to those generated by baseline models. Conclusion In this paper, we propose a novel task: to generate personalized recipes from incomplete input specifications and user histories. On a large novel dataset of 180K recipes and 700K reviews, we show that our personalized generative models can generate plausible, personalized, and coherent recipes preferred by human evaluators for consumption. We also introduce a set of automatic coherence measures for instructional texts as well as personalization metrics to support our claims. Our future work includes generating structured representations of recipes to handle ingredient properties, as well as accounting for references to collections of ingredients (e.g. “dry mix"). Acknowledgements. This work is partly supported by NSF #1750063. We thank all reviewers for their constructive suggestions, as well as Rei M., Sujoy P., Alicia L., Eric H., Tim S., Kathy C., Allen C., and Micah I. for their feedback. Appendix ::: Food.com: Dataset Details Our raw data consists of 270K recipes and 1.4M user-recipe interactions (reviews) scraped from Food.com, covering a period of 18 years (January 2000 to December 2018). See tab:int-stats for dataset summary statistics, and tab:samplegk for sample information about one user-recipe interaction and the recipe involved. Appendix ::: Generated Examples See tab:samplechx for a sample recipe for chicken chili and tab:samplewaffle for a sample recipe for sweet waffles. Human Evaluation We prepared a set of 15 pairwise comparisons per evaluation session, and collected 930 pairwise evaluations (310 per personalized model) over 62 sessions. For each pair, users were given a partial recipe specification (name and 3-5 key ingredients), as well as two generated recipes labeled `A' and `B'. One recipe is generated from our baseline encoder-decoder model and one recipe is generated by one of our three personalized models (Prior Tech, Prior Name, Prior Recipe). The order of recipe presentation (A/B) is randomly selected for each question. A screenshot of the user evaluation interface is given in fig:exeval. We ask the user to indicate which recipe they find more coherent, and which recipe best accomplishes the goal indicated by the recipe name. A screenshot of this survey interface is given in fig:exeval2.
Where do they get the recipes from?
1bdc990c7e948724ab04e70867675a334fdd3051
Where Ġdo Ġthey Ġget Ġthe Ġrecipes Ġfrom ?
from ĠFood . com
<s> Where Ġdo Ġthey Ġget Ġthe Ġrecipes Ġfrom ? </s> Introduction </s> In Ġthe Ġkitchen , Ġwe Ġincreasingly Ġrely Ġon Ġinstructions Ġfrom Ġcooking Ġwebsites : Ġrecipes . ĠA Ġcook Ġwith Ġa Ġpred ile ction Ġfor ĠAsian Ġcuisine Ġmay Ġwish Ġto Ġprepare Ġchicken Ġcurry , Ġbut Ġmay Ġnot Ġknow Ġall Ġnecessary Ġingredients Ġapart Ġfrom Ġa Ġfew Ġbasics . ĠThese Ġusers Ġwith Ġlimited Ġknowledge Ġcannot Ġrely Ġon Ġexisting Ġrecipe Ġgeneration Ġapproaches Ġthat Ġfocus Ġon Ġcreating Ġcoherent Ġrecipes Ġgiven Ġall Ġingredients Ġand Ġa Ġrecipe Ġname ĠB IB REF 0 . ĠSuch Ġmodels Ġdo Ġnot Ġaddress Ġissues Ġof Ġpersonal Ġpreference Ġ( e . g . Ġculinary Ġtastes , Ġgarn ish Ġchoices ) Ġand Ġincomplete Ġrecipe Ġdetails . ĠWe Ġpropose Ġto Ġapproach Ġboth Ġproblems Ġvia Ġpersonalized Ġgeneration Ġof Ġplausible , Ġuser - specific Ġrecipes Ġusing Ġuser Ġpreferences Ġextracted Ġfrom Ġpreviously Ġconsumed Ġrecipes . </s> Our Ġwork Ġcombines Ġtwo Ġimportant Ġtasks Ġfrom Ġnatural Ġlanguage Ġprocessing Ġand Ġrecomm ender Ġsystems : Ġdata - to - text Ġgeneration ĠB IB REF 1 Ġand Ġpersonalized Ġrecommendation ĠB IB REF 2 . ĠOur Ġmodel Ġtakes Ġas Ġuser Ġinput Ġthe Ġname Ġof Ġa Ġspecific Ġdish , Ġa Ġfew Ġkey Ġingredients , Ġand Ġa Ġcalorie Ġlevel . ĠWe Ġpass Ġthese Ġloose Ġinput Ġspecifications Ġto Ġan Ġenc oder - dec oder Ġframework Ġand Ġattend Ġon Ġuser Ġprofiles âĢĶ learn ed Ġlatent Ġrepresentations Ġof Ġrecipes Ġpreviously Ġconsumed Ġby Ġa Ġuser âĢĶ to Ġgenerate Ġa Ġrecipe Ġpersonalized Ġto Ġthe Ġuser 's Ġtastes . ĠWe Ġfuse Ġthese Ġ` user - aware ' Ġrepresentations Ġwith Ġdec oder Ġoutput Ġin Ġan Ġattention Ġfusion Ġlayer Ġto Ġjointly Ġdetermine Ġtext Ġgeneration . ĠQuant itative Ġ( per plex ity , Ġuser - ranking ) Ġand Ġqualitative Ġanalysis Ġon Ġuser - aware Ġmodel Ġoutputs Ġconfirm Ġthat Ġpersonal ization Ġindeed Ġassists Ġin Ġgenerating Ġplausible Ġrecipes Ġfrom Ġincomplete Ġingredients . </s> While Ġpersonalized Ġtext Ġgeneration Ġhas Ġseen Ġsuccess Ġin Ġconve ying Ġuser Ġwriting Ġstyles Ġin Ġthe Ġproduct Ġreview ĠB IB REF 3 , ĠB IB REF 4 Ġand Ġdialogue ĠB IB REF 5 Ġspaces , Ġwe Ġare Ġthe Ġfirst Ġto Ġconsider Ġit Ġfor Ġthe Ġproblem Ġof Ġrecipe Ġgeneration , Ġwhere Ġoutput Ġquality Ġis Ġheavily Ġdependent Ġon Ġthe Ġcontent Ġof Ġthe Ġinstructions âĢĶ such Ġas Ġingredients Ġand Ġcooking Ġtechniques . </s> To Ġsummarize , Ġour Ġmain Ġcontributions Ġare Ġas Ġfollows : </s> We Ġexplore Ġa Ġnew Ġtask Ġof Ġgenerating Ġplausible Ġand Ġpersonalized Ġrecipes Ġfrom Ġincomplete Ġinput Ġspecifications Ġby Ġleveraging Ġhistorical Ġuser Ġpreferences ; </s> We Ġrelease Ġa Ġnew Ġdataset Ġof Ġ180 K + Ġrecipes Ġand Ġ700 K + Ġuser Ġreviews Ġfor Ġthis Ġtask ; </s> We Ġintroduce Ġnew Ġevaluation Ġstrategies Ġfor Ġgeneration Ġquality Ġin Ġinstructional Ġtexts , Ġcent ering Ġon Ġquantitative Ġmeasures Ġof Ġco herence . ĠWe Ġalso Ġshow Ġqual itatively Ġand Ġquant itatively Ġthat Ġpersonalized Ġmodels Ġgenerate Ġhigh - quality Ġand Ġspecific Ġrecipes Ġthat Ġalign Ġwith Ġhistorical Ġuser Ġpreferences . </s> Related ĠWork </s> Large - scale Ġtransformer - based Ġlanguage Ġmodels Ġhave Ġshown Ġsurprising Ġexpress ivity Ġand Ġflu ency Ġin Ġcreative Ġand Ġconditional Ġlong - text Ġgeneration ĠB IB REF 6 , ĠB IB REF 7 . ĠRecent Ġworks Ġhave Ġproposed Ġhierarchical Ġmethods Ġthat Ġcondition Ġon Ġnarrative Ġframeworks Ġto Ġgenerate Ġinternally Ġconsistent Ġlong Ġtexts ĠB IB REF 8 , ĠB IB REF 9 , ĠB IB REF 10 . ĠHere , Ġwe Ġgenerate Ġproced urally Ġstructured Ġrecipes Ġinstead Ġof Ġfree - form Ġnarratives . </s> Recipe Ġgeneration Ġbelongs Ġto Ġthe Ġfield Ġof Ġdata - to - text Ġnatural Ġlanguage Ġgeneration ĠB IB REF 1 , Ġwhich Ġsees Ġother Ġapplications Ġin Ġautomated Ġjournalism ĠB IB REF 11 , Ġquestion - ans w ering ĠB IB REF 12 , Ġand Ġabstract ive Ġsummar ization ĠB IB REF 13 , Ġamong Ġothers . ĠB IB REF 14 , ĠB IB REF 15 Ġmodel Ġrecipes Ġas Ġa Ġstructured Ġcollection Ġof Ġingredient Ġentities Ġacted Ġupon Ġby Ġcooking Ġactions . ĠB IB REF 0 Ġimposes Ġa Ġ` check list ' Ġattention Ġconstraint Ġemphasizing Ġhitherto Ġunused Ġingredients Ġduring Ġgeneration . ĠB IB REF 16 Ġattend Ġover Ġexplicit Ġingredient Ġreferences Ġin Ġthe Ġprior Ġrecipe Ġstep . ĠSimilar Ġhierarchical Ġapproaches Ġthat Ġinfer Ġa Ġfull Ġingredient Ġlist Ġto Ġconst rain Ġgeneration Ġwill Ġnot Ġhelp Ġpersonal ize Ġrecipes , Ġand Ġwould Ġbe Ġinf eas ible Ġin Ġour Ġsetting Ġdue Ġto Ġthe Ġpotentially Ġuncon str ained Ġnumber Ġof Ġingredients Ġ( from Ġa Ġspace Ġof Ġ10 K +) Ġin Ġa Ġrecipe . ĠWe Ġinstead Ġlearn Ġhistorical Ġpreferences Ġto Ġguide Ġfull Ġrecipe Ġgeneration . </s> A Ġrecent Ġline Ġof Ġwork Ġhas Ġexplored Ġuser - Ġand Ġitem - dependent Ġaspect - aware Ġreview Ġgeneration ĠB IB REF 3 , ĠB IB REF 4 . ĠThis Ġwork Ġis Ġrelated Ġto Ġours Ġin Ġthat Ġit Ġcombines Ġcontextual Ġlanguage Ġgeneration Ġwith Ġpersonal ization . ĠHere , Ġwe Ġattend Ġover Ġhistorical Ġuser Ġpreferences Ġfrom Ġpreviously Ġconsumed Ġrecipes Ġto Ġgenerate Ġrecipe Ġcontent , Ġrather Ġthan Ġwriting Ġstyles . </s> App roach </s> Our Ġmodel 's Ġinput Ġspecification Ġconsists Ġof : Ġthe Ġrecipe Ġname Ġas Ġa Ġsequence Ġof Ġtokens , Ġa Ġpartial Ġlist Ġof Ġingredients , Ġand Ġa Ġcaloric Ġlevel Ġ( high , Ġmedium , Ġlow ). ĠIt Ġoutputs Ġthe Ġrecipe Ġinstructions Ġas Ġa Ġtoken Ġsequence : Ġ$\ math cal Ġ{ W } _ r = \ l brace Ġw _{ r , 0 }, Ġ\ d ots Ġ, Ġw _{ r , T }\ r brace Ġ$ Ġfor Ġa Ġrecipe Ġ$ r $ Ġof Ġlength Ġ$ T $. ĠTo Ġpersonal ize Ġoutput , Ġwe Ġuse Ġhistorical Ġrecipe Ġinteractions Ġof Ġa Ġuser Ġ$ u Ġ\ in Ġ\ math cal Ġ{ U } $. </s> Enc oder : ĠOur Ġenc oder Ġhas Ġthree Ġembed ding Ġlayers : Ġvocabulary Ġembed ding Ġ$\ math cal Ġ{ V } $, Ġingredient Ġembed ding Ġ$\ math cal Ġ{ I } $, Ġand Ġcaloric - level Ġembed ding Ġ$\ math cal Ġ{ C } $. ĠEach Ġtoken Ġin Ġthe Ġ( length Ġ$ L _ n $ ) Ġrecipe Ġname Ġis Ġembedded Ġvia Ġ$\ math cal Ġ{ V } $ ; Ġthe Ġembedded Ġtoken Ġsequence Ġis Ġpassed Ġto Ġa Ġtwo - lay ered Ġbid irection al ĠGR U Ġ( Bi GR U ) ĠB IB REF 17 , Ġwhich Ġoutputs Ġhidden Ġstates Ġfor Ġnames Ġ$\ l brace Ġ\ math bf Ġ{ n } _{ \ text { enc }, j } Ġ\ in Ġ\ math bb Ġ{ R } ^{ 2 d _ h }\ r brace Ġ$ , Ġwith Ġhidden Ġsize Ġ$ d _ h $. ĠSimilarly Ġeach Ġof Ġthe Ġ$ L _ i $ Ġinput Ġingredients Ġis Ġembedded Ġvia Ġ$\ math cal Ġ{ I } $, Ġand Ġthe Ġembedded Ġingredient Ġsequence Ġis Ġpassed Ġto Ġanother Ġtwo - lay ered ĠBi GR U Ġto Ġoutput Ġingredient Ġhidden Ġstates Ġas Ġ$\ l brace Ġ\ math bf Ġ{ i } _{ \ text { enc }, j } Ġ\ in Ġ\ math bb Ġ{ R } ^{ 2 d _ h }\ r brace Ġ$ . ĠThe Ġcaloric Ġlevel Ġis Ġembedded Ġvia Ġ$\ math cal Ġ{ C } $ Ġand Ġpassed Ġthrough Ġa Ġprojection Ġlayer Ġwith Ġweights Ġ$ W _ c $ Ġto Ġgenerate Ġcalorie Ġhidden Ġrepresentation Ġ$\ math bf Ġ{ c } _{ \ text { enc }} Ġ\ in Ġ\ math bb Ġ{ R } ^{ 2 d _ h } $. </s> Ing red ient ĠAttention : ĠWe Ġapply Ġattention ĠB IB REF 18 Ġover Ġthe Ġencoded Ġingredients Ġto Ġuse Ġenc oder Ġoutputs Ġat Ġeach Ġdecoding Ġtime Ġstep . ĠWe Ġdefine Ġan Ġattention - score Ġfunction Ġ$\ alpha Ġ$ Ġwith Ġkey Ġ$ K $ Ġand Ġquery Ġ$ Q $ : </s> with Ġtrain able Ġweights Ġ$ W _{ \ alpha Ġ} $, Ġbias Ġ$\ math bf Ġ{ b } _{ \ alpha Ġ} $, Ġand Ġnormal ization Ġterm Ġ$ Z $. ĠAt Ġdecoding Ġtime Ġ$ t $, Ġwe Ġcalculate Ġthe Ġingredient Ġcontext Ġ$\ math bf Ġ{ a } _{ t } ^{ i } Ġ\ in Ġ\ math bb Ġ{ R } ^{ d _ h } $ Ġas : </s> Dec oder : ĠThe Ġdec oder Ġis Ġa Ġtwo - layer ĠGR U Ġwith Ġhidden Ġstate Ġ$ h _ t $ Ġconditioned Ġon Ġprevious Ġhidden Ġstate Ġ$ h _{ t - 1 } $ Ġand Ġinput Ġtoken Ġ$ w _{ r , Ġt } $ Ġfrom Ġthe Ġoriginal Ġrecipe Ġtext . ĠWe Ġproject Ġthe Ġconc aten ated Ġenc oder Ġoutputs Ġas Ġthe Ġinitial Ġdec oder Ġhidden Ġstate : </s> To Ġbias Ġgeneration Ġtoward Ġuser Ġpreferences , Ġwe Ġattend Ġover Ġa Ġuser 's Ġpreviously Ġreviewed Ġrecipes Ġto Ġjointly Ġdetermine Ġthe Ġfinal Ġoutput Ġtoken Ġdistribution . ĠWe Ġconsider Ġtwo Ġdifferent Ġschemes Ġto Ġmodel Ġpreferences Ġfrom Ġuser Ġhistories : Ġ( 1 ) Ġrecipe Ġinteractions , Ġand Ġ( 2 ) Ġtechniques Ġseen Ġtherein Ġ( defined Ġin Ġdata ). ĠB IB REF 19 , ĠB IB REF 20 , ĠB IB REF 21 Ġexplore Ġsimilar Ġschemes Ġfor Ġpersonalized Ġrecommendation . </s> Prior ĠRecipe ĠAttention : ĠWe Ġobtain Ġthe Ġset Ġof Ġprior Ġrecipes Ġfor Ġa Ġuser Ġ$ u $ : Ġ$ R ^ + _ u $, Ġwhere Ġeach Ġrecipe Ġcan Ġbe Ġrepresented Ġby Ġan Ġembed ding Ġfrom Ġa Ġrecipe Ġembed ding Ġlayer Ġ$\ math cal Ġ{ R } $ Ġor Ġan Ġaverage Ġof Ġthe Ġname Ġtokens Ġembedded Ġby Ġ$\ math cal Ġ{ V } $. ĠWe Ġattend Ġover Ġthe Ġ$ k $ - most Ġrecent Ġprior Ġrecipes , Ġ$ R ^{ k + } _ u $, Ġto Ġaccount Ġfor Ġtemporal Ġdrift Ġof Ġuser Ġpreferences ĠB IB REF 22 . ĠThese Ġembed d ings Ġare Ġused Ġin Ġthe Ġ` Prior ĠRecipe ' Ġand Ġ` Prior ĠName ' Ġmodels , Ġrespectively . </s> Given Ġa Ġrecipe Ġrepresentation Ġ$\ math bf Ġ{ r } Ġ\ in Ġ\ math bb Ġ{ R } ^{ d _ r } $ Ġ( where Ġ$ d _ r $ Ġis Ġrecipe - Ġor Ġvocabulary - embed ding Ġsize Ġdepending Ġon Ġthe Ġrecipe Ġrepresentation ) Ġthe Ġprior Ġrecipe Ġattention Ġcontext Ġ$\ math bf Ġ{ a } _{ t } ^{ r _ u } $ Ġis Ġcalculated Ġas </s> Prior ĠTechnique ĠAttention : ĠWe Ġcalculate Ġprior Ġtechnique Ġpreference Ġ( used Ġin Ġthe Ġ` Prior ĠTech ` Ġmodel ) Ġby Ġnormal izing Ġco - occ urrence Ġbetween Ġusers Ġand Ġtechniques Ġseen Ġin Ġ$ R ^ + _ u $, Ġto Ġobtain Ġa Ġpreference Ġvector Ġ$\ r ho Ġ_ { u } $. ĠEach Ġtechnique Ġ$ x $ Ġis Ġembedded Ġvia Ġa Ġtechnique Ġembed ding Ġlayer Ġ$\ math cal Ġ{ X } $ Ġto Ġ$\ math bf Ġ{ x }\ in Ġ\ math bb Ġ{ R } ^{ d _ x } $. ĠPrior Ġtechnique Ġattention Ġis Ġcalculated Ġas </s> where , Ġinspired Ġby Ġcopy Ġmechanisms ĠB IB REF 23 , ĠB IB REF 24 , Ġwe Ġadd Ġ$\ r ho Ġ_ { u , x } $ Ġfor Ġtechnique Ġ$ x $ Ġto Ġemphasize Ġthe Ġattention Ġby Ġthe Ġuser 's Ġprior Ġtechnique Ġpreference . </s> Att ention ĠFusion ĠLayer : ĠWe Ġfuse Ġall Ġcontexts Ġcalculated Ġat Ġtime Ġ$ t $, Ġconc aten ating Ġthem Ġwith Ġdec oder ĠGR U Ġoutput Ġand Ġprevious Ġtoken Ġembed ding : </s> We Ġthen Ġcalculate Ġthe Ġtoken Ġprobability : </s> and Ġmaximize Ġthe Ġlog - like lihood Ġof Ġthe Ġgenerated Ġsequence Ġconditioned Ġon Ġinput Ġspecifications Ġand Ġuser Ġpreferences . Ġfig : ex Ġshows Ġa Ġcase Ġwhere Ġthe ĠPrior ĠName Ġmodel Ġattends Ġstrongly Ġon Ġpreviously Ġconsumed Ġsav ory Ġrecipes Ġto Ġsuggest Ġthe Ġusage Ġof Ġan Ġadditional Ġingredient Ġ( ` cil ant ro '). </s> Recipe ĠDat as et : ĠFood . com </s> We Ġcollect Ġa Ġnovel Ġdataset Ġof Ġ230 K + Ġrecipe Ġtexts Ġand Ġ1 M + Ġuser Ġinteractions Ġ( review s ) Ġover Ġ18 Ġyears Ġ( 2000 - 2018 ) Ġfrom ĠFood . com . ĠHere , Ġwe Ġrestrict Ġto Ġrecipes Ġwith Ġat Ġleast Ġ3 Ġsteps , Ġand Ġat Ġleast Ġ4 Ġand Ġno Ġmore Ġthan Ġ20 Ġingredients . ĠWe Ġdiscard Ġusers Ġwith Ġfewer Ġthan Ġ4 Ġreviews , Ġgiving Ġ180 K + Ġrecipes Ġand Ġ700 K + Ġreviews , Ġwith Ġsplits Ġas Ġin Ġtab : reci pe ix n stats . </s> Our Ġmodel Ġmust Ġlearn Ġto Ġgenerate Ġfrom Ġa Ġdiverse Ġrecipe Ġspace : Ġin Ġour Ġtraining Ġdata , Ġthe Ġaverage Ġrecipe Ġlength Ġis Ġ117 Ġtokens Ġwith Ġa Ġmaximum Ġof Ġ256 . ĠThere Ġare Ġ13 K Ġunique Ġingredients Ġacross Ġall Ġrecipes . ĠRare Ġwords Ġdominate Ġthe Ġvocabulary : Ġ95 % Ġof Ġwords Ġappear Ġ$ < $ 100 Ġtimes , Ġaccounting Ġfor Ġonly Ġ1 . 65 % Ġof Ġall Ġword Ġusage . ĠAs Ġsuch , Ġwe Ġperform ĠByte - P air ĠEnc oding Ġ( B PE ) Ġtoken ization ĠB IB REF 25 , ĠB IB REF 26 , Ġgiving Ġa Ġtraining Ġvocabulary Ġof Ġ15 K Ġtokens Ġacross Ġ19 M Ġtotal Ġmentions . ĠUser Ġprofiles Ġare Ġsimilarly Ġdiverse : Ġ50 % Ġof Ġusers Ġhave Ġconsumed Ġ$\ le Ġ$ 6 Ġrecipes , Ġwhile Ġ10 % Ġof Ġusers Ġhave Ġconsumed Ġ$ > $ 45 Ġrecipes . </s> We Ġorder Ġreviews Ġby Ġtimestamp , Ġkeeping Ġthe Ġmost Ġrecent Ġreview Ġfor Ġeach Ġuser Ġas Ġthe Ġtest Ġset , Ġthe Ġsecond Ġmost Ġrecent Ġfor Ġvalidation , Ġand Ġthe Ġremainder Ġfor Ġtraining Ġ( sequ ential Ġleave - one - out Ġevaluation ĠB IB REF 27 ). ĠWe Ġevaluate Ġonly Ġon Ġrecipes Ġnot Ġin Ġthe Ġtraining Ġset . </s> We Ġmanually Ġconstruct Ġa Ġlist Ġof Ġ58 Ġcooking Ġtechniques Ġfrom Ġ384 Ġcooking Ġactions Ġcollected Ġby ĠB IB REF 15 ; Ġthe Ġmost Ġcommon Ġtechniques Ġ( b ake , Ġcombine , Ġpour , Ġboil ) Ġaccount Ġfor Ġ36 . 5 % Ġof Ġtechnique Ġmentions . ĠWe Ġapproximate Ġtechnique Ġadherence Ġvia Ġstring Ġmatch Ġbetween Ġthe Ġrecipe Ġtext Ġand Ġtechnique Ġlist . </s> Exper iments Ġand ĠResults </s> For Ġtraining Ġand Ġevaluation , Ġwe Ġprovide Ġour Ġmodel Ġwith Ġthe Ġfirst Ġ3 - 5 Ġingredients Ġlisted Ġin Ġeach Ġrecipe . ĠWe Ġdecode Ġrecipe Ġtext Ġvia Ġtop -$ k $ Ġsampling ĠB IB REF 7 , Ġfinding Ġ$ k = 3 $ Ġto Ġproduce Ġsatisfactory Ġresults . ĠWe Ġuse Ġa Ġhidden Ġsize Ġ$ d _ h = 256 $ Ġfor Ġboth Ġthe Ġenc oder Ġand Ġdec oder . ĠEmb edd ing Ġdimensions Ġfor Ġvocabulary , Ġingredient , Ġrecipe , Ġtechniques , Ġand Ġcaloric Ġlevel Ġare Ġ300 , Ġ10 , Ġ50 , Ġ50 , Ġand Ġ5 Ġ( respect ively ). ĠFor Ġprior Ġrecipe Ġattention , Ġwe Ġset Ġ$ k = 20 $, Ġthe Ġ80 th Ġ% - ile Ġfor Ġthe Ġnumber Ġof Ġuser Ġinteractions . ĠWe Ġuse Ġthe ĠAdam Ġoptim izer ĠB IB REF 28 Ġwith Ġa Ġlearning Ġrate Ġof Ġ$ 10 ^{ - 3 } $, Ġan ne aled Ġwith Ġa Ġdecay Ġrate Ġof Ġ0 . 9 ĠB IB REF 29 . ĠWe Ġalso Ġuse Ġteacher - forcing ĠB IB REF 30 Ġin Ġall Ġtraining Ġepoch s . </s> In Ġthis Ġwork , Ġwe Ġinvestigate Ġhow Ġleveraging Ġhistorical Ġuser Ġpreferences Ġcan Ġimprove Ġgeneration Ġquality Ġover Ġstrong Ġbas elines Ġin Ġour Ġsetting . ĠWe Ġcompare Ġour Ġpersonalized Ġmodels Ġagainst Ġtwo Ġbas elines . ĠThe Ġfirst Ġis Ġa Ġname - based ĠNe arest - Neigh bor Ġmodel Ġ( NN ). ĠWe Ġinitially Ġadapted Ġthe ĠNeural ĠCheck list ĠModel Ġof ĠB IB REF 0 Ġas Ġa Ġbaseline ; Ġhowever , Ġwe Ġultimately Ġuse Ġa Ġsimple ĠEnc oder - Dec oder Ġbaseline Ġwith Ġingredient Ġattention Ġ( Enc - Dec ), Ġwhich Ġprovides Ġcomparable Ġperformance Ġand Ġlower Ġcomplexity . ĠAll Ġpersonalized Ġmodels Ġoutper form Ġbaseline Ġin ĠB PE Ġperplex ity Ġ( tab : met rics ont est ) Ġwith ĠPrior ĠName Ġperforming Ġthe Ġbest . ĠWhile Ġour Ġmodels Ġexhibit Ġcomparable Ġperformance Ġto Ġbaseline Ġin ĠB LE U - 1 / 4 Ġand ĠR OU GE - L , Ġwe Ġgenerate Ġmore Ġdiverse Ġ( Dist inct - 1 / 2 : Ġpercentage Ġof Ġdistinct Ġun ig rams Ġand Ġbig rams ) Ġand Ġacceptable Ġrecipes . ĠB LE U Ġand ĠR OU GE Ġare Ġnot Ġthe Ġmost Ġappropriate Ġmetrics Ġfor Ġgeneration Ġquality . ĠA Ġ` correct ' Ġrecipe Ġcan Ġbe Ġwritten Ġin Ġmany Ġways Ġwith Ġthe Ġsame Ġmain Ġentities Ġ( ing redients ). ĠAs ĠB LE U - 1 / 4 Ġcapture Ġstructural Ġinformation Ġvia Ġn - gram Ġmatching , Ġthey Ġare Ġnot Ġcorrelated Ġwith Ġsubjective Ġrecipe Ġquality . ĠThis Ġmirrors Ġobservations Ġfrom ĠB IB REF 31 , ĠB IB REF 8 . </s> We Ġobserve Ġthat Ġpersonalized Ġmodels Ġmake Ġmore Ġdiverse Ġrecipes Ġthan Ġbaseline . ĠThey Ġthus Ġperform Ġbetter Ġin ĠB LE U - 1 Ġwith Ġmore Ġkey Ġentities Ġ( ing red ient Ġmentions ) Ġpresent , Ġbut Ġworse Ġin ĠB LE U - 4 , Ġas Ġthese Ġrecipes Ġare Ġwritten Ġin Ġa Ġpersonalized Ġway Ġand Ġdev iate Ġfrom Ġgold Ġon Ġthe Ġphr as al Ġlevel . ĠSimilarly , Ġthe Ġ` Prior ĠName ' Ġmodel Ġgenerates Ġmore Ġun ig ram - d iverse Ġrecipes Ġthan Ġother Ġpersonalized Ġmodels Ġand Ġob tains Ġa Ġcorrespond ingly Ġlower ĠB LE U - 1 Ġscore . </s> Qual itative ĠAnalysis : ĠWe Ġpresent Ġsample Ġoutputs Ġfor Ġa Ġcocktail Ġrecipe Ġin Ġtab : sample rec ipes , Ġand Ġadditional Ġrecipes Ġin Ġthe Ġappendix . ĠGeneration Ġquality Ġprogressively Ġimproves Ġfrom Ġgeneric Ġbaseline Ġoutput Ġto Ġa Ġblended Ġcocktail Ġproduced Ġby Ġour Ġbest Ġperforming Ġmodel . ĠModels Ġattending Ġover Ġprior Ġrecipes Ġexplicitly Ġreference Ġingredients . ĠThe ĠPrior ĠName Ġmodel Ġfurther Ġsuggests Ġthe Ġaddition Ġof Ġlemon Ġand Ġmint , Ġwhich Ġare Ġreasonably Ġassociated Ġwith Ġpreviously Ġconsumed Ġrecipes Ġlike Ġcoconut Ġm ousse Ġand Ġpork Ġskew ers . </s> Personal ization : ĠTo Ġmeasure Ġpersonal ization , Ġwe Ġevaluate Ġhow Ġclosely Ġthe Ġgenerated Ġtext Ġcorresponds Ġto Ġa Ġparticular Ġuser Ġprofile . ĠWe Ġcompute Ġthe Ġlikelihood Ġof Ġgenerated Ġrecipes Ġusing Ġidentical Ġinput Ġspecifications Ġbut Ġconditioned Ġon Ġten Ġdifferent Ġuser Ġprofiles âĢĶ one Ġ` gold ' Ġuser Ġwho Ġconsumed Ġthe Ġoriginal Ġrecipe , Ġand Ġnine Ġrandomly Ġgenerated Ġuser Ġprofiles . ĠFollowing ĠB IB REF 8 , Ġwe Ġexpect Ġthe Ġhighest Ġlikelihood Ġfor Ġthe Ġrecipe Ġconditioned Ġon Ġthe Ġgold Ġuser . ĠWe Ġmeasure Ġuser Ġmatching Ġaccuracy Ġ( U MA )âĢĶ the Ġproportion Ġwhere Ġthe Ġgold Ġuser Ġis Ġranked Ġhighest âĢĶ and ĠMean ĠRec ip rocal ĠRank Ġ( MR R ) ĠB IB REF 32 Ġof Ġthe Ġgold Ġuser . ĠAll Ġpersonalized Ġmodels Ġbeat Ġbas elines Ġin Ġboth Ġmetrics , Ġshowing Ġour Ġmodels Ġpersonal ize Ġgenerated Ġrecipes Ġto Ġthe Ġgiven Ġuser Ġprofiles . ĠThe ĠPrior ĠName Ġmodel Ġachieves Ġthe Ġbest ĠU MA Ġand ĠMR R Ġby Ġa Ġlarge Ġmargin , Ġrevealing Ġthat Ġprior Ġrecipe Ġnames Ġare Ġstrong Ġsignals Ġfor Ġpersonal ization . ĠMoreover , Ġthe Ġaddition Ġof Ġattention Ġmechanisms Ġto Ġcapture Ġthese Ġsignals Ġimproves Ġlanguage Ġmodeling Ġperformance Ġover Ġa Ġstrong Ġnon - personal ized Ġbaseline . </s> Recipe ĠLevel ĠCo herence : ĠA Ġplausible Ġrecipe Ġshould Ġpossess Ġa Ġcoherent Ġstep Ġorder , Ġand Ġwe Ġevaluate Ġthis Ġvia Ġa Ġmetric Ġfor Ġrecipe - level Ġco herence . ĠWe Ġuse Ġthe Ġneural Ġscoring Ġmodel Ġfrom ĠB IB REF 33 Ġto Ġmeasure Ġrecipe - level Ġco herence Ġfor Ġeach Ġgenerated Ġrecipe . ĠEach Ġrecipe Ġstep Ġis Ġencoded Ġby ĠB ERT ĠB IB REF 34 . ĠOur Ġscoring Ġmodel Ġis Ġa ĠGR U Ġnetwork Ġthat Ġlearns Ġthe Ġoverall Ġrecipe Ġstep Ġordering Ġstructure Ġby Ġminimizing Ġthe Ġcos ine Ġsimilarity Ġof Ġrecipe Ġstep Ġhidden Ġrepresentations Ġpresented Ġin Ġthe Ġcorrect Ġand Ġreverse Ġorders . ĠOnce Ġpret rained , Ġour Ġscorer Ġcalculates Ġthe Ġsimilarity Ġof Ġa Ġgenerated Ġrecipe Ġto Ġthe Ġforward Ġand Ġbackwards Ġordering Ġof Ġits Ġcorresponding Ġgold Ġlabel , Ġgiving Ġa Ġscore Ġequal Ġto Ġthe Ġdifference Ġbetween Ġthe Ġformer Ġand Ġlatter . ĠA Ġhigher Ġscore Ġindicates Ġbetter Ġstep Ġordering Ġ( with Ġa Ġmaximum Ġscore Ġof Ġ2 ). Ġtab : co herence met rics Ġshows Ġthat Ġour Ġpersonalized Ġmodels Ġachieve Ġaverage Ġrecipe - level Ġco herence Ġscores Ġof Ġ1 . 78 - 1 . 82 , Ġsurpass ing Ġthe Ġbaseline Ġat Ġ1 . 77 . </s> Recipe ĠStep ĠEnt ail ment : ĠLocal Ġco herence Ġis Ġalso Ġcrucial Ġto Ġa Ġuser Ġfollowing Ġa Ġrecipe : Ġit Ġis Ġcrucial Ġthat Ġsubsequent Ġsteps Ġare Ġlogically Ġconsistent Ġwith Ġprior Ġones . ĠWe Ġmodel Ġlocal Ġco herence Ġas Ġan Ġentail ment Ġtask : Ġpredicting Ġthe Ġlikelihood Ġthat Ġa Ġrecipe Ġstep Ġfollows Ġthe Ġpreceding . ĠWe Ġsample Ġseveral Ġconsecutive Ġ( positive ) Ġand Ġnon - con secut ive Ġ( negative ) Ġpairs Ġof Ġsteps Ġfrom Ġeach Ġrecipe . ĠWe Ġtrain Ġa ĠB ERT ĠB IB REF 34 Ġmodel Ġto Ġpredict Ġthe Ġentail ment Ġscore Ġof Ġa Ġpair Ġof Ġsteps Ġseparated Ġby Ġa Ġ[ SE P ] Ġtoken , Ġusing Ġthe Ġfinal Ġrepresentation Ġof Ġthe Ġ[ CL S ] Ġtoken . ĠThe Ġstep Ġentail ment Ġscore Ġis Ġcomputed Ġas Ġthe Ġaverage Ġof Ġscores Ġfor Ġeach Ġset Ġof Ġconsecutive Ġsteps Ġin Ġeach Ġrecipe , Ġaveraged Ġover Ġevery Ġgenerated Ġrecipe Ġfor Ġa Ġmodel , Ġas Ġshown Ġin Ġtab : co herence met rics . </s> Human ĠEvaluation : ĠWe Ġpresented Ġ310 Ġpairs Ġof Ġrecipes Ġfor Ġpair wise Ġcomparison ĠB IB REF 8 Ġ( details Ġin Ġappendix ) Ġbetween Ġbaseline Ġand Ġeach Ġpersonalized Ġmodel , Ġwith Ġresults Ġshown Ġin Ġtab : met rics ont est . ĠOn Ġaverage , Ġhuman Ġeval u ators Ġpreferred Ġpersonalized Ġmodel Ġoutputs Ġto Ġbaseline Ġ63 % Ġof Ġthe Ġtime , Ġconfirming Ġthat Ġpersonalized Ġattention Ġimproves Ġthe Ġsemantic Ġplaus ibility Ġof Ġgenerated Ġrecipes . ĠWe Ġalso Ġperformed Ġa Ġsmall - scale Ġhuman Ġco herence Ġsurvey Ġover Ġ90 Ġrecipes , Ġin Ġwhich Ġ60 % Ġof Ġusers Ġfound Ġrecipes Ġgenerated Ġby Ġpersonalized Ġmodels Ġto Ġbe Ġmore Ġcoherent Ġand Ġpreferable Ġto Ġthose Ġgenerated Ġby Ġbaseline Ġmodels . </s> Conclusion </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġnovel Ġtask : Ġto Ġgenerate Ġpersonalized Ġrecipes Ġfrom Ġincomplete Ġinput Ġspecifications Ġand Ġuser Ġhistories . ĠOn Ġa Ġlarge Ġnovel Ġdataset Ġof Ġ180 K Ġrecipes Ġand Ġ700 K Ġreviews , Ġwe Ġshow Ġthat Ġour Ġpersonalized Ġgener ative Ġmodels Ġcan Ġgenerate Ġplausible , Ġpersonalized , Ġand Ġcoherent Ġrecipes Ġpreferred Ġby Ġhuman Ġeval u ators Ġfor Ġconsumption . ĠWe Ġalso Ġintroduce Ġa Ġset Ġof Ġautomatic Ġco herence Ġmeasures Ġfor Ġinstructional Ġtexts Ġas Ġwell Ġas Ġpersonal ization Ġmetrics Ġto Ġsupport Ġour Ġclaims . ĠOur Ġfuture Ġwork Ġincludes Ġgenerating Ġstructured Ġrepresentations Ġof Ġrecipes Ġto Ġhandle Ġingredient Ġproperties , Ġas Ġwell Ġas Ġaccounting Ġfor Ġreferences Ġto Ġcollections Ġof Ġingredients Ġ( e . g . ĠâĢ ľ dry Ġmix "). </s> Acknowled gements . ĠThis Ġwork Ġis Ġpartly Ġsupported Ġby ĠNS F Ġ# 17 500 63 . ĠWe Ġthank Ġall Ġreviewers Ġfor Ġtheir Ġconstructive Ġsuggestions , Ġas Ġwell Ġas ĠRei ĠM ., ĠSu joy ĠP ., ĠAlicia ĠL ., ĠEric ĠH ., ĠTim ĠS ., ĠKathy ĠC ., ĠAllen ĠC ., Ġand ĠMic ah ĠI . Ġfor Ġtheir Ġfeedback . </s> App endix Ġ: :: ĠFood . com : ĠDat as et ĠDetails </s> Our Ġraw Ġdata Ġconsists Ġof Ġ270 K Ġrecipes Ġand Ġ1 . 4 M Ġuser - reci pe Ġinteractions Ġ( review s ) Ġscrap ed Ġfrom ĠFood . com , Ġcovering Ġa Ġperiod Ġof Ġ18 Ġyears Ġ( January Ġ2000 Ġto ĠDecember Ġ2018 ). ĠSee Ġtab : int - stats Ġfor Ġdataset Ġsummary Ġstatistics , Ġand Ġtab : sample g k Ġfor Ġsample Ġinformation Ġabout Ġone Ġuser - reci pe Ġinteraction Ġand Ġthe Ġrecipe Ġinvolved . </s> App endix Ġ: :: ĠGener ated ĠExamples </s> See Ġtab : sample ch x Ġfor Ġa Ġsample Ġrecipe Ġfor Ġchicken Ġchili Ġand Ġtab : sample w affle Ġfor Ġa Ġsample Ġrecipe Ġfor Ġsweet Ġw affles . </s> Human ĠEvaluation </s> We Ġprepared Ġa Ġset Ġof Ġ15 Ġpair wise Ġcomparisons Ġper Ġevaluation Ġsession , Ġand Ġcollected Ġ9 30 Ġpair wise Ġevaluations Ġ( 310 Ġper Ġpersonalized Ġmodel ) Ġover Ġ62 Ġsessions . ĠFor Ġeach Ġpair , Ġusers Ġwere Ġgiven Ġa Ġpartial Ġrecipe Ġspecification Ġ( name Ġand Ġ3 - 5 Ġkey Ġingredients ), Ġas Ġwell Ġas Ġtwo Ġgenerated Ġrecipes Ġlabeled Ġ` A ' Ġand Ġ` B '. ĠOne Ġrecipe Ġis Ġgenerated Ġfrom Ġour Ġbaseline Ġenc oder - dec oder Ġmodel Ġand Ġone Ġrecipe Ġis Ġgenerated Ġby Ġone Ġof Ġour Ġthree Ġpersonalized Ġmodels Ġ( Prior ĠTech , ĠPrior ĠName , ĠPrior ĠRecipe ). ĠThe Ġorder Ġof Ġrecipe Ġpresentation Ġ( A / B ) Ġis Ġrandomly Ġselected Ġfor Ġeach Ġquestion . ĠA Ġscreenshot Ġof Ġthe Ġuser Ġevaluation Ġinterface Ġis Ġgiven Ġin Ġfig : ex eval . ĠWe Ġask Ġthe Ġuser Ġto Ġindicate Ġwhich Ġrecipe Ġthey Ġfind Ġmore Ġcoherent , Ġand Ġwhich Ġrecipe Ġbest Ġaccompl ishes Ġthe Ġgoal Ġindicated Ġby Ġthe Ġrecipe Ġname . ĠA Ġscreenshot Ġof Ġthis Ġsurvey Ġinterface Ġis Ġgiven Ġin Ġfig : ex eval 2 .