the Poly-encoder from BIBREF7 humeau2019real, Feed Yourself BIBREF13 is an open-domain dialogue agent with a self-feeding model, Kvmemnn BIBREF14 is a key-value memory network with a knowledge base that uses a key-value retrieval mechanism to train over multiple domains simultaneously, We compare against four dialogue system baselines: Kvmemnn, Feed Yourself, Poly-encoder, and a BERT bi-ranker baseline trained on the Persona-Chat dataset using the same training hyperparameters (including learning rate scheduler and length capping settings) described in Section SECREF20., a BERT bi-ranker
EXTRACTIVE
Kvmemnn,  Feed Yourself, Poly-encoder, BERT bi-ranker
EXTRACTIVE
Open-domain chatbots are more generic dialogue systems. An example is the Poly-encoder from BIBREF7 humeau2019real. It outperforms the Bi-encoder BIBREF8, BIBREF9 and matches the performance of the Cross-encoder BIBREF10, BIBREF11 while maintaining reasonable computation time. It performs strongly on downstream language understanding tasks involving pairwise comparisons, and demonstrates state-of-the-art results on the ConvAI2 challenge BIBREF12. Feed Yourself BIBREF13 is an open-domain dialogue agent with a self-feeding model. When the conversation goes well, the dialogue becomes part of the training data, and when the conversation does not, the agent asks for feedback. Lastly, Kvmemnn BIBREF14 is a key-value memory network with a knowledge base that uses a key-value retrieval mechanism to train over multiple domains simultaneously. We use all three of these models as baselines for comparison. While these can handle a greater variety of tasks, they do not respond with text that aligns with particular human-like characteristics. We compare against four dialogue system baselines: Kvmemnn, Feed Yourself, Poly-encoder, and a BERT bi-ranker baseline trained on the Persona-Chat dataset using the same training hyperparameters (including learning rate scheduler and length capping settings) described in Section SECREF20. For the first three models, we use the provided pretrained (on Persona-Chat) models. We evaluate all four on our five evaluation characters discussed in Section SECREF28.
We compare against four dialogue system baselines: Kvmemnn, Feed Yourself, Poly-encoder, and a BERT bi-ranker baseline trained on the Persona-Chat dataset using the same training hyperparameters (including learning rate scheduler and length capping settings) described in Section SECREF20. For the first three models, we use the provided pretrained (on Persona-Chat) models. We evaluate all four on our five evaluation characters discussed in Section SECREF28.
1910.08293
Introduction </s> Attempts Ġtoward Ġconstructing Ġhuman - like Ġdialogue Ġagents Ġhave Ġmet Ġsignificant Ġdifficulties , Ġsuch Ġas Ġmaintaining Ġconversation Ġconsistency ĠB IB REF 0 . ĠThis Ġis Ġlargely Ġdue Ġto Ġin abilities Ġof Ġdialogue Ġagents Ġto Ġengage Ġthe Ġuser Ġemotionally Ġbecause Ġof Ġan Ġinconsistent Ġpersonality ĠB IB REF 1 . ĠMany Ġagents Ġuse Ġpersonality Ġmodels Ġthat Ġattempt Ġto Ġmap Ġpersonality Ġattributes Ġinto Ġlower Ġdimensional Ġspaces Ġ( e . g . Ġthe ĠBig ĠFive ĠB IB REF 2 ). ĠHowever , Ġthese Ġrepresent Ġhuman Ġpersonality Ġat Ġa Ġvery Ġhigh - level Ġand Ġlack Ġdepth . ĠThey Ġprohibit Ġthe Ġability Ġto Ġlink Ġspecific Ġand Ġdetailed Ġpersonality Ġtraits Ġto Ġcharacters , Ġand Ġto Ġconstruct Ġlarge Ġdatasets Ġwhere Ġdialogue Ġis Ġtrace able Ġback Ġto Ġthese Ġtraits . </s> For Ġthis Ġreason , Ġwe Ġpropose ĠHuman ĠLevel ĠAttributes Ġ( HL As ), Ġwhich Ġwe Ġdefine Ġas Ġcharacteristics Ġof Ġfictional Ġcharacters Ġrepresentative Ġof Ġtheir Ġprofile Ġand Ġidentity . ĠWe Ġbase ĠHL As Ġon Ġtropes Ġcollected Ġfrom ĠTV ĠTrop es ĠB IB REF 3 , Ġwhich Ġare Ġdetermined Ġby Ġviewers ' Ġimpressions Ġof Ġthe Ġcharacters . ĠSee ĠFigure ĠFIG REF 1 Ġfor Ġan Ġexample . ĠBased Ġon Ġthe Ġhypothesis Ġthat Ġprofile Ġand Ġidentity Ġcontribute Ġeffectively Ġto Ġlanguage Ġstyle ĠB IB REF 4 , Ġwe Ġpropose Ġthat Ġmodeling Ġconversation Ġwith ĠHL As Ġis Ġa Ġmeans Ġfor Ġconstructing Ġa Ġdialogue Ġagent Ġwith Ġstable Ġhuman - like Ġcharacteristics . ĠBy Ġcollecting Ġdialog ues Ġfrom Ġa Ġvariety Ġof Ġcharacters Ġalong Ġwith Ġthis ĠH LA Ġinformation , Ġwe Ġpresent Ġa Ġnovel Ġlab elling Ġof Ġthis Ġdialogue Ġdata Ġwhere Ġit Ġis Ġtrace able Ġback Ġto Ġboth Ġits Ġcontext Ġand Ġassociated Ġhuman - like Ġqualities . </s> We Ġalso Ġpropose Ġa Ġsystem Ġcalled ĠAL O HA Ġ( Art ificial ĠLearning ĠOn ĠHuman ĠAttributes ) Ġas Ġa Ġnovel Ġmethod Ġof Ġincorporating ĠHL As Ġinto Ġdialogue Ġagents . ĠAL O HA Ġmaps Ġcharacters Ġto Ġa Ġlatent Ġspace Ġbased Ġon Ġtheir ĠHL As , Ġdetermines Ġwhich Ġare Ġmost Ġsimilar Ġin Ġprofile Ġand Ġidentity , Ġand Ġrecovers Ġlanguage Ġstyles Ġof Ġspecific Ġcharacters . ĠWe Ġtest Ġthe Ġperformance Ġof ĠAL O HA Ġin Ġcharacter Ġlanguage Ġstyle Ġrecovery Ġagainst Ġfour Ġbas elines , Ġdemonstrating Ġout performance Ġand Ġsystem Ġstability . ĠWe Ġalso Ġrun Ġa Ġhuman Ġevaluation Ġsupporting Ġour Ġresults . ĠOur Ġmajor Ġcontributions Ġare : Ġ( 1 ) ĠWe Ġpropose ĠHL As Ġas Ġpersonality Ġaspects Ġof Ġfictional Ġcharacters Ġfrom Ġthe Ġaudience 's Ġperspective Ġbased Ġon Ġtropes ; Ġ( 2 ) ĠWe Ġprovide Ġa Ġlarge Ġdialogue Ġdataset Ġtrace able Ġback Ġto Ġboth Ġits Ġcontext Ġand Ġassociated Ġhuman - like Ġattributes ; Ġ( 3 ) ĠWe Ġpropose Ġa Ġsystem Ġcalled ĠAL O HA Ġthat Ġis Ġable Ġto Ġrecommend Ġresponses Ġlinked Ġto Ġspecific Ġcharacters . ĠWe Ġdemonstrate Ġthat ĠAL O HA , Ġcombined Ġwith Ġthe Ġproposed Ġdataset , Ġoutper forms Ġbas elines . ĠAL O HA Ġalso Ġshows Ġstable Ġperformance Ġregardless Ġof Ġthe Ġcharacter 's Ġidentity , Ġgenre Ġof Ġthe Ġshow , Ġand Ġcontext Ġof Ġthe Ġdialogue . ĠWe Ġplan Ġto Ġrelease Ġall Ġof ĠAL O HA 's Ġdata Ġand Ġcode . </s> Related ĠWork </s> Task Ġcompletion Ġchat bots Ġ( T CC ), Ġor Ġtask - oriented Ġchat bots , Ġare Ġdialogue Ġagents Ġused Ġto Ġfulfill Ġspecific Ġpurposes , Ġsuch Ġas Ġhelping Ġcustomers Ġbook Ġairline Ġtickets , Ġor Ġa Ġgovernment Ġinquiry Ġsystem . ĠExamples Ġinclude Ġthe ĠA IM L Ġbased Ġchat bot ĠB IB REF 5 Ġand ĠD IV A ĠFramework ĠB IB REF 6 . ĠWhile ĠT CC Ġare Ġlow Ġcost , Ġeasily Ġconfig urable , Ġand Ġreadily Ġavailable , Ġthey Ġare Ġrestricted Ġto Ġworking Ġwell Ġfor Ġparticular Ġdomains Ġand Ġtasks . </s> Open - domain Ġchat bots Ġare Ġmore Ġgeneric Ġdialogue Ġsystems . ĠAn Ġexample Ġis Ġthe ĠPoly - enc oder Ġfrom ĠB IB REF 7 Ġhum e au 2019 real . ĠIt Ġoutper forms Ġthe ĠBi - enc oder ĠB IB REF 8 , ĠB IB REF 9 Ġand Ġmatches Ġthe Ġperformance Ġof Ġthe ĠCross - enc oder ĠB IB REF 10 , ĠB IB REF 11 Ġwhile Ġmaintaining Ġreasonable Ġcomputation Ġtime . ĠIt Ġperforms Ġstrongly Ġon Ġdownstream Ġlanguage Ġunderstanding Ġtasks Ġinvolving Ġpair wise Ġcomparisons , Ġand Ġdemonstrates Ġstate - of - the - art Ġresults Ġon Ġthe ĠConv AI 2 Ġchallenge ĠB IB REF 12 . ĠFeed ĠYourself ĠB IB REF 13 Ġis Ġan Ġopen - domain Ġdialogue Ġagent Ġwith Ġa Ġself - feeding Ġmodel . ĠWhen Ġthe Ġconversation Ġgoes Ġwell , Ġthe Ġdialogue Ġbecomes Ġpart Ġof Ġthe Ġtraining Ġdata , Ġand Ġwhen Ġthe Ġconversation Ġdoes Ġnot , Ġthe Ġagent Ġasks Ġfor Ġfeedback . ĠLastly , ĠK v mem nn ĠB IB REF 14 Ġis Ġa Ġkey - value Ġmemory Ġnetwork Ġwith Ġa Ġknowledge Ġbase Ġthat Ġuses Ġa Ġkey - value Ġretrieval Ġmechanism Ġto Ġtrain Ġover Ġmultiple Ġdomains Ġsimultaneously . ĠWe Ġuse Ġall Ġthree Ġof Ġthese Ġmodels Ġas Ġbas elines Ġfor Ġcomparison . ĠWhile Ġthese Ġcan Ġhandle Ġa Ġgreater Ġvariety Ġof Ġtasks , Ġthey Ġdo Ġnot Ġrespond Ġwith Ġtext Ġthat Ġalign s Ġwith Ġparticular Ġhuman - like Ġcharacteristics . </s> BI B REF 15 Ġli 2016 person a Ġdefines Ġpersona Ġ( com pos ite Ġof Ġelements Ġof Ġidentity ) Ġas Ġa Ġpossible Ġsolution Ġat Ġthe Ġword Ġlevel , Ġusing Ġback prop ag ation Ġto Ġalign Ġresponses Ġvia Ġword Ġembed d ings . ĠB IB REF 16 Ġbart l 2017 ret ri eval Ġuses Ġsentence Ġembed d ings Ġand Ġa Ġretrieval Ġmodel Ġto Ġachieve Ġhigher Ġaccuracy Ġon Ġdialogue Ġcontext . ĠB IB REF 17 Ġli u 2019 em otion Ġapplies Ġemotion Ġstates Ġof Ġsentences Ġas Ġenc od ings Ġto Ġselect Ġappropriate Ġresponses . ĠB IB REF 18 Ġp ich l 2018 al quist Ġuses Ġknowledge Ġaggregation Ġand Ġhierarchy Ġof Ġsub - dial og ues Ġfor Ġhigh Ġuser Ġengagement . ĠHowever , Ġthese Ġagents Ġrepresent Ġpersonality Ġat Ġa Ġhigh - level Ġand Ġlack Ġdetailed Ġhuman Ġqualities . ĠL IGHT ĠB IB REF 19 Ġmodels Ġadventure Ġgame Ġcharacters ' Ġdialog ues , Ġactions , Ġand Ġemotions . ĠIt Ġfocuses Ġon Ġthe Ġagent Ġidentities Ġ( e . g . Ġthief , Ġking , Ġservant ) Ġwhich Ġincludes Ġlimited Ġinformation Ġon Ġrealistic Ġhuman Ġbehaviours . ĠB IB REF 20 Ġpas un uru 2018 game Ġmodels Ġonline Ġsoccer Ġgames Ġas Ġdynamic Ġvisual Ġcontext . ĠB IB REF 21 Ġw ang 2016 learning Ġmodels Ġuser Ġdialogue Ġto Ġcomplete Ġtasks Ġinvolving Ġcertain Ġconfigurations Ġof Ġblocks . ĠB IB REF 22 Ġant ol 2015 v qa Ġmodels Ġopen - ended Ġquestions , Ġbut Ġis Ġlimited Ġto Ġvisual Ġcontexts . ĠB IB REF 23 Ġb ordes 2016 learning Ġtracks Ġuser Ġdialog ues Ġbut Ġis Ġgoal - oriented . ĠB IB REF 24 Ġil iny kh 2019 meet up Ġtracks Ġplayers ' Ġdialog ues Ġand Ġmovements Ġin Ġa Ġvisual Ġenvironment , Ġand Ġis Ġgrounded Ġon Ġnavigation Ġtasks . ĠAll Ġof Ġthese Ġperform Ġwell Ġin Ġtheir Ġrespective Ġfictional Ġenvironments , Ġbut Ġare Ġnot Ġa Ġstrong Ġrepresentation Ġof Ġhuman Ġdialogue Ġin Ġreality . </s> Method ology Ġ: :: ĠHuman ĠLevel ĠAttributes Ġ( HL A ) </s> We Ġcollect ĠH LA Ġdata Ġfrom ĠTV ĠTrop es ĠB IB REF 3 , Ġa Ġknowledge - based Ġwebsite Ġdedicated Ġto Ġpop Ġculture , Ġcontaining Ġinformation Ġon Ġa Ġplethora Ġof Ġcharacters Ġfrom Ġa Ġvariety Ġof Ġsources . ĠSimilar Ġto ĠWikipedia , Ġits Ġcontent Ġis Ġprovided Ġand Ġedited Ġcollabor atively Ġby Ġa Ġmassive Ġuser - base . ĠThese Ġattributes Ġare Ġdetermined Ġby Ġhuman Ġviewers Ġand Ġtheir Ġimpressions Ġof Ġthe Ġcharacters , Ġand Ġare Ġcorrelated Ġwith Ġhuman - like Ġcharacteristics . ĠWe Ġbelieve Ġthat ĠTV ĠTrop es Ġis Ġbetter Ġfor Ġour Ġpurpose Ġof Ġfictional Ġcharacter Ġmodeling Ġthan Ġdata Ġsources Ġused Ġin Ġworks Ġsuch Ġas ĠB IB REF 25 Ġsh uster 2019 eng aging Ġbecause ĠTV ĠTrop es ' Ġcontent Ġproviders Ġare Ġrewarded Ġfor Ġcorrectly Ġproviding Ġcontent Ġthrough Ġcommunity Ġacknowledgement . </s> TV ĠTrop es Ġdefines Ġtropes Ġas Ġattributes Ġof Ġstorytelling Ġthat Ġthe Ġaudience Ġrecognizes Ġand Ġunderstands . ĠWe Ġuse Ġtropes Ġas ĠHL As Ġto Ġcalculate Ġcorrelations Ġwith Ġspecific Ġtarget Ġcharacters . ĠWe Ġcollect Ġdata Ġfrom Ġnumerous Ġcharacters Ġfrom Ġa Ġvariety Ġof ĠTV Ġshows , Ġmovies , Ġand Ġanime . ĠWe Ġfilter Ġand Ġkeep Ġcharacters Ġwith Ġat Ġleast Ġfive ĠH LA , Ġas Ġthose Ġwith Ġfewer Ġare Ġnot Ġcomplex Ġenough Ġto Ġbe Ġcorrectly Ġmodeled Ġdue Ġto Ġreasons Ġsuch Ġas Ġlack Ġof Ġdata . ĠWe Ġend Ġup Ġeliminating Ġ5 . 86 % Ġof Ġtotal Ġcharacters , Ġand Ġend Ġup Ġwith Ġ45 , 8 21 Ġcharacters Ġand Ġ12 , 815 Ġunique ĠH LA , Ġresulting Ġin Ġ9 45 , 519 Ġtotal Ġcharacter - HL A Ġpairs . ĠEach Ġcollected Ġcharacter Ġhas Ġ20 . 64 ĠHL As Ġon Ġaverage . ĠSee ĠFigure ĠFIG REF 1 Ġfor Ġan Ġexample Ġcharacter Ġand Ġtheir ĠHL As . </s> Method ology Ġ: :: ĠOverall ĠTask </s> Our Ġtask Ġis Ġthe Ġfollowing , Ġwhere Ġ$ t $ Ġdenotes ĠâĢ ľ target ": </s> Given Ġa Ġtarget Ġcharacter Ġ$ c _ t $ Ġwith ĠH LA Ġset Ġ$ H _ t $, Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġ$ c _ t $ Ġwithout Ġany Ġdialogue Ġof Ġ$ c _ t $ Ġprovided . </s> For Ġexample , Ġif ĠSheldon ĠCooper Ġfrom ĠThe ĠBig ĠBang ĠTheory Ġis Ġ$ c _ t $, Ġthen Ġ$ H _ t $ Ġis Ġthe Ġset Ġof ĠH LA Ġon Ġthe Ġleft Ġside Ġof ĠFigure ĠFIG REF 1 . </s> We Ġdefine Ġthe Ġlanguage Ġstyle Ġof Ġa Ġcharacter Ġas Ġits Ġdiction , Ġtone , Ġand Ġspeech Ġpatterns . ĠIt Ġis Ġa Ġcharacter Ġspecific Ġlanguage Ġmodel Ġrefined Ġfrom Ġa Ġgeneral Ġlanguage Ġmodel . ĠWe Ġmust Ġlearn Ġto Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġ$ c _ t $ Ġwithout Ġits Ġdialogue Ġas Ġour Ġobjective Ġis Ġto Ġimitate Ġhuman - like Ġqualities , Ġand Ġhence Ġthe Ġmodel Ġmust Ġunderstand Ġthe Ġlanguage Ġstyles Ġof Ġcharacters Ġbased Ġon Ġtheir Ġtraits . ĠIf Ġwe Ġfeed Ġ$ c _ t $ ' s Ġdialogue Ġduring Ġtraining , Ġthe Ġmodel Ġwill Ġlikely Ġnot Ġeffectively Ġlearn Ġto Ġimitate Ġlanguage Ġstyles Ġbased Ġon ĠHL As , Ġbut Ġbased Ġon Ġthe Ġcorrelation Ġbetween Ġtext Ġin Ġthe Ġtraining Ġand Ġtesting Ġdialog ues ĠB IB REF 26 . </s> We Ġdefine Ġcharacter Ġspace Ġas Ġthe Ġcharacter Ġrepresentations Ġwithin Ġthe ĠH LA Ġlatent Ġspace Ġ( see ĠFigure ĠFIG REF 4 ), Ġand Ġthe Ġset Ġ$ C Ġ= Ġ\ l brace Ġc _ 1 , c _ 2 ,... , c _ n \ r brace Ġ$ Ġas Ġthe Ġset Ġof Ġall Ġcharacters . ĠWe Ġdefine ĠObs ervation Ġ( O BS ) Ġas Ġthe Ġinput Ġthat Ġis Ġfed Ġinto Ġany Ġdialogue Ġmodel . ĠThis Ġcan Ġbe Ġa Ġsingle Ġor Ġmultiple Ġlines Ġof Ġdialogue Ġalong Ġwith Ġother Ġinformation . ĠThe Ġgoal Ġof Ġthe Ġdialogue Ġmodel Ġis Ġto Ġfind Ġthe Ġbest Ġresponse Ġto Ġthis ĠO BS . </s> Method ology Ġ: :: ĠAL O HA </s> We Ġpropose Ġa Ġthree - component Ġsystem Ġcalled ĠAL O HA Ġto Ġsolve Ġthe Ġtask Ġ( see ĠFigure ĠFIG REF 6 ). ĠThe Ġfirst Ġcomponent , ĠCharacter ĠSpace ĠModule Ġ( CS M ), Ġgenerates Ġthe Ġcharacter Ġspace Ġand Ġcalculates Ġconfidence Ġlevels Ġusing Ġsingular Ġvalue Ġdecom position ĠB IB REF 27 Ġbetween Ġcharacters Ġ$ c _ j $ Ġ( for Ġ$ j Ġ= Ġ1 $ Ġto Ġ$ n $ Ġwhere Ġ$ j Ġ\ ne Ġt $ ) Ġand Ġ$ c _ t $ Ġin Ġthe ĠH LA - oriented Ġneighborhood . </s> The Ġsecond Ġcomponent , ĠCharacter ĠCommunity ĠModule Ġ( CC M ), Ġranks Ġthe Ġsimilarity Ġbetween Ġour Ġtarget Ġcharacter Ġ$ c _ t $ Ġwith Ġany Ġother Ġcharacter Ġ$ c _ j $ Ġby Ġthe Ġrelative Ġdistance Ġbetween Ġthem Ġin Ġthe Ġcharacter Ġspace . </s> The Ġthird Ġcomponent , ĠLanguage ĠStyle ĠRecovery ĠModule Ġ( LS RM ), Ġrecovers Ġthe Ġlanguage Ġstyle Ġof Ġ$ c _ t $ Ġwithout Ġits Ġdialogue Ġby Ġtraining Ġthe ĠB ERT Ġbi - rank er Ġmodel ĠB IB REF 28 Ġto Ġrank Ġresponses Ġfrom Ġsimilar Ġcharacters . ĠOur Ġresults Ġdemonstrate Ġhigher Ġaccuracy Ġat Ġretrieving Ġthe Ġground Ġtruth Ġresponse Ġfrom Ġ$ c _ t $. ĠOur Ġsystem Ġis Ġalso Ġable Ġto Ġpick Ġresponses Ġwhich Ġare Ġcorrect Ġboth Ġin Ġcontext Ġas Ġwell Ġas Ġcharacter Ġspace . </s> H ence , Ġthe Ġoverall Ġprocess Ġfor ĠAL O HA Ġworks Ġas Ġfollows . ĠFirst , Ġgiven Ġa Ġset Ġof Ġcharacters , Ġdetermine Ġthe Ġcharacter Ġspace Ġusing Ġthe ĠCS M . ĠNext , Ġgiven Ġa Ġspecific Ġtarget Ġcharacter , Ġdetermine Ġthe Ġpositive Ġcommunity Ġand Ġnegative Ġset Ġof Ġassociated Ġcharacters Ġusing Ġthe ĠCC M . ĠLastly , Ġusing Ġthe Ġpositive Ġcommunity Ġand Ġnegative Ġset Ġdetermined Ġabove Ġalong Ġwith Ġa Ġdialogue Ġdataset , Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġthe Ġtarget . </s> Method ology Ġ: :: ĠCharacter ĠSpace ĠModule Ġ( CS M ) </s> CS M Ġlearns Ġhow Ġto Ġrank Ġcharacters . ĠWe Ġcan Ġmeasure Ġthe Ġinter depend encies Ġbetween Ġthe ĠH LA Ġvariables ĠB IB REF 29 Ġand Ġrank Ġthe Ġsimilarity Ġbetween Ġthe ĠTV Ġshow Ġcharacters . ĠWe Ġuse Ġimplicit Ġfeedback Ġinstead Ġof Ġneighborhood Ġmodels Ġ( e . g . Ġcos ine Ġsimilarity ) Ġbecause Ġit Ġcan Ġcompute Ġlatent Ġfactors Ġto Ġtransform Ġboth Ġcharacters Ġand ĠHL As Ġinto Ġthe Ġsame Ġlatent Ġspace , Ġmaking Ġthem Ġdirectly Ġcomparable . </s> We Ġdefine Ġa Ġmatrix Ġ$ P $ Ġthat Ġcontains Ġbinary Ġvalues , Ġwith Ġ$ P _{ u , i } Ġ= Ġ1 $ Ġif Ġcharacter Ġ$ u $ Ġhas ĠH LA Ġ$ i $ Ġin Ġour Ġdataset , Ġand Ġ$ P _{ u , i } Ġ= Ġ0 $ Ġotherwise . ĠWe Ġdefine Ġa Ġconstant Ġ$\ alpha Ġ$ Ġthat Ġmeasures Ġour Ġconfidence Ġin Ġobserving Ġvarious Ġcharacter - HL A Ġpairs Ġas Ġpositive . Ġ$\ alpha Ġ$ Ġcontrols Ġhow Ġmuch Ġthe Ġmodel Ġpenal izes Ġthe Ġerror Ġif Ġthe Ġground Ġtruth Ġis Ġ$ P _{ u , i } Ġ= Ġ1 $. ĠIf Ġ$ P _{ u , i } Ġ= Ġ1 $ Ġand Ġthe Ġmodel Ġguesses Ġincorrectly , Ġwe Ġpenal ize Ġby Ġ$\ alpha Ġ$ Ġtimes Ġthe Ġloss . ĠBut Ġif Ġ$ P _{ u , i } Ġ= Ġ0 $ Ġand Ġthe Ġmodel Ġguesses Ġa Ġvalue Ġgreater Ġthan Ġ0 , Ġwe Ġdo Ġnot Ġpenal ize Ġas Ġ$\ alpha Ġ$ Ġhas Ġno Ġimpact . ĠThis Ġis Ġbecause Ġ$ P _{ u , i } Ġ= Ġ0 $ Ġcan Ġeither Ġrepresent Ġa Ġtrue Ġnegative Ġor Ġbe Ġdue Ġto Ġa Ġlack Ġof Ġdata , Ġand Ġhence Ġis Ġless Ġreliable Ġfor Ġpenal ization . ĠSee ĠEqu ation ĠDIS PLAY _ FORM 8 . ĠWe Ġfind Ġthat Ġusing Ġ$\ alpha Ġ= 20 $ Ġprovides Ġdecent Ġresults . </s> We Ġfurther Ġdefine Ġtwo Ġdense Ġvectors Ġ$ X _ u $ Ġand Ġ$ Y _ i $. ĠWe Ġcall Ġ$ X _ u $ Ġthe ĠâĢ ľ lat ent Ġfactors Ġfor Ġcharacter Ġ$ u $ ", Ġand Ġ$ Y _ i $ Ġthe ĠâĢ ľ lat ent Ġfactors Ġfor ĠH LA Ġ$ i $ ". ĠThe Ġdot Ġproduct Ġof Ġthese Ġtwo Ġvectors Ġproduces Ġa Ġvalue Ġ($ X _ u ^ TY _ i $ ) Ġthat Ġapprox imates Ġ$ P _{ u , i } $ Ġ( see ĠFigure ĠFIG REF 9 ). ĠThis Ġis Ġanalogous Ġto Ġfact oring Ġthe Ġmatrix Ġ$ P $ Ġinto Ġtwo Ġseparate Ġmat rices , Ġwhere Ġone Ġcontains Ġthe Ġlatent Ġfactors Ġfor Ġcharacters , Ġand Ġthe Ġother Ġcontains Ġthe Ġlatent Ġfactors Ġfor ĠHL As . ĠWe Ġfind Ġthat Ġ$ X _ u $ Ġand Ġ$ Y _ i $ Ġbeing Ġ36 - dimensional Ġproduces Ġdecent Ġresults . ĠTo Ġbring Ġ$ X _ u ^ TY _ i $ Ġas Ġclose Ġas Ġpossible Ġto Ġ$ P _{ u , i } $, Ġwe Ġminimize Ġthe Ġfollowing Ġloss Ġfunction Ġusing Ġthe ĠCon jug ate ĠGrad ient ĠMethod ĠB IB REF 30 : </s> The Ġfirst Ġterm Ġpenal izes Ġdifferences Ġbetween Ġthe Ġmodel 's Ġprediction Ġ($ X _ u ^ TY _ i $ ) Ġand Ġthe Ġactual Ġvalue Ġ($ P _{ u , i } $ ). ĠThe Ġsecond Ġterm Ġis Ġan ĠL 2 Ġregular izer Ġto Ġreduce Ġover fitting . ĠWe Ġfind Ġ$\ lambda Ġ= Ġ100 $ Ġprovides Ġdecent Ġresults Ġfor Ġ500 Ġiterations Ġ( see ĠSection ĠSEC REF 26 ). </s> Method ology Ġ: :: ĠCharacter ĠCommunity ĠModule Ġ( CC M ) </s> CC M Ġaims Ġto Ġdivide Ġcharacters Ġ( other Ġthan Ġ$ c _ t $ ) Ġinto Ġa Ġpositive Ġcommunity Ġand Ġa Ġnegative Ġset . ĠWe Ġdefine Ġthis Ġpositive Ġcommunity Ġas Ġcharacters Ġthat Ġare Ġdensely Ġconnected Ġinternally Ġto Ġ$ c _ t $ Ġwithin Ġthe Ġcharacter Ġspace , Ġand Ġthe Ġnegative Ġset Ġas Ġthe Ġremaining Ġcharacters . ĠWe Ġcan Ġthen Ġsample Ġdialogue Ġfrom Ġcharacters Ġin Ġthe Ġnegative Ġset Ġto Ġact Ġas Ġthe Ġdistract ors Ġ( ess entially Ġnegative Ġsamples ) Ġduring ĠL SR M Ġtraining . </s> As Ġcommunity Ġfinding Ġis Ġan Ġill - defined Ġproblem ĠB IB REF 31 , Ġwe Ġchoose Ġto Ġtreat ĠCC M Ġas Ġa Ġsimple Ġund irect ed , Ġun weight ed Ġgraph . ĠWe Ġuse Ġthe Ġvalues Ġlearned Ġin Ġthe ĠCS M Ġfor Ġ$ X _ u $ Ġand Ġ$ Y _ i $ Ġfor Ġvarious Ġvalues Ġof Ġ$ u $ Ġand Ġ$ i $, Ġwhich Ġapproximate Ġthe Ġmatrix Ġ$ P $. ĠSimilar Ġto ĠB IB REF 29 Ġh u 2008 coll abor ative , Ġwe Ġcan Ġcalculate Ġthe Ġcorrelation Ġbetween Ġtwo Ġrows Ġ( and Ġhence Ġtwo Ġcharacters ). </s> We Ġthen Ġemploy Ġa Ġtwo - level Ġconnection Ġrepresentation Ġby Ġranking Ġall Ġcharacters Ġagainst Ġeach Ġother Ġin Ġterms Ġof Ġtheir Ġcorrelation Ġwith Ġ$ c _ t $. ĠFor Ġthe Ġfirst Ġlevel , Ġthe Ġset Ġ$ S ^{ FL } $ Ġis Ġthe Ġtop Ġ10 % Ġ( 45 82 ) Ġmost Ġhighly Ġcorrelated Ġcharacters Ġwith Ġ$ c _ t $ Ġout Ġof Ġthe Ġ45 , 820 Ġtotal Ġother Ġcharacters Ġthat Ġwe Ġhave ĠH LA Ġdata Ġfor . ĠFor Ġthe Ġsecond Ġlevel , Ġfor Ġeach Ġcharacter Ġ$ s _ i $ Ġin Ġ$ S ^{ FL } $, Ġwe Ġdetermine Ġthe Ġ30 Ġmost Ġheavily Ġcorrelated Ġcharacters Ġwith Ġ$ s _ i $ Ġas Ġset Ġ$ S ^{ SL } _ i $. ĠThe Ġpositive Ġset Ġ$ S ^{ pos } $ Ġare Ġthe Ġcharacters Ġwhich Ġare Ġpresent Ġin Ġat Ġleast Ġ10 Ġ$ S ^{ SL } _ i $ Ġsets . ĠWe Ġcall Ġthis Ġvalue Ġ10 Ġthe Ġminimum Ġfrequency . ĠAll Ġother Ġcharacters Ġin Ġour Ġdialogue Ġdataset Ġmake Ġup Ġthe Ġnegative Ġset Ġ$ S ^{ neg } $. ĠThese Ġact Ġas Ġour Ġpositive Ġcommunity Ġand Ġnegative Ġset , Ġrespectively . ĠSee ĠAl gorithm Ġ1 Ġin ĠAppendix ĠA Ġfor Ġdetails , Ġand ĠFigure ĠFIG REF 11 Ġfor Ġan Ġexample . </s> Method ology Ġ: :: ĠLanguage ĠStyle ĠRecovery ĠModule Ġ( LS RM ) </s> LS RM Ġcreates Ġa Ġdialogue Ġagent Ġthat Ġalign s Ġwith Ġobserved Ġcharacteristics Ġof Ġhuman Ġcharacters Ġby Ġusing Ġthe Ġpositive Ġcharacter Ġcommunity Ġand Ġnegative Ġset Ġdetermined Ġin Ġthe ĠCC M , Ġalong Ġwith Ġa Ġdialogue Ġdataset , Ġto Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġ$ c _ t $ Ġwithout Ġits Ġdialogue . ĠWe Ġuse Ġthe ĠB ERT Ġbi - rank er Ġmodel Ġfrom Ġthe ĠFacebook ĠPar l AI Ġframework ĠB IB REF 32 , Ġwhere Ġthe Ġmodel Ġhas Ġthe Ġability Ġto Ġretrieve Ġthe Ġbest Ġresponse Ġout Ġof Ġ20 Ġcandidate Ġresponses . ĠB IB REF 12 , ĠB IB REF 19 , ĠB IB REF 0 Ġchoose Ġ20 Ġcandidate Ġresponses , Ġand Ġfor Ġcomparison Ġpurposes , Ġwe Ġdo Ġthe Ġsame . </s> Method ology Ġ: :: ĠLanguage ĠStyle ĠRecovery ĠModule Ġ( LS RM ) Ġ: :: ĠB ERT </s> BI B REF 28 Ġis Ġfirst Ġtrained Ġon Ġmassive Ġamounts Ġof Ġunl abel ed Ġtext Ġdata . ĠIt Ġjointly Ġconditions Ġon Ġtext Ġon Ġboth Ġthe Ġleft Ġand Ġright , Ġwhich Ġprovides Ġa Ġdeep Ġbi - direction al Ġrepresentation Ġof Ġsentence Ġinference . ĠB ERT Ġis Ġproven Ġto Ġperform Ġwell Ġon Ġa Ġwide Ġrange Ġof Ġtasks Ġby Ġsimply Ġfine - tun ing Ġon Ġone Ġadditional Ġlayer . ĠWe Ġare Ġinterested Ġin Ġits Ġability Ġto Ġpredict Ġthe Ġnext Ġsentence , Ġcalled ĠNext ĠSent ence ĠPrediction . ĠWe Ġperform Ġfurther Ġfine - tun ing Ġon ĠB ERT Ġfor Ġour Ġtarget Ġcharacter Ġlanguage Ġstyle Ġretrieval Ġtask Ġto Ġproduce Ġour ĠL SR M Ġmodel Ġby Ġoptimizing Ġboth Ġthe Ġencoding Ġlayers Ġand Ġthe Ġadditional Ġlayer . ĠWe Ġuse ĠB ERT Ġto Ġcreate Ġvector Ġrepresentations Ġfor Ġthe ĠO BS Ġand Ġfor Ġeach Ġcandidate Ġresponse . ĠBy Ġpassing Ġthe Ġfirst Ġoutput Ġof ĠB ERT 's Ġ12 Ġlayers Ġthrough Ġan Ġadditional Ġlinear Ġlayer , Ġthese Ġrepresentations Ġcan Ġbe Ġobtained Ġas Ġ768 - dimensional Ġsentence - level Ġembed d ings . ĠIt Ġuses Ġthe Ġdot Ġproduct Ġbetween Ġthese Ġembed d ings Ġto Ġscore Ġcandidate Ġresponses Ġand Ġis Ġtrained Ġusing Ġthe Ġranking Ġloss . </s> Method ology Ġ: :: ĠLanguage ĠStyle ĠRecovery ĠModule Ġ( LS RM ) Ġ: :: ĠCandidate Ġresponse Ġselection </s> is Ġsimilar Ġto Ġthe Ġprocedure Ġfrom Ġprevious Ġwork Ġdone Ġon Ġgrounded Ġdialogue Ġagents ĠB IB REF 0 , ĠB IB REF 19 . ĠAlong Ġwith Ġthe Ġground Ġtruth Ġresponse , Ġwe Ġrandomly Ġsample Ġ19 Ġdistract or Ġresponses Ġfrom Ġother Ġcharacters Ġfrom Ġa Ġuniform Ġdistribution Ġof Ġcharacters , Ġand Ġcall Ġthis Ġprocess Ġuniform Ġcharacter Ġsampling . ĠBased Ġon Ġour Ġobservations , Ġthis Ġrandom Ġsampling Ġprovides Ġmultiple Ġcontext Ġcorrect Ġresponses . ĠHence , Ġthe ĠB ERT Ġbi - rank er Ġmodel Ġis Ġtrained Ġby Ġlearning Ġto Ġchoose Ġcontext Ġcorrect Ġresponses , Ġand Ġthe Ġmodel Ġlearns Ġto Ġrecover Ġa Ġdomain - general Ġlanguage Ġmodel Ġthat Ġincludes Ġtraining Ġon Ġevery Ġcharacter . ĠThis Ġresults Ġin Ġa ĠUniform ĠModel Ġthat Ġcan Ġselect Ġcontext Ġcorrect Ġresponses , Ġbut Ġnot Ġresponses Ġcorresponding Ġto Ġa Ġtarget Ġcharacter Ġwith Ġspecific ĠHL As . </s> We Ġthen Ġfine - t une Ġon Ġthe Ġabove Ġmodel Ġto Ġproduce Ġour ĠL SR M Ġmodel Ġwith Ġa Ġmodification : Ġwe Ġrandomly Ġsample Ġthe Ġ19 Ġdistract or Ġresponses Ġfrom Ġonly Ġthe Ġnegative Ġcharacter Ġset Ġinstead . ĠWe Ġchoose Ġthe Ġresponses Ġthat Ġhave Ġsimilar Ġgram matical Ġstructures Ġand Ġsemantics Ġto Ġthe Ġground Ġtruth Ġresponse , Ġand Ġcall Ġthis Ġprocess Ġnegative Ġcharacter Ġsampling . ĠThis Ġguides Ġthe Ġmodel Ġaway Ġfrom Ġthe Ġlanguage Ġstyle Ġof Ġthese Ġnegative Ġcharacters Ġto Ġimprove Ġperformance Ġat Ġretrieving Ġresponses Ġfor Ġtarget Ġcharacters Ġwith Ġspecific ĠHL As . ĠOur Ġresults Ġdemonstrate Ġhigher Ġaccuracy Ġat Ġretrieving Ġthe Ġcorrect Ġresponse Ġfrom Ġcharacter Ġ$ c _ t $, Ġwhich Ġis Ġthe Ġground Ġtruth . </s> Exper iment Ġ: :: ĠDialogue ĠDat as et </s> To Ġtrain Ġthe ĠUniform ĠModel Ġand ĠL SR M , Ġwe Ġcollect Ġdialog ues Ġfrom Ġ327 Ġmajor Ġcharacters Ġ( a Ġsubset Ġof Ġthe Ġ45 , 8 21 Ġcharacters Ġwe Ġhave ĠH LA Ġdata Ġfor ) Ġin Ġ38 ĠTV Ġshows Ġfrom Ġvarious Ġexisting Ġsources Ġof Ġclean Ġdata Ġon Ġthe Ġinternet , Ġresulting Ġin Ġa Ġtotal Ġof Ġ1 , 04 2 , 647 Ġdialogue Ġlines . ĠWe Ġuse Ġa Ġsetup Ġsimilar Ġto Ġthe ĠPersona - Chat Ġdataset ĠB IB REF 0 Ġand ĠCornell ĠMovie - Dial ogs ĠCorpus ĠB IB REF 33 , Ġas Ġour Ġcollected Ġdialog ues Ġare Ġalso Ġpaired Ġin Ġterms Ġof Ġvalid Ġconversations . ĠSee ĠFigure ĠFIG REF 1 Ġfor Ġan Ġexample Ġof Ġthese Ġdialogue Ġlines . </s> Exper iment Ġ: :: ĠH LA ĠObs ervation ĠGuid ance Ġ( HL A - OG ) </s> We Ġdefine ĠH LA ĠObs ervation ĠGuid ance Ġ( HL A - OG ) Ġas Ġexplicitly Ġpassing Ġa Ġsmall Ġsubset Ġof Ġthe Ġmost Ġimportant ĠHL As Ġof Ġa Ġgiven Ġcharacter Ġas Ġpart Ġof Ġthe ĠO BS Ġrather Ġthan Ġjust Ġan Ġinitial Ġline Ġof Ġdialogue . ĠThis Ġis Ġadapted Ġfrom Ġthe Ġprocess Ġused Ġin ĠB IB REF 0 Ġz hang 2018 personal izing Ġand ĠB IB REF 10 Ġwolf 2019 trans f ert rans fo Ġwhich Ġwe Ġcall ĠPersona ĠProf iling . ĠSpecifically , Ġwe Ġpass Ġfour ĠHL As Ġthat Ġare Ġrandomly Ġdrawn Ġfrom Ġthe Ġtop Ġ40 Ġmost Ġimportant ĠHL As Ġof Ġthe Ġcharacter . ĠWe Ġuse ĠH LA - OG Ġduring Ġtraining Ġof Ġthe ĠL SR M Ġand Ġtesting Ġof Ġall Ġmodels . ĠThis Ġis Ġbecause Ġthe Ġbas elines Ġ( see ĠSection ĠSEC REF 31 ) Ġalready Ġfollow Ġa Ġsimilar Ġprocess Ġ( Person a ĠProf iling ) Ġfor Ġtraining . ĠFor Ġthe ĠUniform ĠModel , Ġwe Ġtrain Ġusing ĠNext ĠSent ence ĠPrediction Ġ( see ĠSection ĠSEC REF 12 ). ĠFor Ġtesting , ĠH LA - OG Ġis Ġnecessary Ġas Ġit Ġprovides Ġinformation Ġabout Ġwhich ĠHL As Ġthe Ġmodels Ġshould Ġattempt Ġto Ġimitate Ġin Ġtheir Ġresponse Ġselection . ĠJust Ġpassing Ġan Ġinitial Ġline Ġof Ġdialogue Ġrepl icates Ġa Ġtypical Ġdialogue Ġresponse Ġtask Ġwithout ĠHL As . ĠSee ĠTable ĠT AB REF 19 . ĠFurther , Ġwe Ġalso Ġtest Ġour ĠL SR M Ġby Ġexplicitly Ġpassing Ġfour ĠHL As Ġof Ġ` none ' Ġalong Ġwith Ġthe Ġinitial Ġline Ġof Ġdialogue Ġas Ġthe ĠO BS Ġ( No ĠH LA - OG Ġin ĠTable ĠT AB REF 19 ). </s> Exper iment Ġ: :: ĠTraining ĠDetails Ġ: :: ĠB ERT Ġbi - rank er </s> is Ġtrained Ġby Ġus Ġon Ġthe ĠPersona - Chat Ġdataset Ġfor Ġthe ĠConv AI 2 Ġchallenge . ĠSimilar Ġto ĠB IB REF 0 Ġz hang 2018 personal izing , Ġwe Ġcap Ġthe Ġlength Ġof Ġthe ĠO BS Ġat Ġ360 Ġtokens Ġand Ġthe Ġlength Ġof Ġeach Ġcandidate Ġresponse Ġat Ġ72 Ġtokens . ĠWe Ġuse Ġa Ġbatch Ġsize Ġof Ġ64 , Ġlearning Ġrate Ġof Ġ5 e - 5 , Ġand Ġperform Ġwarm - up Ġupdates Ġfor Ġ100 Ġiterations . ĠThe Ġlearning Ġrate Ġsched uler Ġuses ĠSG D Ġoptim izer Ġwith ĠN ester ov 's Ġaccelerated Ġgradient Ġdescent ĠB IB REF 34 Ġand Ġis Ġset Ġto Ġhave Ġa Ġdecay Ġof Ġ0 . 4 Ġand Ġto Ġreduce Ġon Ġplateau . </s> Exper iment Ġ: :: ĠTraining ĠDetails Ġ: :: ĠUniform ĠModel </s> is Ġproduced Ġby Ġfin et uning Ġthe ĠB ERT Ġbi - rank er Ġon Ġthe Ġdialogue Ġdata Ġdiscussed Ġin ĠSection ĠSEC REF 15 Ġusing Ġuniform Ġcharacter Ġsampling . ĠWe Ġuse Ġthe Ġsame Ġhyper param eters Ġas Ġthe ĠB ERT Ġbi - rank er Ġalong Ġwith Ġhalf - pre cision Ġoperations Ġ( i . e . Ġfloat 16 Ġoperations ) Ġto Ġincrease Ġbatch Ġsize Ġas Ġrecommended ĠB IB REF 7 . </s> Exper iment Ġ: :: ĠTraining ĠDetails Ġ: :: ĠL SR M </s> is Ġproduced Ġby Ġfin et uning Ġon Ġthe ĠUniform ĠModel Ġdiscussed Ġabove Ġusing Ġnegative Ġcharacter Ġsampling . ĠWe Ġuse Ġthe Ġsame Ġhyper param eters Ġas Ġthe ĠB ERT Ġbi - rank er Ġalong Ġwith Ġhalf - pre cision Ġoperations Ġ( i . e . Ġfloat 16 Ġoperations ) Ġto Ġincrease Ġbatch Ġsize Ġas Ġrecommended . </s> E val uation Ġ: :: ĠCS M ĠEvaluation </s> We Ġbegin Ġby Ġevaluating Ġthe Ġability Ġof Ġthe ĠCS M Ġcomponent Ġof Ġour Ġsystem Ġto Ġcorrectly Ġgenerate Ġthe Ġcharacter Ġspace . ĠTo Ġdo Ġso , Ġduring Ġtraining , Ġ30 % Ġof Ġthe Ġcharacter - HL A Ġpairs Ġ( which Ġare Ġeither Ġ0 Ġor Ġ1 ) Ġare Ġmasked , Ġand Ġthis Ġis Ġused Ġas Ġa Ġvalidation Ġset Ġ( see ĠFigure ĠFIG REF 9 ). ĠFor Ġeach Ġcharacter Ġ$ c $, Ġthe Ġmodel Ġgenerates Ġa Ġlist Ġof Ġthe Ġ12 , 815 Ġunique ĠHL As Ġranked Ġsimilarly Ġto ĠB IB REF 29 Ġh u 2008 coll abor ative Ġfor Ġ$ c $. ĠWe Ġlook Ġat Ġthe Ġrecall Ġof Ġour ĠCS M Ġmodel , Ġwhich Ġmeasures Ġthe Ġpercentage Ġof Ġtotal Ġground Ġtruth ĠHL As Ġ( over Ġall Ġcharacters Ġ$ c $ ) Ġpresent Ġwithin Ġthe Ġtop ĠN Ġranked ĠHL As Ġfor Ġall Ġ$ c $ Ġby Ġour Ġmodel . ĠThat Ġis : </s> where Ġ$ HL A _{ c } ^{ gt } $ Ġare Ġthe Ġground Ġtruth ĠHL As Ġfor Ġ$ c $, Ġand Ġ$ HL A _{ c } ^{ t N } $ Ġare Ġthe Ġtop ĠN Ġranked ĠHL As Ġby Ġthe Ġmodel Ġfor Ġ$ c $. ĠWe Ġuse Ġ$ N Ġ= Ġ100 $, Ġand Ġour Ġmodel Ġachieves Ġ25 . 08 % Ġrecall . </s> To Ġinspect Ġthe ĠCS M Ġperformance , Ġwe Ġuse Ġthe ĠT - dist ributed ĠSt och astic ĠNeighbor ĠEmb edd ing Ġ( t - S NE ) ĠB IB REF 35 Ġto Ġreduce Ġeach Ġhigh - dimension ality Ġdata Ġpoint Ġto Ġtwo - dim ensions Ġvia ĠK ull back - Le ib ler ĠDiver gence ĠB IB REF 36 . ĠThis Ġallows Ġus Ġto Ġmap Ġour Ġcharacter Ġspace Ġinto Ġtwo - dim ensions , Ġwhere Ġsimilar Ġcharacters Ġfrom Ġour Ġembed ding Ġspace Ġhave Ġhigher Ġprobability Ġof Ġbeing Ġmapped Ġclose Ġby . ĠWe Ġsampled Ġcharacters Ġfrom Ġfour Ġdifferent Ġgroups Ġor Ġregions . ĠAs Ġseen Ġin ĠFigure ĠFIG REF 4 , Ġour Ġlearned Ġcharacter Ġspace Ġeffectively Ġgroups Ġthese Ġcharacters , Ġas Ġsimilar Ġcharacters Ġare Ġadjacent Ġto Ġone Ġanother Ġin Ġfour Ġregions . </s> E val uation Ġ: :: ĠAutomatic ĠEvaluation ĠSetup Ġ: :: ĠFive - F old ĠCross ĠVal idation </s> is Ġused Ġfor Ġtraining Ġand Ġtesting Ġof Ġthe ĠUniform ĠModel Ġand ĠL SR M . ĠThe Ġfolds Ġare Ġdivided Ġrandomly Ġby Ġthe ĠTV Ġshows Ġin Ġour Ġdialogue Ġdata . ĠWe Ġuse Ġthe Ġdialogue Ġdata Ġfor Ġ80 % Ġof Ġthese Ġshows Ġas Ġthe Ġfour - f olds Ġfor Ġtraining , Ġand Ġthe Ġdialogue Ġdata Ġfor Ġthe Ġremaining Ġ20 % Ġas Ġthe Ġfifth - fold Ġfor Ġvalidation / testing . ĠThe Ġdialogue Ġdata Ġused Ġis Ġdiscussed Ġin ĠSection ĠSEC REF 15 . ĠThis Ġensures Ġno Ġmatter Ġhow Ġour Ġdata Ġis Ġdistributed , Ġeach Ġpart Ġof Ġit Ġis Ġtested , Ġallowing Ġour Ġevaluation Ġto Ġbe Ġmore Ġrobust Ġto Ġdifferent Ġcharacters . ĠSee ĠAppendix ĠC Ġfor Ġfive - fold Ġcross Ġvalidation Ġdetails Ġand Ġstatistics . </s> E val uation Ġ: :: ĠAutomatic ĠEvaluation ĠSetup Ġ: :: ĠFive ĠEvaluation ĠCharacters </s> are Ġchosen , Ġone Ġfrom Ġeach Ġof Ġthe Ġfive Ġtesting Ġsets Ġabove . ĠEach Ġis Ġa Ġwell - known Ġcharacter Ġfrom Ġa Ġseparate ĠTV Ġshow , Ġand Ġacts Ġas Ġa Ġtarget Ġcharacter Ġ$ c _ t $ Ġfor Ġevaluation Ġof Ġevery Ġmodel . ĠWe Ġchoose ĠSheldon ĠCooper Ġfrom ĠThe ĠBig ĠBang ĠTheory , ĠJean - Luc ĠPicard Ġfrom ĠStar ĠTrek , ĠMonica ĠG eller Ġfrom ĠFriends , ĠGil ĠGr iss om Ġfrom ĠCSI , Ġand ĠM arge ĠSimpson Ġfrom ĠThe ĠSimpsons . ĠWe Ġchoose Ġcharacters Ġof Ġsignificantly Ġdifferent Ġidentities Ġand Ġprofiles Ġ( int elligent Ġscientist , Ġship Ġcaptain , Ġoutgoing Ġfriend , Ġpolice Ġleader , Ġand Ġresponsible Ġmother , Ġrespectively ) Ġfrom Ġshows Ġof Ġa Ġvariety Ġof Ġgenres Ġto Ġensure Ġthat Ġwe Ġcan Ġsuccessfully Ġrecover Ġthe Ġlanguage Ġstyles Ġof Ġvarious Ġtypes Ġof Ġcharacters . ĠWe Ġchoose Ġwell - known Ġcharacters Ġbecause Ġhumans Ġrequire Ġknowledge Ġon Ġthe Ġcharacters Ġthey Ġare Ġevaluating Ġ( see ĠSection ĠSEC REF 40 ). </s> For Ġeach Ġof Ġthese Ġfive Ġevaluation Ġcharacters , Ġall Ġthe Ġdialogue Ġlines Ġfrom Ġthe Ġcharacter Ġact Ġas Ġthe Ġground Ġtruth Ġresponses . ĠThe Ġinitial Ġdialogue Ġlines Ġare Ġthe Ġcorresponding Ġdialogue Ġlines Ġto Ġwhich Ġthese Ġground Ġtruth Ġresponses Ġare Ġresponding . ĠFor Ġeach Ġinitial Ġdialogue Ġline , Ġwe Ġrandomly Ġsample Ġ19 Ġother Ġcandidate Ġresponses Ġfrom Ġthe Ġassociated Ġtesting Ġset Ġusing Ġuniform Ġcharacter Ġsampling . ĠNote Ġthat Ġthis Ġis Ġfor Ġevaluation , Ġand Ġhence Ġwe Ġuse Ġthe Ġsame Ġuniform Ġcharacter Ġsampling Ġmethod Ġfor Ġall Ġmodels Ġincluding ĠAL O HA . ĠThe Ġuse Ġof Ġnegative Ġcharacter Ġsampling Ġis Ġonly Ġin ĠAL O HA 's Ġtraining . </s> E val uation Ġ: :: ĠBas elines </s> We Ġcompare Ġagainst Ġfour Ġdialogue Ġsystem Ġbas elines : ĠK v mem nn , ĠFeed ĠYourself , ĠPoly - enc oder , Ġand Ġa ĠB ERT Ġbi - rank er Ġbaseline Ġtrained Ġon Ġthe ĠPersona - Chat Ġdataset Ġusing Ġthe Ġsame Ġtraining Ġhyper param eters Ġ( including Ġlearning Ġrate Ġsched uler Ġand Ġlength Ġca pping Ġsettings ) Ġdescribed Ġin ĠSection ĠSEC REF 20 . ĠFor Ġthe Ġfirst Ġthree Ġmodels , Ġwe Ġuse Ġthe Ġprovided Ġpret rained Ġ( on ĠPersona - Chat ) Ġmodels . ĠWe Ġevaluate Ġall Ġfour Ġon Ġour Ġfive Ġevaluation Ġcharacters Ġdiscussed Ġin ĠSection ĠSEC REF 28 . </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: ĠHits @ n / N </s> is Ġthe Ġaccuracy Ġof Ġthe Ġcorrect Ġground Ġtruth Ġresponse Ġbeing Ġwithin Ġthe Ġtop Ġ$ n $ Ġranked Ġcandidate Ġresponses Ġout Ġof Ġ$ N $ Ġtotal Ġcandidates . ĠWe Ġmeasure ĠHits @ 1 / 20 , ĠHits @ 5 / 20 , Ġand ĠHits @ 10 / 20 . </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: ĠMean ĠRank </s> is Ġthe Ġaverage Ġrank Ġthat Ġa Ġmodel Ġassigns Ġthe Ġground Ġtruth Ġresponse Ġamong Ġthe Ġ20 Ġtotal Ġcandidates . </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: ĠMean ĠRec ip rocal ĠRank Ġ( MR R ) </s> BI B REF 37 Ġlooks Ġat Ġthe Ġmean Ġof Ġthe Ġmultipl icative Ġin vers es Ġof Ġthe Ġrank Ġof Ġeach Ġcorrect Ġanswer Ġout Ġof Ġa Ġsample Ġof Ġqueries Ġ$ Q $ : </s> where Ġ$ rank _ i $ Ġrefers Ġto Ġthe Ġrank Ġposition Ġof Ġthe Ġcorrect Ġresponse Ġfor Ġthe Ġ$ i $ - th Ġquery , Ġand Ġ$ | Q | $ Ġrefers Ġto Ġthe Ġtotal Ġnumber Ġof Ġqueries Ġin Ġ$ Q $. </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: Ġ@ ! ST ART @ $ F _ 1 $ @ ! END @ - score </s> equ als Ġ$ 2 Ġ* Ġ\ frac { pre cision * rec all }{ pre cision + rec all } $. ĠFor Ġdialogue , Ġprecision Ġis Ġthe Ġfraction Ġof Ġwords Ġin Ġthe Ġchosen Ġresponse Ġcontained Ġin Ġthe Ġground Ġtruth Ġresponse , Ġand Ġrecall Ġis Ġthe Ġfraction Ġof Ġwords Ġin Ġthe Ġground Ġtruth Ġresponse Ġcontained Ġin Ġthe Ġchosen Ġresponse . </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: ĠB LE U </s> BI B REF 38 Ġgenerally Ġindicates Ġhow Ġclose Ġtwo Ġpieces Ġof Ġtext Ġare Ġin Ġcontent Ġand Ġstructure , Ġwith Ġhigher Ġvalues Ġindicating Ġgreater Ġsimilarity . ĠWe Ġreport Ġour Ġfinal ĠB LE U Ġscores Ġas Ġthe Ġaverage Ġscores Ġof Ġ1 Ġto Ġ4 - gram s . </s> E val uation Ġ: :: ĠHuman ĠEvaluation ĠSetup </s> We Ġconduct Ġa Ġhuman Ġevaluation Ġwith Ġ12 Ġparticipants , Ġ8 Ġmale Ġand Ġ4 Ġfemale , Ġwho Ġare Ġaffiliated Ġproject Ġresearchers Ġaged Ġ20 - 39 Ġat Ġthe ĠUniversity Ġof Ġ[ AN ON ]. ĠWe Ġchoose Ġthe Ġsame Ġfive Ġevaluation Ġcharacters Ġas Ġin ĠSection ĠSEC REF 28 . ĠTo Ġcontrol Ġbias , Ġeach Ġparticipant Ġevaluates Ġone Ġor Ġtwo Ġcharacters . ĠFor Ġeach Ġcharacter , Ġwe Ġrandomly Ġselect Ġ10 Ġtesting Ġsamples Ġ( each Ġincludes Ġan Ġinitial Ġline Ġof Ġdialogue Ġalong Ġwith Ġ20 Ġcandidate Ġresponses , Ġone Ġof Ġwhich Ġis Ġthe Ġground Ġtruth ) Ġfrom Ġthe Ġsame Ġtesting Ġdata Ġfor Ġthe Ġautomatic Ġevaluation Ġdiscussed Ġin ĠSection ĠSEC REF 28 . </s> These Ġten Ġsamples Ġmake Ġup Ġa Ġsingle Ġquestionnaire Ġpresented Ġin Ġfull Ġto Ġeach Ġparticipant Ġevaluating Ġthe Ġcorresponding Ġcharacter , Ġand Ġthe Ġparticipant Ġis Ġasked Ġto Ġselect Ġthe Ġsingle Ġtop Ġresponse Ġthey Ġthink Ġthe Ġcharacter Ġwould Ġmost Ġlikely Ġrespond Ġwith Ġfor Ġeach Ġof Ġthe Ġten Ġinitial Ġdialogue Ġlines . ĠSee ĠFigure ĠFIG REF 41 Ġfor Ġan Ġexample . ĠWe Ġmask Ġany Ġcharacter Ġnames Ġwithin Ġthe Ġcandidate Ġresponses Ġto Ġprevent Ġhuman Ġparticipants Ġfrom Ġusing Ġnames Ġto Ġidentify Ġwhich Ġshow Ġthe Ġresponse Ġis Ġfrom . </s> Each Ġcandidate Ġis Ġpresc reen ed Ġto Ġensure Ġthey Ġhave Ġsufficient Ġknowledge Ġof Ġthe Ġcharacter Ġto Ġbe Ġa Ġparticipant . ĠWe Ġask Ġthree Ġpresc reen ing Ġquestions Ġwhere Ġthe Ġparticipant Ġhas Ġto Ġidentify Ġan Ġimage , Ġrelationship , Ġand Ġoccupation Ġof Ġthe Ġcharacter . ĠAll Ġ12 Ġof Ġour Ġparticipants Ġpassed Ġthe Ġthe Ġpresc reen ing . </s> Results Ġand ĠAnalysis Ġ: :: ĠEvaluation ĠResults </s> Table ĠT AB REF 44 Ġshows Ġaverage Ġresults Ġof Ġour Ġautomatic Ġand Ġhuman Ġevaluations . ĠTable ĠT AB REF 45 Ġshows Ġaverage ĠHits @ 1 / 20 Ġscores Ġby Ġevaluation Ġcharacter . ĠSee ĠAppendix ĠF Ġfor Ġdetailed Ġevaluation Ġresults . ĠAL O HA Ġis Ġthe Ġmodel Ġwith ĠH LA - OG Ġduring Ġtraining Ġand Ġtesting , Ġand ĠAL O HA Ġ( No ĠH LA - OG ) Ġis Ġthe Ġmodel Ġwith ĠH LA - OG Ġduring Ġtraining Ġbut Ġtested Ġwith Ġthe Ġfour ĠHL As Ġin Ġthe ĠO BS Ġmarked Ġas Ġ` none ' Ġ( see ĠSection ĠSEC REF 17 ). ĠSee ĠAppendix ĠG Ġfor Ġdemo Ġinteractions Ġbetween Ġa Ġhuman , ĠB ERT Ġbi - rank er Ġbaseline , Ġand ĠAL O HA Ġfor Ġall Ġfive Ġevaluation Ġcharacters . </s> Results Ġand ĠAnalysis Ġ: :: ĠEvaluation ĠChallenges </s> The Ġevaluation Ġof Ġour Ġtask Ġ( ret rieving Ġthe Ġlanguage Ġstyle Ġof Ġa Ġspecific Ġcharacter ) Ġis Ġchallenging Ġand Ġhence Ġthe Ġfive - fold Ġcross Ġvalidation Ġis Ġnecessary Ġfor Ġthe Ġfollowing Ġreasons : </s> The Ġability Ġto Ġchoose Ġa Ġcontext Ġcorrect Ġresponse Ġwithout Ġattributes Ġof Ġspecific Ġcharacters Ġmay Ġbe Ġhard Ġto Ġseparate Ġfrom Ġour Ġtarget Ġmetric , Ġwhich Ġis Ġthe Ġability Ġto Ġretrieve Ġthe Ġcorrect Ġresponse Ġof Ġa Ġtarget Ġcharacter Ġby Ġits ĠHL As . ĠHowever , Ġfrom Ġmanual Ġobservation , Ġwe Ġnoticed Ġthat Ġin Ġthe Ġ20 Ġchosen Ġcandidate Ġresponses , Ġthere Ġare Ġtypically Ġnumerous Ġcontext Ġcorrect Ġresponses , Ġbut Ġonly Ġone Ġground Ġtruth Ġfor Ġthe Ġtarget Ġcharacter Ġ( for Ġan Ġexample , Ġsee ĠFigure ĠFIG REF 41 ). ĠHence , Ġa Ġmodel Ġthat Ġonly Ġchooses Ġdialogue Ġbased Ġon Ġcontext Ġis Ġdistinguish able Ġfrom Ġone Ġthat Ġlearns ĠHL As . </s> Ret rieving Ġresponses Ġfor Ġthe Ġtarget Ġcharacter Ġdepends Ġon Ġthe Ġother Ġcandidate Ġresponses . ĠFor Ġexample , Ġdialogue Ġretrieval Ġperformance Ġfor ĠGr iss om Ġfrom ĠCSI , Ġwhich Ġis Ġa Ġcrime / police Ġcontext , Ġis Ġhigher Ġthan Ġother Ġevaluation Ġcharacters Ġ( see ĠTable ĠT AB REF 45 ), Ġpotentially Ġdue Ġto Ġother Ġcandidate Ġresponses Ġnot Ġfalling Ġwithin Ġthe Ġsame Ġcrime / police Ġcontext . </s> Results Ġand ĠAnalysis Ġ: :: ĠPerformance : ĠAL O HA Ġvs . ĠHumans </s> As Ġobserved Ġfrom ĠTable ĠT AB REF 44 , ĠAL O HA Ġhas Ġa Ġperformance Ġrelatively Ġclose Ġto Ġhumans . ĠHuman ĠHits @ 1 / 20 Ġscores Ġhave Ġa Ġmean Ġof Ġ40 . 67 % Ġand Ġa Ġmedian Ġover Ġcharacters Ġof Ġ40 %. ĠThe Ġlimited Ġhuman Ġevaluation Ġsample Ġsize Ġlimits Ġwhat Ġcan Ġbe Ġinferred , Ġbut Ġit Ġindicates Ġthat Ġthe Ġproblem Ġis Ġsolved Ġto Ġthe Ġextent Ġthat ĠAL O HA Ġis Ġable Ġperform Ġrelatively Ġclose Ġto Ġhumans Ġon Ġaverage . ĠNotice Ġthat Ġeven Ġhumans Ġdo Ġnot Ġperform Ġextremely Ġwell , Ġdemonstrating Ġthat Ġthis Ġtask Ġof Ġcharacter Ġbased Ġdialogue Ġretrieval Ġis Ġmore Ġdifficult Ġthan Ġtypical Ġdialogue Ġretrieval Ġtasks ĠB IB REF 19 , ĠB IB REF 12 . </s> Looking Ġmore Ġclosely Ġat Ġeach Ġcharacter Ġfrom ĠTable ĠT AB REF 45 , Ġwe Ġcan Ġsee Ġthat Ġhuman Ġevaluation Ġscores Ġare Ġhigher Ġfor ĠSheldon Ġand ĠGr iss om . ĠThis Ġmay Ġbe Ġdue Ġto Ġthese Ġcharacters Ġhaving Ġmore Ġdistinct Ġpersonalities , Ġmaking Ġthem Ġmore Ġmemorable . </s> We Ġalso Ġlook Ġat ĠPearson Ġcorrelation Ġvalues Ġof Ġthe ĠHits @ 1 / 20 Ġscores Ġacross Ġthe Ġfive Ġevaluation Ġcharacters . ĠFor Ġhuman Ġversus ĠUniform ĠModel , Ġthis Ġis Ġ- 0 . 469 4 , Ġdemonstrating Ġthat Ġthe ĠUniform ĠModel , Ġwithout Ġknowledge Ġof ĠHL As , Ġfails Ġto Ġimitate Ġhuman Ġimpressions . ĠFor Ġhuman Ġversus ĠAL O HA , Ġthis Ġis Ġ0 . 42 50 , Ġdemonstrating Ġthat Ġour Ġsystem Ġis Ġable Ġto Ġretrieve Ġcharacter Ġresponses Ġsomewhat Ġsimilarly Ġto Ġhuman Ġimpressions . ĠLastly , Ġfor Ġhuman Ġversus Ġthe Ġdifference Ġin Ġscores Ġbetween ĠAL O HA Ġand ĠUniform ĠModel , Ġthis Ġis Ġ0 . 78 15 . ĠThe Ġdifference Ġbetween ĠAL O HA Ġand Ġthe ĠUniform ĠModel , Ġwhich Ġis Ġbased Ġon Ġthe Ġadditional Ġknowledge Ġof Ġthe ĠHL As , Ġis Ġhence Ġshown Ġto Ġimprove Ġupon Ġthe ĠUniform ĠModel Ġsimilarly Ġto Ġhuman Ġimpressions . ĠThis Ġdemonstrates Ġthat ĠHL As Ġare Ġindeed Ġan Ġaccurate Ġmethod Ġof Ġmodeling Ġhuman Ġimpressions Ġof Ġcharacter Ġattributes , Ġand Ġalso Ġdemonstrates Ġthat Ġour Ġsystem , ĠAL O HA , Ġis Ġable Ġto Ġeffectively Ġuse Ġthese ĠHL As Ġto Ġimprove Ġupon Ġdialogue Ġretrieval Ġperformance . </s> Results Ġand ĠAnalysis Ġ: :: ĠPerformance : ĠAL O HA Ġvs . ĠBas elines </s> AL O HA , Ġcombined Ġwith Ġthe ĠHL As Ġand Ġdialogue Ġdataset , Ġachieves Ġa Ġsignificant Ġimprovement Ġon Ġthe Ġtarget Ġcharacter Ġlanguage Ġstyle Ġretrieval Ġtask Ġcompared Ġto Ġthe Ġbaseline Ġopen - domain Ġchat bot Ġmodels . ĠAs Ġobserved Ġfrom ĠTable ĠT AB REF 44 , ĠAL O HA Ġachieves Ġa Ġsignificant Ġboost Ġin ĠHits @ n / N Ġaccuracy Ġand Ġother Ġmetrics Ġfor Ġretrieving Ġthe Ġcorrect Ġresponse Ġof Ġfive Ġdiverse Ġcharacters Ġwith Ġdifferent Ġidentities Ġ( see ĠSection ĠSEC REF 28 ). </s> Results Ġand ĠAnalysis Ġ: :: ĠPerformance : ĠAL O HA Ġvs . ĠUniform ĠModel </s> We Ġobserve Ġa Ġnoticeable Ġimprovement Ġin Ġperformance Ġbetween ĠAL O HA Ġand Ġthe ĠUniform ĠModel Ġin Ġrecovering Ġthe Ġlanguage Ġstyles Ġof Ġspecific Ġcharacters Ġthat Ġis Ġconsistent Ġacross Ġall Ġfive Ġfolds Ġ( see ĠTables ĠT AB REF 44 Ġand ĠT AB REF 45 ), Ġindicating Ġthat Ġlack Ġof Ġknowledge Ġof ĠHL As Ġlimits Ġthe Ġability Ġof Ġthe Ġmodel Ġto Ġsuccessfully Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġspecific Ġcharacters . ĠWe Ġclaim Ġthat , Ġto Ġthe Ġbest Ġof Ġour Ġknowledge , Ġwe Ġhave Ġmade Ġthe Ġfirst Ġstep Ġin Ġusing ĠH LA - based Ġcharacter Ġdialogue Ġclust ering Ġto Ġimprove Ġupon Ġpersonality Ġlearning Ġfor Ġchat bots . </s> AL O HA Ġdemonstrates Ġan Ġaccuracy Ġboost Ġfor Ġall Ġfive Ġevaluation Ġcharacters , Ġshowing Ġthat Ġthe Ġsystem Ġis Ġrobust Ġand Ġstable Ġand Ġhas Ġthe Ġability Ġto Ġrecover Ġthe Ġdialogue Ġstyles Ġof Ġfictional Ġcharacters Ġregardless Ġof Ġthe Ġcharacter 's Ġprofile Ġand Ġidentity , Ġgenre Ġof Ġthe Ġshow , Ġand Ġcontext Ġof Ġthe Ġdialogue . </s> Results Ġand ĠAnalysis Ġ: :: ĠPerformance : ĠH LA - OG </s> As Ġobserved Ġfrom ĠTable ĠT AB REF 44 , ĠAL O HA Ġperforms Ġslightly Ġbetter Ġoverall Ġcompared Ġto ĠAL O HA Ġ( No ĠH LA - OG ). ĠTable ĠT AB REF 45 Ġshows Ġthat Ġthis Ġslight Ġperformance Ġincrease Ġis Ġconsistent Ġacross Ġfour Ġof Ġthe Ġfive Ġevaluation Ġcharacters . ĠIn Ġthe Ġcase Ġof ĠSheldon , Ġthe ĠH LA - OG Ġmodel Ġperforms Ġa Ġbit Ġworse . ĠThis Ġis Ġpossibly Ġdue Ġto Ġthe Ġlarge Ġnumber Ġof ĠSheldon 's ĠHL As Ġ( 217 ) Ġcompared Ġto Ġthe Ġother Ġfour Ġevaluation Ġcharacters Ġ( average Ġof Ġ93 . 75 ), Ġalong Ġwith Ġthe Ġlimited Ġamount Ġof ĠHL As Ġwe Ġare Ġusing Ġfor Ġguidance Ġdue Ġto Ġthe Ġmodels ' Ġlimited Ġmemory . ĠIn Ġgeneral , ĠH LA ĠObs ervation ĠGuid ance Ġduring Ġtesting Ġappears Ġto Ġimprove Ġupon Ġthe Ġperformance Ġof ĠAL O HA , Ġbut Ġthis Ġimprovement Ġis Ġminimal . </s> Conclusion Ġand ĠFuture ĠWork </s> We Ġproposed ĠHuman ĠLevel ĠAttributes Ġ( HL As ) Ġas Ġa Ġnovel Ġapproach Ġto Ġmodel Ġhuman - like Ġattributes Ġof Ġcharacters , Ġand Ġcollected Ġa Ġlarge Ġvolume Ġof Ġdialogue Ġdata Ġfor Ġvarious Ġcharacters Ġwith Ġcomplete Ġand Ġrobust Ġprofiles . ĠWe Ġalso Ġproposed Ġand Ġevaluated Ġa Ġsystem , ĠAL O HA , Ġthat Ġuses ĠHL As Ġto Ġrecommend Ġtailored Ġresponses Ġtrace able Ġto Ġspecific Ġcharacters , Ġand Ġdemonstrated Ġits Ġout performance Ġof Ġthe Ġbas elines Ġand Ġability Ġto Ġeffectively Ġrecover Ġlanguage Ġstyles Ġof Ġvarious Ġcharacters , Ġshowing Ġpromise Ġfor Ġlearning Ġcharacter Ġor Ġpersonality Ġstyles . ĠAL O HA Ġwas Ġalso Ġshown Ġto Ġbe Ġstable Ġregardless Ġof Ġthe Ġcharacter 's Ġidentity , Ġgenre Ġof Ġshow , Ġand Ġcontext Ġof Ġdialogue . </s> Pot ential Ġdirections Ġfor Ġfuture Ġwork Ġinclude Ġtraining ĠAL O HA Ġwith Ġa Ġmulti - turn Ġresponse Ġapproach ĠB IB REF 0 Ġthat Ġtracks Ġdialogue Ġover Ġmultiple Ġresponses , Ġas Ġwe Ġcould Ġnot Ġacquire Ġmulti - turn Ġdialogue Ġdata Ġfor ĠTV Ġshows . ĠAnother Ġpotential Ġis Ġthe Ġmodeling Ġof Ġthe Ġdialog Ġcounterpart Ġ( e . g . Ġthe Ġdialogue Ġof Ġother Ġcharacters Ġspeaking Ġto Ġthe Ġtarget Ġcharacter ). ĠFurther , Ġperforming Ġsemantic Ġtext Ġexchange Ġon Ġthe Ġchosen Ġresponse Ġwith Ġa Ġmodel Ġsuch Ġas ĠSM ERT I ĠB IB REF 39 Ġmay Ġimprove Ġthe Ġability Ġof ĠAL O HA Ġto Ġcon verse Ġwith Ġhumans . ĠThis Ġis Ġbecause Ġthe Ġresponse Ġmay Ġbe Ġcontext Ġand ĠH LA Ġcorrect , Ġbut Ġincorrect Ġsem antically Ġ( e . g . Ġthe Ġresponse Ġmay Ġsay Ġthe Ġweather Ġis Ġsunny Ġwhen Ġit Ġis Ġactually Ġrainy ). ĠH LA - aligned Ġgener ative Ġmodels Ġis Ġanother Ġarea Ġof Ġexploration . ĠTypically , Ġgener ative Ġmodels Ġproduce Ġtext Ġthat Ġis Ġless Ġfluent , Ġbut Ġfurther Ġwork Ġin Ġthis Ġarea Ġmay Ġlead Ġto Ġbetter Ġresults . ĠLastly , Ġa Ġmore Ġdiverse Ġand Ġlarger Ġparticipant Ġpool Ġis Ġrequired Ġdue Ġto Ġthe Ġlimited Ġsize Ġof Ġour Ġhuman Ġevaluation .
Introduction Attempts toward constructing human-like dialogue agents have met significant difficulties, such as maintaining conversation consistency BIBREF0. This is largely due to inabilities of dialogue agents to engage the user emotionally because of an inconsistent personality BIBREF1. Many agents use personality models that attempt to map personality attributes into lower dimensional spaces (e.g. the Big Five BIBREF2). However, these represent human personality at a very high-level and lack depth. They prohibit the ability to link specific and detailed personality traits to characters, and to construct large datasets where dialogue is traceable back to these traits. For this reason, we propose Human Level Attributes (HLAs), which we define as characteristics of fictional characters representative of their profile and identity. We base HLAs on tropes collected from TV Tropes BIBREF3, which are determined by viewers' impressions of the characters. See Figure FIGREF1 for an example. Based on the hypothesis that profile and identity contribute effectively to language style BIBREF4, we propose that modeling conversation with HLAs is a means for constructing a dialogue agent with stable human-like characteristics. By collecting dialogues from a variety of characters along with this HLA information, we present a novel labelling of this dialogue data where it is traceable back to both its context and associated human-like qualities. We also propose a system called ALOHA (Artificial Learning On Human Attributes) as a novel method of incorporating HLAs into dialogue agents. ALOHA maps characters to a latent space based on their HLAs, determines which are most similar in profile and identity, and recovers language styles of specific characters. We test the performance of ALOHA in character language style recovery against four baselines, demonstrating outperformance and system stability. We also run a human evaluation supporting our results. Our major contributions are: (1) We propose HLAs as personality aspects of fictional characters from the audience's perspective based on tropes; (2) We provide a large dialogue dataset traceable back to both its context and associated human-like attributes; (3) We propose a system called ALOHA that is able to recommend responses linked to specific characters. We demonstrate that ALOHA, combined with the proposed dataset, outperforms baselines. ALOHA also shows stable performance regardless of the character's identity, genre of the show, and context of the dialogue. We plan to release all of ALOHA's data and code. Related Work Task completion chatbots (TCC), or task-oriented chatbots, are dialogue agents used to fulfill specific purposes, such as helping customers book airline tickets, or a government inquiry system. Examples include the AIML based chatbot BIBREF5 and DIVA Framework BIBREF6. While TCC are low cost, easily configurable, and readily available, they are restricted to working well for particular domains and tasks. Open-domain chatbots are more generic dialogue systems. An example is the Poly-encoder from BIBREF7 humeau2019real. It outperforms the Bi-encoder BIBREF8, BIBREF9 and matches the performance of the Cross-encoder BIBREF10, BIBREF11 while maintaining reasonable computation time. It performs strongly on downstream language understanding tasks involving pairwise comparisons, and demonstrates state-of-the-art results on the ConvAI2 challenge BIBREF12. Feed Yourself BIBREF13 is an open-domain dialogue agent with a self-feeding model. When the conversation goes well, the dialogue becomes part of the training data, and when the conversation does not, the agent asks for feedback. Lastly, Kvmemnn BIBREF14 is a key-value memory network with a knowledge base that uses a key-value retrieval mechanism to train over multiple domains simultaneously. We use all three of these models as baselines for comparison. While these can handle a greater variety of tasks, they do not respond with text that aligns with particular human-like characteristics. BIBREF15 li2016persona defines persona (composite of elements of identity) as a possible solution at the word level, using backpropagation to align responses via word embeddings. BIBREF16 bartl2017retrieval uses sentence embeddings and a retrieval model to achieve higher accuracy on dialogue context. BIBREF17 liu2019emotion applies emotion states of sentences as encodings to select appropriate responses. BIBREF18 pichl2018alquist uses knowledge aggregation and hierarchy of sub-dialogues for high user engagement. However, these agents represent personality at a high-level and lack detailed human qualities. LIGHT BIBREF19 models adventure game characters' dialogues, actions, and emotions. It focuses on the agent identities (e.g. thief, king, servant) which includes limited information on realistic human behaviours. BIBREF20 pasunuru2018game models online soccer games as dynamic visual context. BIBREF21 wang2016learning models user dialogue to complete tasks involving certain configurations of blocks. BIBREF22 antol2015vqa models open-ended questions, but is limited to visual contexts. BIBREF23 bordes2016learning tracks user dialogues but is goal-oriented. BIBREF24 ilinykh2019meetup tracks players' dialogues and movements in a visual environment, and is grounded on navigation tasks. All of these perform well in their respective fictional environments, but are not a strong representation of human dialogue in reality. Methodology ::: Human Level Attributes (HLA) We collect HLA data from TV Tropes BIBREF3, a knowledge-based website dedicated to pop culture, containing information on a plethora of characters from a variety of sources. Similar to Wikipedia, its content is provided and edited collaboratively by a massive user-base. These attributes are determined by human viewers and their impressions of the characters, and are correlated with human-like characteristics. We believe that TV Tropes is better for our purpose of fictional character modeling than data sources used in works such as BIBREF25 shuster2019engaging because TV Tropes' content providers are rewarded for correctly providing content through community acknowledgement. TV Tropes defines tropes as attributes of storytelling that the audience recognizes and understands. We use tropes as HLAs to calculate correlations with specific target characters. We collect data from numerous characters from a variety of TV shows, movies, and anime. We filter and keep characters with at least five HLA, as those with fewer are not complex enough to be correctly modeled due to reasons such as lack of data. We end up eliminating 5.86% of total characters, and end up with 45,821 characters and 12,815 unique HLA, resulting in 945,519 total character-HLA pairs. Each collected character has 20.64 HLAs on average. See Figure FIGREF1 for an example character and their HLAs. Methodology ::: Overall Task Our task is the following, where $t$ denotes “target": Given a target character $c_t$ with HLA set $H_t$, recover the language style of $c_t$ without any dialogue of $c_t$ provided. For example, if Sheldon Cooper from The Big Bang Theory is $c_t$, then $H_t$ is the set of HLA on the left side of Figure FIGREF1. We define the language style of a character as its diction, tone, and speech patterns. It is a character specific language model refined from a general language model. We must learn to recover the language style of $c_t$ without its dialogue as our objective is to imitate human-like qualities, and hence the model must understand the language styles of characters based on their traits. If we feed $c_t$'s dialogue during training, the model will likely not effectively learn to imitate language styles based on HLAs, but based on the correlation between text in the training and testing dialogues BIBREF26. We define character space as the character representations within the HLA latent space (see Figure FIGREF4), and the set $C = \lbrace c_1,c_2,...,c_n\rbrace $ as the set of all characters. We define Observation (OBS) as the input that is fed into any dialogue model. This can be a single or multiple lines of dialogue along with other information. The goal of the dialogue model is to find the best response to this OBS. Methodology ::: ALOHA We propose a three-component system called ALOHA to solve the task (see Figure FIGREF6). The first component, Character Space Module (CSM), generates the character space and calculates confidence levels using singular value decomposition BIBREF27 between characters $c_j$ (for $j = 1$ to $n$ where $j \ne t$) and $c_t$ in the HLA-oriented neighborhood. The second component, Character Community Module (CCM), ranks the similarity between our target character $c_t$ with any other character $c_j$ by the relative distance between them in the character space. The third component, Language Style Recovery Module (LSRM), recovers the language style of $c_t$ without its dialogue by training the BERT bi-ranker model BIBREF28 to rank responses from similar characters. Our results demonstrate higher accuracy at retrieving the ground truth response from $c_t$. Our system is also able to pick responses which are correct both in context as well as character space. Hence, the overall process for ALOHA works as follows. First, given a set of characters, determine the character space using the CSM. Next, given a specific target character, determine the positive community and negative set of associated characters using the CCM. Lastly, using the positive community and negative set determined above along with a dialogue dataset, recover the language style of the target. Methodology ::: Character Space Module (CSM) CSM learns how to rank characters. We can measure the interdependencies between the HLA variables BIBREF29 and rank the similarity between the TV show characters. We use implicit feedback instead of neighborhood models (e.g. cosine similarity) because it can compute latent factors to transform both characters and HLAs into the same latent space, making them directly comparable. We define a matrix $P$ that contains binary values, with $P_{u,i} = 1$ if character $u$ has HLA $i$ in our dataset, and $P_{u,i} = 0$ otherwise. We define a constant $\alpha $ that measures our confidence in observing various character-HLA pairs as positive. $\alpha $ controls how much the model penalizes the error if the ground truth is $P_{u,i} = 1$. If $P_{u,i} = 1$ and the model guesses incorrectly, we penalize by $\alpha $ times the loss. But if $P_{u,i} = 0$ and the model guesses a value greater than 0, we do not penalize as $\alpha $ has no impact. This is because $P_{u,i} = 0$ can either represent a true negative or be due to a lack of data, and hence is less reliable for penalization. See Equation DISPLAY_FORM8. We find that using $\alpha =20$ provides decent results. We further define two dense vectors $X_u$ and $Y_i$. We call $X_u$ the “latent factors for character $u$", and $Y_i$ the “latent factors for HLA $i$". The dot product of these two vectors produces a value ($X_u^TY_i$) that approximates $P_{u,i}$ (see Figure FIGREF9). This is analogous to factoring the matrix $P$ into two separate matrices, where one contains the latent factors for characters, and the other contains the latent factors for HLAs. We find that $X_u$ and $Y_i$ being 36-dimensional produces decent results. To bring $X_u^TY_i$ as close as possible to $P_{u,i}$, we minimize the following loss function using the Conjugate Gradient Method BIBREF30: The first term penalizes differences between the model's prediction ($X_u^TY_i$) and the actual value ($P_{u,i}$). The second term is an L2 regularizer to reduce overfitting. We find $\lambda = 100$ provides decent results for 500 iterations (see Section SECREF26). Methodology ::: Character Community Module (CCM) CCM aims to divide characters (other than $c_t$) into a positive community and a negative set. We define this positive community as characters that are densely connected internally to $c_t$ within the character space, and the negative set as the remaining characters. We can then sample dialogue from characters in the negative set to act as the distractors (essentially negative samples) during LSRM training. As community finding is an ill-defined problem BIBREF31, we choose to treat CCM as a simple undirected, unweighted graph. We use the values learned in the CSM for $X_u$ and $Y_i$ for various values of $u$ and $i$, which approximate the matrix $P$. Similar to BIBREF29 hu2008collaborative, we can calculate the correlation between two rows (and hence two characters). We then employ a two-level connection representation by ranking all characters against each other in terms of their correlation with $c_t$. For the first level, the set $S^{FL}$ is the top 10% (4582) most highly correlated characters with $c_t$ out of the 45,820 total other characters that we have HLA data for. For the second level, for each character $s_i$ in $S^{FL}$, we determine the 30 most heavily correlated characters with $s_i$ as set $S^{SL}_i$. The positive set $S^{pos}$ are the characters which are present in at least 10 $S^{SL}_i$ sets. We call this value 10 the minimum frequency. All other characters in our dialogue dataset make up the negative set $S^{neg}$. These act as our positive community and negative set, respectively. See Algorithm 1 in Appendix A for details, and Figure FIGREF11 for an example. Methodology ::: Language Style Recovery Module (LSRM) LSRM creates a dialogue agent that aligns with observed characteristics of human characters by using the positive character community and negative set determined in the CCM, along with a dialogue dataset, to recover the language style of $c_t$ without its dialogue. We use the BERT bi-ranker model from the Facebook ParlAI framework BIBREF32, where the model has the ability to retrieve the best response out of 20 candidate responses. BIBREF12, BIBREF19, BIBREF0 choose 20 candidate responses, and for comparison purposes, we do the same. Methodology ::: Language Style Recovery Module (LSRM) ::: BERT BIBREF28 is first trained on massive amounts of unlabeled text data. It jointly conditions on text on both the left and right, which provides a deep bi-directional representation of sentence inference. BERT is proven to perform well on a wide range of tasks by simply fine-tuning on one additional layer. We are interested in its ability to predict the next sentence, called Next Sentence Prediction. We perform further fine-tuning on BERT for our target character language style retrieval task to produce our LSRM model by optimizing both the encoding layers and the additional layer. We use BERT to create vector representations for the OBS and for each candidate response. By passing the first output of BERT's 12 layers through an additional linear layer, these representations can be obtained as 768-dimensional sentence-level embeddings. It uses the dot product between these embeddings to score candidate responses and is trained using the ranking loss. Methodology ::: Language Style Recovery Module (LSRM) ::: Candidate response selection is similar to the procedure from previous work done on grounded dialogue agents BIBREF0, BIBREF19. Along with the ground truth response, we randomly sample 19 distractor responses from other characters from a uniform distribution of characters, and call this process uniform character sampling. Based on our observations, this random sampling provides multiple context correct responses. Hence, the BERT bi-ranker model is trained by learning to choose context correct responses, and the model learns to recover a domain-general language model that includes training on every character. This results in a Uniform Model that can select context correct responses, but not responses corresponding to a target character with specific HLAs. We then fine-tune on the above model to produce our LSRM model with a modification: we randomly sample the 19 distractor responses from only the negative character set instead. We choose the responses that have similar grammatical structures and semantics to the ground truth response, and call this process negative character sampling. This guides the model away from the language style of these negative characters to improve performance at retrieving responses for target characters with specific HLAs. Our results demonstrate higher accuracy at retrieving the correct response from character $c_t$, which is the ground truth. Experiment ::: Dialogue Dataset To train the Uniform Model and LSRM, we collect dialogues from 327 major characters (a subset of the 45,821 characters we have HLA data for) in 38 TV shows from various existing sources of clean data on the internet, resulting in a total of 1,042,647 dialogue lines. We use a setup similar to the Persona-Chat dataset BIBREF0 and Cornell Movie-Dialogs Corpus BIBREF33, as our collected dialogues are also paired in terms of valid conversations. See Figure FIGREF1 for an example of these dialogue lines. Experiment ::: HLA Observation Guidance (HLA-OG) We define HLA Observation Guidance (HLA-OG) as explicitly passing a small subset of the most important HLAs of a given character as part of the OBS rather than just an initial line of dialogue. This is adapted from the process used in BIBREF0 zhang2018personalizing and BIBREF10 wolf2019transfertransfo which we call Persona Profiling. Specifically, we pass four HLAs that are randomly drawn from the top 40 most important HLAs of the character. We use HLA-OG during training of the LSRM and testing of all models. This is because the baselines (see Section SECREF31) already follow a similar process (Persona Profiling) for training. For the Uniform Model, we train using Next Sentence Prediction (see Section SECREF12). For testing, HLA-OG is necessary as it provides information about which HLAs the models should attempt to imitate in their response selection. Just passing an initial line of dialogue replicates a typical dialogue response task without HLAs. See Table TABREF19. Further, we also test our LSRM by explicitly passing four HLAs of `none' along with the initial line of dialogue as the OBS (No HLA-OG in Table TABREF19). Experiment ::: Training Details ::: BERT bi-ranker is trained by us on the Persona-Chat dataset for the ConvAI2 challenge. Similar to BIBREF0 zhang2018personalizing, we cap the length of the OBS at 360 tokens and the length of each candidate response at 72 tokens. We use a batch size of 64, learning rate of 5e-5, and perform warm-up updates for 100 iterations. The learning rate scheduler uses SGD optimizer with Nesterov's accelerated gradient descent BIBREF34 and is set to have a decay of 0.4 and to reduce on plateau. Experiment ::: Training Details ::: Uniform Model is produced by finetuning the BERT bi-ranker on the dialogue data discussed in Section SECREF15 using uniform character sampling. We use the same hyperparameters as the BERT bi-ranker along with half-precision operations (i.e. float16 operations) to increase batch size as recommended BIBREF7. Experiment ::: Training Details ::: LSRM is produced by finetuning on the Uniform Model discussed above using negative character sampling. We use the same hyperparameters as the BERT bi-ranker along with half-precision operations (i.e. float16 operations) to increase batch size as recommended. Evaluation ::: CSM Evaluation We begin by evaluating the ability of the CSM component of our system to correctly generate the character space. To do so, during training, 30% of the character-HLA pairs (which are either 0 or 1) are masked, and this is used as a validation set (see Figure FIGREF9). For each character $c$, the model generates a list of the 12,815 unique HLAs ranked similarly to BIBREF29 hu2008collaborative for $c$. We look at the recall of our CSM model, which measures the percentage of total ground truth HLAs (over all characters $c$) present within the top N ranked HLAs for all $c$ by our model. That is: where $HLA_{c}^{gt}$ are the ground truth HLAs for $c$, and $HLA_{c}^{tN}$ are the top N ranked HLAs by the model for $c$. We use $N = 100$, and our model achieves 25.08% recall. To inspect the CSM performance, we use the T-distributed Stochastic Neighbor Embedding (t-SNE) BIBREF35 to reduce each high-dimensionality data point to two-dimensions via Kullback-Leibler Divergence BIBREF36. This allows us to map our character space into two-dimensions, where similar characters from our embedding space have higher probability of being mapped close by. We sampled characters from four different groups or regions. As seen in Figure FIGREF4, our learned character space effectively groups these characters, as similar characters are adjacent to one another in four regions. Evaluation ::: Automatic Evaluation Setup ::: Five-Fold Cross Validation is used for training and testing of the Uniform Model and LSRM. The folds are divided randomly by the TV shows in our dialogue data. We use the dialogue data for 80% of these shows as the four-folds for training, and the dialogue data for the remaining 20% as the fifth-fold for validation/testing. The dialogue data used is discussed in Section SECREF15. This ensures no matter how our data is distributed, each part of it is tested, allowing our evaluation to be more robust to different characters. See Appendix C for five-fold cross validation details and statistics. Evaluation ::: Automatic Evaluation Setup ::: Five Evaluation Characters are chosen, one from each of the five testing sets above. Each is a well-known character from a separate TV show, and acts as a target character $c_t$ for evaluation of every model. We choose Sheldon Cooper from The Big Bang Theory, Jean-Luc Picard from Star Trek, Monica Geller from Friends, Gil Grissom from CSI, and Marge Simpson from The Simpsons. We choose characters of significantly different identities and profiles (intelligent scientist, ship captain, outgoing friend, police leader, and responsible mother, respectively) from shows of a variety of genres to ensure that we can successfully recover the language styles of various types of characters. We choose well-known characters because humans require knowledge on the characters they are evaluating (see Section SECREF40). For each of these five evaluation characters, all the dialogue lines from the character act as the ground truth responses. The initial dialogue lines are the corresponding dialogue lines to which these ground truth responses are responding. For each initial dialogue line, we randomly sample 19 other candidate responses from the associated testing set using uniform character sampling. Note that this is for evaluation, and hence we use the same uniform character sampling method for all models including ALOHA. The use of negative character sampling is only in ALOHA's training. Evaluation ::: Baselines We compare against four dialogue system baselines: Kvmemnn, Feed Yourself, Poly-encoder, and a BERT bi-ranker baseline trained on the Persona-Chat dataset using the same training hyperparameters (including learning rate scheduler and length capping settings) described in Section SECREF20. For the first three models, we use the provided pretrained (on Persona-Chat) models. We evaluate all four on our five evaluation characters discussed in Section SECREF28. Evaluation ::: Key Evaluation Metrics ::: Hits@n/N is the accuracy of the correct ground truth response being within the top $n$ ranked candidate responses out of $N$ total candidates. We measure Hits@1/20, Hits@5/20, and Hits@10/20. Evaluation ::: Key Evaluation Metrics ::: Mean Rank is the average rank that a model assigns the ground truth response among the 20 total candidates. Evaluation ::: Key Evaluation Metrics ::: Mean Reciprocal Rank (MRR) BIBREF37 looks at the mean of the multiplicative inverses of the rank of each correct answer out of a sample of queries $Q$: where $rank_i$ refers to the rank position of the correct response for the $i$-th query, and $|Q|$ refers to the total number of queries in $Q$. Evaluation ::: Key Evaluation Metrics ::: @!START@$F_1$@!END@-score equals $2 * \frac{precision*recall}{precision+recall}$. For dialogue, precision is the fraction of words in the chosen response contained in the ground truth response, and recall is the fraction of words in the ground truth response contained in the chosen response. Evaluation ::: Key Evaluation Metrics ::: BLEU BIBREF38 generally indicates how close two pieces of text are in content and structure, with higher values indicating greater similarity. We report our final BLEU scores as the average scores of 1 to 4-grams. Evaluation ::: Human Evaluation Setup We conduct a human evaluation with 12 participants, 8 male and 4 female, who are affiliated project researchers aged 20-39 at the University of [ANON]. We choose the same five evaluation characters as in Section SECREF28. To control bias, each participant evaluates one or two characters. For each character, we randomly select 10 testing samples (each includes an initial line of dialogue along with 20 candidate responses, one of which is the ground truth) from the same testing data for the automatic evaluation discussed in Section SECREF28. These ten samples make up a single questionnaire presented in full to each participant evaluating the corresponding character, and the participant is asked to select the single top response they think the character would most likely respond with for each of the ten initial dialogue lines. See Figure FIGREF41 for an example. We mask any character names within the candidate responses to prevent human participants from using names to identify which show the response is from. Each candidate is prescreened to ensure they have sufficient knowledge of the character to be a participant. We ask three prescreening questions where the participant has to identify an image, relationship, and occupation of the character. All 12 of our participants passed the the prescreening. Results and Analysis ::: Evaluation Results Table TABREF44 shows average results of our automatic and human evaluations. Table TABREF45 shows average Hits@1/20 scores by evaluation character. See Appendix F for detailed evaluation results. ALOHA is the model with HLA-OG during training and testing, and ALOHA (No HLA-OG) is the model with HLA-OG during training but tested with the four HLAs in the OBS marked as `none' (see Section SECREF17). See Appendix G for demo interactions between a human, BERT bi-ranker baseline, and ALOHA for all five evaluation characters. Results and Analysis ::: Evaluation Challenges The evaluation of our task (retrieving the language style of a specific character) is challenging and hence the five-fold cross validation is necessary for the following reasons: The ability to choose a context correct response without attributes of specific characters may be hard to separate from our target metric, which is the ability to retrieve the correct response of a target character by its HLAs. However, from manual observation, we noticed that in the 20 chosen candidate responses, there are typically numerous context correct responses, but only one ground truth for the target character (for an example, see Figure FIGREF41). Hence, a model that only chooses dialogue based on context is distinguishable from one that learns HLAs. Retrieving responses for the target character depends on the other candidate responses. For example, dialogue retrieval performance for Grissom from CSI, which is a crime/police context, is higher than other evaluation characters (see Table TABREF45), potentially due to other candidate responses not falling within the same crime/police context. Results and Analysis ::: Performance: ALOHA vs. Humans As observed from Table TABREF44, ALOHA has a performance relatively close to humans. Human Hits@1/20 scores have a mean of 40.67% and a median over characters of 40%. The limited human evaluation sample size limits what can be inferred, but it indicates that the problem is solved to the extent that ALOHA is able perform relatively close to humans on average. Notice that even humans do not perform extremely well, demonstrating that this task of character based dialogue retrieval is more difficult than typical dialogue retrieval tasks BIBREF19, BIBREF12. Looking more closely at each character from Table TABREF45, we can see that human evaluation scores are higher for Sheldon and Grissom. This may be due to these characters having more distinct personalities, making them more memorable. We also look at Pearson correlation values of the Hits@1/20 scores across the five evaluation characters. For human versus Uniform Model, this is -0.4694, demonstrating that the Uniform Model, without knowledge of HLAs, fails to imitate human impressions. For human versus ALOHA, this is 0.4250, demonstrating that our system is able to retrieve character responses somewhat similarly to human impressions. Lastly, for human versus the difference in scores between ALOHA and Uniform Model, this is 0.7815. The difference between ALOHA and the Uniform Model, which is based on the additional knowledge of the HLAs, is hence shown to improve upon the Uniform Model similarly to human impressions. This demonstrates that HLAs are indeed an accurate method of modeling human impressions of character attributes, and also demonstrates that our system, ALOHA, is able to effectively use these HLAs to improve upon dialogue retrieval performance. Results and Analysis ::: Performance: ALOHA vs. Baselines ALOHA, combined with the HLAs and dialogue dataset, achieves a significant improvement on the target character language style retrieval task compared to the baseline open-domain chatbot models. As observed from Table TABREF44, ALOHA achieves a significant boost in Hits@n/N accuracy and other metrics for retrieving the correct response of five diverse characters with different identities (see Section SECREF28). Results and Analysis ::: Performance: ALOHA vs. Uniform Model We observe a noticeable improvement in performance between ALOHA and the Uniform Model in recovering the language styles of specific characters that is consistent across all five folds (see Tables TABREF44 and TABREF45), indicating that lack of knowledge of HLAs limits the ability of the model to successfully recover the language style of specific characters. We claim that, to the best of our knowledge, we have made the first step in using HLA-based character dialogue clustering to improve upon personality learning for chatbots. ALOHA demonstrates an accuracy boost for all five evaluation characters, showing that the system is robust and stable and has the ability to recover the dialogue styles of fictional characters regardless of the character's profile and identity, genre of the show, and context of the dialogue. Results and Analysis ::: Performance: HLA-OG As observed from Table TABREF44, ALOHA performs slightly better overall compared to ALOHA (No HLA-OG). Table TABREF45 shows that this slight performance increase is consistent across four of the five evaluation characters. In the case of Sheldon, the HLA-OG model performs a bit worse. This is possibly due to the large number of Sheldon's HLAs (217) compared to the other four evaluation characters (average of 93.75), along with the limited amount of HLAs we are using for guidance due to the models' limited memory. In general, HLA Observation Guidance during testing appears to improve upon the performance of ALOHA, but this improvement is minimal. Conclusion and Future Work We proposed Human Level Attributes (HLAs) as a novel approach to model human-like attributes of characters, and collected a large volume of dialogue data for various characters with complete and robust profiles. We also proposed and evaluated a system, ALOHA, that uses HLAs to recommend tailored responses traceable to specific characters, and demonstrated its outperformance of the baselines and ability to effectively recover language styles of various characters, showing promise for learning character or personality styles. ALOHA was also shown to be stable regardless of the character's identity, genre of show, and context of dialogue. Potential directions for future work include training ALOHA with a multi-turn response approach BIBREF0 that tracks dialogue over multiple responses, as we could not acquire multi-turn dialogue data for TV shows. Another potential is the modeling of the dialog counterpart (e.g. the dialogue of other characters speaking to the target character). Further, performing semantic text exchange on the chosen response with a model such as SMERTI BIBREF39 may improve the ability of ALOHA to converse with humans. This is because the response may be context and HLA correct, but incorrect semantically (e.g. the response may say the weather is sunny when it is actually rainy). HLA-aligned generative models is another area of exploration. Typically, generative models produce text that is less fluent, but further work in this area may lead to better results. Lastly, a more diverse and larger participant pool is required due to the limited size of our human evaluation.
What baseline models are used?
357eb9f0c07fa45e482d998a8268bd737beb827f
What Ġbaseline Ġmodels Ġare Ġused ?
the ĠPoly - enc oder Ġfrom ĠB IB REF 7 Ġhum e au 2019 real , ĠFeed ĠYourself ĠB IB REF 13 Ġis Ġan Ġopen - domain Ġdialogue Ġagent Ġwith Ġa Ġself - feeding Ġmodel , ĠK v mem nn ĠB IB REF 14 Ġis Ġa Ġkey - value Ġmemory Ġnetwork Ġwith Ġa Ġknowledge Ġbase Ġthat Ġuses Ġa Ġkey - value Ġretrieval Ġmechanism Ġto Ġtrain Ġover Ġmultiple Ġdomains Ġsimultaneously , ĠWe Ġcompare Ġagainst Ġfour Ġdialogue Ġsystem Ġbas elines : ĠK v mem nn , ĠFeed ĠYourself , ĠPoly - enc oder , Ġand Ġa ĠB ERT Ġbi - rank er Ġbaseline Ġtrained Ġon Ġthe ĠPersona - Chat Ġdataset Ġusing Ġthe Ġsame Ġtraining Ġhyper param eters Ġ( including Ġlearning Ġrate Ġsched uler Ġand Ġlength Ġca pping Ġsettings ) Ġdescribed Ġin ĠSection ĠSEC REF 20 ., Ġa ĠB ERT Ġbi - rank er
<s> What Ġbaseline Ġmodels Ġare Ġused ? </s> Introduction </s> Attempts Ġtoward Ġconstructing Ġhuman - like Ġdialogue Ġagents Ġhave Ġmet Ġsignificant Ġdifficulties , Ġsuch Ġas Ġmaintaining Ġconversation Ġconsistency ĠB IB REF 0 . ĠThis Ġis Ġlargely Ġdue Ġto Ġin abilities Ġof Ġdialogue Ġagents Ġto Ġengage Ġthe Ġuser Ġemotionally Ġbecause Ġof Ġan Ġinconsistent Ġpersonality ĠB IB REF 1 . ĠMany Ġagents Ġuse Ġpersonality Ġmodels Ġthat Ġattempt Ġto Ġmap Ġpersonality Ġattributes Ġinto Ġlower Ġdimensional Ġspaces Ġ( e . g . Ġthe ĠBig ĠFive ĠB IB REF 2 ). ĠHowever , Ġthese Ġrepresent Ġhuman Ġpersonality Ġat Ġa Ġvery Ġhigh - level Ġand Ġlack Ġdepth . ĠThey Ġprohibit Ġthe Ġability Ġto Ġlink Ġspecific Ġand Ġdetailed Ġpersonality Ġtraits Ġto Ġcharacters , Ġand Ġto Ġconstruct Ġlarge Ġdatasets Ġwhere Ġdialogue Ġis Ġtrace able Ġback Ġto Ġthese Ġtraits . </s> For Ġthis Ġreason , Ġwe Ġpropose ĠHuman ĠLevel ĠAttributes Ġ( HL As ), Ġwhich Ġwe Ġdefine Ġas Ġcharacteristics Ġof Ġfictional Ġcharacters Ġrepresentative Ġof Ġtheir Ġprofile Ġand Ġidentity . ĠWe Ġbase ĠHL As Ġon Ġtropes Ġcollected Ġfrom ĠTV ĠTrop es ĠB IB REF 3 , Ġwhich Ġare Ġdetermined Ġby Ġviewers ' Ġimpressions Ġof Ġthe Ġcharacters . ĠSee ĠFigure ĠFIG REF 1 Ġfor Ġan Ġexample . ĠBased Ġon Ġthe Ġhypothesis Ġthat Ġprofile Ġand Ġidentity Ġcontribute Ġeffectively Ġto Ġlanguage Ġstyle ĠB IB REF 4 , Ġwe Ġpropose Ġthat Ġmodeling Ġconversation Ġwith ĠHL As Ġis Ġa Ġmeans Ġfor Ġconstructing Ġa Ġdialogue Ġagent Ġwith Ġstable Ġhuman - like Ġcharacteristics . ĠBy Ġcollecting Ġdialog ues Ġfrom Ġa Ġvariety Ġof Ġcharacters Ġalong Ġwith Ġthis ĠH LA Ġinformation , Ġwe Ġpresent Ġa Ġnovel Ġlab elling Ġof Ġthis Ġdialogue Ġdata Ġwhere Ġit Ġis Ġtrace able Ġback Ġto Ġboth Ġits Ġcontext Ġand Ġassociated Ġhuman - like Ġqualities . </s> We Ġalso Ġpropose Ġa Ġsystem Ġcalled ĠAL O HA Ġ( Art ificial ĠLearning ĠOn ĠHuman ĠAttributes ) Ġas Ġa Ġnovel Ġmethod Ġof Ġincorporating ĠHL As Ġinto Ġdialogue Ġagents . ĠAL O HA Ġmaps Ġcharacters Ġto Ġa Ġlatent Ġspace Ġbased Ġon Ġtheir ĠHL As , Ġdetermines Ġwhich Ġare Ġmost Ġsimilar Ġin Ġprofile Ġand Ġidentity , Ġand Ġrecovers Ġlanguage Ġstyles Ġof Ġspecific Ġcharacters . ĠWe Ġtest Ġthe Ġperformance Ġof ĠAL O HA Ġin Ġcharacter Ġlanguage Ġstyle Ġrecovery Ġagainst Ġfour Ġbas elines , Ġdemonstrating Ġout performance Ġand Ġsystem Ġstability . ĠWe Ġalso Ġrun Ġa Ġhuman Ġevaluation Ġsupporting Ġour Ġresults . ĠOur Ġmajor Ġcontributions Ġare : Ġ( 1 ) ĠWe Ġpropose ĠHL As Ġas Ġpersonality Ġaspects Ġof Ġfictional Ġcharacters Ġfrom Ġthe Ġaudience 's Ġperspective Ġbased Ġon Ġtropes ; Ġ( 2 ) ĠWe Ġprovide Ġa Ġlarge Ġdialogue Ġdataset Ġtrace able Ġback Ġto Ġboth Ġits Ġcontext Ġand Ġassociated Ġhuman - like Ġattributes ; Ġ( 3 ) ĠWe Ġpropose Ġa Ġsystem Ġcalled ĠAL O HA Ġthat Ġis Ġable Ġto Ġrecommend Ġresponses Ġlinked Ġto Ġspecific Ġcharacters . ĠWe Ġdemonstrate Ġthat ĠAL O HA , Ġcombined Ġwith Ġthe Ġproposed Ġdataset , Ġoutper forms Ġbas elines . ĠAL O HA Ġalso Ġshows Ġstable Ġperformance Ġregardless Ġof Ġthe Ġcharacter 's Ġidentity , Ġgenre Ġof Ġthe Ġshow , Ġand Ġcontext Ġof Ġthe Ġdialogue . ĠWe Ġplan Ġto Ġrelease Ġall Ġof ĠAL O HA 's Ġdata Ġand Ġcode . </s> Related ĠWork </s> Task Ġcompletion Ġchat bots Ġ( T CC ), Ġor Ġtask - oriented Ġchat bots , Ġare Ġdialogue Ġagents Ġused Ġto Ġfulfill Ġspecific Ġpurposes , Ġsuch Ġas Ġhelping Ġcustomers Ġbook Ġairline Ġtickets , Ġor Ġa Ġgovernment Ġinquiry Ġsystem . ĠExamples Ġinclude Ġthe ĠA IM L Ġbased Ġchat bot ĠB IB REF 5 Ġand ĠD IV A ĠFramework ĠB IB REF 6 . ĠWhile ĠT CC Ġare Ġlow Ġcost , Ġeasily Ġconfig urable , Ġand Ġreadily Ġavailable , Ġthey Ġare Ġrestricted Ġto Ġworking Ġwell Ġfor Ġparticular Ġdomains Ġand Ġtasks . </s> Open - domain Ġchat bots Ġare Ġmore Ġgeneric Ġdialogue Ġsystems . ĠAn Ġexample Ġis Ġthe ĠPoly - enc oder Ġfrom ĠB IB REF 7 Ġhum e au 2019 real . ĠIt Ġoutper forms Ġthe ĠBi - enc oder ĠB IB REF 8 , ĠB IB REF 9 Ġand Ġmatches Ġthe Ġperformance Ġof Ġthe ĠCross - enc oder ĠB IB REF 10 , ĠB IB REF 11 Ġwhile Ġmaintaining Ġreasonable Ġcomputation Ġtime . ĠIt Ġperforms Ġstrongly Ġon Ġdownstream Ġlanguage Ġunderstanding Ġtasks Ġinvolving Ġpair wise Ġcomparisons , Ġand Ġdemonstrates Ġstate - of - the - art Ġresults Ġon Ġthe ĠConv AI 2 Ġchallenge ĠB IB REF 12 . ĠFeed ĠYourself ĠB IB REF 13 Ġis Ġan Ġopen - domain Ġdialogue Ġagent Ġwith Ġa Ġself - feeding Ġmodel . ĠWhen Ġthe Ġconversation Ġgoes Ġwell , Ġthe Ġdialogue Ġbecomes Ġpart Ġof Ġthe Ġtraining Ġdata , Ġand Ġwhen Ġthe Ġconversation Ġdoes Ġnot , Ġthe Ġagent Ġasks Ġfor Ġfeedback . ĠLastly , ĠK v mem nn ĠB IB REF 14 Ġis Ġa Ġkey - value Ġmemory Ġnetwork Ġwith Ġa Ġknowledge Ġbase Ġthat Ġuses Ġa Ġkey - value Ġretrieval Ġmechanism Ġto Ġtrain Ġover Ġmultiple Ġdomains Ġsimultaneously . ĠWe Ġuse Ġall Ġthree Ġof Ġthese Ġmodels Ġas Ġbas elines Ġfor Ġcomparison . ĠWhile Ġthese Ġcan Ġhandle Ġa Ġgreater Ġvariety Ġof Ġtasks , Ġthey Ġdo Ġnot Ġrespond Ġwith Ġtext Ġthat Ġalign s Ġwith Ġparticular Ġhuman - like Ġcharacteristics . </s> BI B REF 15 Ġli 2016 person a Ġdefines Ġpersona Ġ( com pos ite Ġof Ġelements Ġof Ġidentity ) Ġas Ġa Ġpossible Ġsolution Ġat Ġthe Ġword Ġlevel , Ġusing Ġback prop ag ation Ġto Ġalign Ġresponses Ġvia Ġword Ġembed d ings . ĠB IB REF 16 Ġbart l 2017 ret ri eval Ġuses Ġsentence Ġembed d ings Ġand Ġa Ġretrieval Ġmodel Ġto Ġachieve Ġhigher Ġaccuracy Ġon Ġdialogue Ġcontext . ĠB IB REF 17 Ġli u 2019 em otion Ġapplies Ġemotion Ġstates Ġof Ġsentences Ġas Ġenc od ings Ġto Ġselect Ġappropriate Ġresponses . ĠB IB REF 18 Ġp ich l 2018 al quist Ġuses Ġknowledge Ġaggregation Ġand Ġhierarchy Ġof Ġsub - dial og ues Ġfor Ġhigh Ġuser Ġengagement . ĠHowever , Ġthese Ġagents Ġrepresent Ġpersonality Ġat Ġa Ġhigh - level Ġand Ġlack Ġdetailed Ġhuman Ġqualities . ĠL IGHT ĠB IB REF 19 Ġmodels Ġadventure Ġgame Ġcharacters ' Ġdialog ues , Ġactions , Ġand Ġemotions . ĠIt Ġfocuses Ġon Ġthe Ġagent Ġidentities Ġ( e . g . Ġthief , Ġking , Ġservant ) Ġwhich Ġincludes Ġlimited Ġinformation Ġon Ġrealistic Ġhuman Ġbehaviours . ĠB IB REF 20 Ġpas un uru 2018 game Ġmodels Ġonline Ġsoccer Ġgames Ġas Ġdynamic Ġvisual Ġcontext . ĠB IB REF 21 Ġw ang 2016 learning Ġmodels Ġuser Ġdialogue Ġto Ġcomplete Ġtasks Ġinvolving Ġcertain Ġconfigurations Ġof Ġblocks . ĠB IB REF 22 Ġant ol 2015 v qa Ġmodels Ġopen - ended Ġquestions , Ġbut Ġis Ġlimited Ġto Ġvisual Ġcontexts . ĠB IB REF 23 Ġb ordes 2016 learning Ġtracks Ġuser Ġdialog ues Ġbut Ġis Ġgoal - oriented . ĠB IB REF 24 Ġil iny kh 2019 meet up Ġtracks Ġplayers ' Ġdialog ues Ġand Ġmovements Ġin Ġa Ġvisual Ġenvironment , Ġand Ġis Ġgrounded Ġon Ġnavigation Ġtasks . ĠAll Ġof Ġthese Ġperform Ġwell Ġin Ġtheir Ġrespective Ġfictional Ġenvironments , Ġbut Ġare Ġnot Ġa Ġstrong Ġrepresentation Ġof Ġhuman Ġdialogue Ġin Ġreality . </s> Method ology Ġ: :: ĠHuman ĠLevel ĠAttributes Ġ( HL A ) </s> We Ġcollect ĠH LA Ġdata Ġfrom ĠTV ĠTrop es ĠB IB REF 3 , Ġa Ġknowledge - based Ġwebsite Ġdedicated Ġto Ġpop Ġculture , Ġcontaining Ġinformation Ġon Ġa Ġplethora Ġof Ġcharacters Ġfrom Ġa Ġvariety Ġof Ġsources . ĠSimilar Ġto ĠWikipedia , Ġits Ġcontent Ġis Ġprovided Ġand Ġedited Ġcollabor atively Ġby Ġa Ġmassive Ġuser - base . ĠThese Ġattributes Ġare Ġdetermined Ġby Ġhuman Ġviewers Ġand Ġtheir Ġimpressions Ġof Ġthe Ġcharacters , Ġand Ġare Ġcorrelated Ġwith Ġhuman - like Ġcharacteristics . ĠWe Ġbelieve Ġthat ĠTV ĠTrop es Ġis Ġbetter Ġfor Ġour Ġpurpose Ġof Ġfictional Ġcharacter Ġmodeling Ġthan Ġdata Ġsources Ġused Ġin Ġworks Ġsuch Ġas ĠB IB REF 25 Ġsh uster 2019 eng aging Ġbecause ĠTV ĠTrop es ' Ġcontent Ġproviders Ġare Ġrewarded Ġfor Ġcorrectly Ġproviding Ġcontent Ġthrough Ġcommunity Ġacknowledgement . </s> TV ĠTrop es Ġdefines Ġtropes Ġas Ġattributes Ġof Ġstorytelling Ġthat Ġthe Ġaudience Ġrecognizes Ġand Ġunderstands . ĠWe Ġuse Ġtropes Ġas ĠHL As Ġto Ġcalculate Ġcorrelations Ġwith Ġspecific Ġtarget Ġcharacters . ĠWe Ġcollect Ġdata Ġfrom Ġnumerous Ġcharacters Ġfrom Ġa Ġvariety Ġof ĠTV Ġshows , Ġmovies , Ġand Ġanime . ĠWe Ġfilter Ġand Ġkeep Ġcharacters Ġwith Ġat Ġleast Ġfive ĠH LA , Ġas Ġthose Ġwith Ġfewer Ġare Ġnot Ġcomplex Ġenough Ġto Ġbe Ġcorrectly Ġmodeled Ġdue Ġto Ġreasons Ġsuch Ġas Ġlack Ġof Ġdata . ĠWe Ġend Ġup Ġeliminating Ġ5 . 86 % Ġof Ġtotal Ġcharacters , Ġand Ġend Ġup Ġwith Ġ45 , 8 21 Ġcharacters Ġand Ġ12 , 815 Ġunique ĠH LA , Ġresulting Ġin Ġ9 45 , 519 Ġtotal Ġcharacter - HL A Ġpairs . ĠEach Ġcollected Ġcharacter Ġhas Ġ20 . 64 ĠHL As Ġon Ġaverage . ĠSee ĠFigure ĠFIG REF 1 Ġfor Ġan Ġexample Ġcharacter Ġand Ġtheir ĠHL As . </s> Method ology Ġ: :: ĠOverall ĠTask </s> Our Ġtask Ġis Ġthe Ġfollowing , Ġwhere Ġ$ t $ Ġdenotes ĠâĢ ľ target ": </s> Given Ġa Ġtarget Ġcharacter Ġ$ c _ t $ Ġwith ĠH LA Ġset Ġ$ H _ t $, Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġ$ c _ t $ Ġwithout Ġany Ġdialogue Ġof Ġ$ c _ t $ Ġprovided . </s> For Ġexample , Ġif ĠSheldon ĠCooper Ġfrom ĠThe ĠBig ĠBang ĠTheory Ġis Ġ$ c _ t $, Ġthen Ġ$ H _ t $ Ġis Ġthe Ġset Ġof ĠH LA Ġon Ġthe Ġleft Ġside Ġof ĠFigure ĠFIG REF 1 . </s> We Ġdefine Ġthe Ġlanguage Ġstyle Ġof Ġa Ġcharacter Ġas Ġits Ġdiction , Ġtone , Ġand Ġspeech Ġpatterns . ĠIt Ġis Ġa Ġcharacter Ġspecific Ġlanguage Ġmodel Ġrefined Ġfrom Ġa Ġgeneral Ġlanguage Ġmodel . ĠWe Ġmust Ġlearn Ġto Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġ$ c _ t $ Ġwithout Ġits Ġdialogue Ġas Ġour Ġobjective Ġis Ġto Ġimitate Ġhuman - like Ġqualities , Ġand Ġhence Ġthe Ġmodel Ġmust Ġunderstand Ġthe Ġlanguage Ġstyles Ġof Ġcharacters Ġbased Ġon Ġtheir Ġtraits . ĠIf Ġwe Ġfeed Ġ$ c _ t $ ' s Ġdialogue Ġduring Ġtraining , Ġthe Ġmodel Ġwill Ġlikely Ġnot Ġeffectively Ġlearn Ġto Ġimitate Ġlanguage Ġstyles Ġbased Ġon ĠHL As , Ġbut Ġbased Ġon Ġthe Ġcorrelation Ġbetween Ġtext Ġin Ġthe Ġtraining Ġand Ġtesting Ġdialog ues ĠB IB REF 26 . </s> We Ġdefine Ġcharacter Ġspace Ġas Ġthe Ġcharacter Ġrepresentations Ġwithin Ġthe ĠH LA Ġlatent Ġspace Ġ( see ĠFigure ĠFIG REF 4 ), Ġand Ġthe Ġset Ġ$ C Ġ= Ġ\ l brace Ġc _ 1 , c _ 2 ,... , c _ n \ r brace Ġ$ Ġas Ġthe Ġset Ġof Ġall Ġcharacters . ĠWe Ġdefine ĠObs ervation Ġ( O BS ) Ġas Ġthe Ġinput Ġthat Ġis Ġfed Ġinto Ġany Ġdialogue Ġmodel . ĠThis Ġcan Ġbe Ġa Ġsingle Ġor Ġmultiple Ġlines Ġof Ġdialogue Ġalong Ġwith Ġother Ġinformation . ĠThe Ġgoal Ġof Ġthe Ġdialogue Ġmodel Ġis Ġto Ġfind Ġthe Ġbest Ġresponse Ġto Ġthis ĠO BS . </s> Method ology Ġ: :: ĠAL O HA </s> We Ġpropose Ġa Ġthree - component Ġsystem Ġcalled ĠAL O HA Ġto Ġsolve Ġthe Ġtask Ġ( see ĠFigure ĠFIG REF 6 ). ĠThe Ġfirst Ġcomponent , ĠCharacter ĠSpace ĠModule Ġ( CS M ), Ġgenerates Ġthe Ġcharacter Ġspace Ġand Ġcalculates Ġconfidence Ġlevels Ġusing Ġsingular Ġvalue Ġdecom position ĠB IB REF 27 Ġbetween Ġcharacters Ġ$ c _ j $ Ġ( for Ġ$ j Ġ= Ġ1 $ Ġto Ġ$ n $ Ġwhere Ġ$ j Ġ\ ne Ġt $ ) Ġand Ġ$ c _ t $ Ġin Ġthe ĠH LA - oriented Ġneighborhood . </s> The Ġsecond Ġcomponent , ĠCharacter ĠCommunity ĠModule Ġ( CC M ), Ġranks Ġthe Ġsimilarity Ġbetween Ġour Ġtarget Ġcharacter Ġ$ c _ t $ Ġwith Ġany Ġother Ġcharacter Ġ$ c _ j $ Ġby Ġthe Ġrelative Ġdistance Ġbetween Ġthem Ġin Ġthe Ġcharacter Ġspace . </s> The Ġthird Ġcomponent , ĠLanguage ĠStyle ĠRecovery ĠModule Ġ( LS RM ), Ġrecovers Ġthe Ġlanguage Ġstyle Ġof Ġ$ c _ t $ Ġwithout Ġits Ġdialogue Ġby Ġtraining Ġthe ĠB ERT Ġbi - rank er Ġmodel ĠB IB REF 28 Ġto Ġrank Ġresponses Ġfrom Ġsimilar Ġcharacters . ĠOur Ġresults Ġdemonstrate Ġhigher Ġaccuracy Ġat Ġretrieving Ġthe Ġground Ġtruth Ġresponse Ġfrom Ġ$ c _ t $. ĠOur Ġsystem Ġis Ġalso Ġable Ġto Ġpick Ġresponses Ġwhich Ġare Ġcorrect Ġboth Ġin Ġcontext Ġas Ġwell Ġas Ġcharacter Ġspace . </s> H ence , Ġthe Ġoverall Ġprocess Ġfor ĠAL O HA Ġworks Ġas Ġfollows . ĠFirst , Ġgiven Ġa Ġset Ġof Ġcharacters , Ġdetermine Ġthe Ġcharacter Ġspace Ġusing Ġthe ĠCS M . ĠNext , Ġgiven Ġa Ġspecific Ġtarget Ġcharacter , Ġdetermine Ġthe Ġpositive Ġcommunity Ġand Ġnegative Ġset Ġof Ġassociated Ġcharacters Ġusing Ġthe ĠCC M . ĠLastly , Ġusing Ġthe Ġpositive Ġcommunity Ġand Ġnegative Ġset Ġdetermined Ġabove Ġalong Ġwith Ġa Ġdialogue Ġdataset , Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġthe Ġtarget . </s> Method ology Ġ: :: ĠCharacter ĠSpace ĠModule Ġ( CS M ) </s> CS M Ġlearns Ġhow Ġto Ġrank Ġcharacters . ĠWe Ġcan Ġmeasure Ġthe Ġinter depend encies Ġbetween Ġthe ĠH LA Ġvariables ĠB IB REF 29 Ġand Ġrank Ġthe Ġsimilarity Ġbetween Ġthe ĠTV Ġshow Ġcharacters . ĠWe Ġuse Ġimplicit Ġfeedback Ġinstead Ġof Ġneighborhood Ġmodels Ġ( e . g . Ġcos ine Ġsimilarity ) Ġbecause Ġit Ġcan Ġcompute Ġlatent Ġfactors Ġto Ġtransform Ġboth Ġcharacters Ġand ĠHL As Ġinto Ġthe Ġsame Ġlatent Ġspace , Ġmaking Ġthem Ġdirectly Ġcomparable . </s> We Ġdefine Ġa Ġmatrix Ġ$ P $ Ġthat Ġcontains Ġbinary Ġvalues , Ġwith Ġ$ P _{ u , i } Ġ= Ġ1 $ Ġif Ġcharacter Ġ$ u $ Ġhas ĠH LA Ġ$ i $ Ġin Ġour Ġdataset , Ġand Ġ$ P _{ u , i } Ġ= Ġ0 $ Ġotherwise . ĠWe Ġdefine Ġa Ġconstant Ġ$\ alpha Ġ$ Ġthat Ġmeasures Ġour Ġconfidence Ġin Ġobserving Ġvarious Ġcharacter - HL A Ġpairs Ġas Ġpositive . Ġ$\ alpha Ġ$ Ġcontrols Ġhow Ġmuch Ġthe Ġmodel Ġpenal izes Ġthe Ġerror Ġif Ġthe Ġground Ġtruth Ġis Ġ$ P _{ u , i } Ġ= Ġ1 $. ĠIf Ġ$ P _{ u , i } Ġ= Ġ1 $ Ġand Ġthe Ġmodel Ġguesses Ġincorrectly , Ġwe Ġpenal ize Ġby Ġ$\ alpha Ġ$ Ġtimes Ġthe Ġloss . ĠBut Ġif Ġ$ P _{ u , i } Ġ= Ġ0 $ Ġand Ġthe Ġmodel Ġguesses Ġa Ġvalue Ġgreater Ġthan Ġ0 , Ġwe Ġdo Ġnot Ġpenal ize Ġas Ġ$\ alpha Ġ$ Ġhas Ġno Ġimpact . ĠThis Ġis Ġbecause Ġ$ P _{ u , i } Ġ= Ġ0 $ Ġcan Ġeither Ġrepresent Ġa Ġtrue Ġnegative Ġor Ġbe Ġdue Ġto Ġa Ġlack Ġof Ġdata , Ġand Ġhence Ġis Ġless Ġreliable Ġfor Ġpenal ization . ĠSee ĠEqu ation ĠDIS PLAY _ FORM 8 . ĠWe Ġfind Ġthat Ġusing Ġ$\ alpha Ġ= 20 $ Ġprovides Ġdecent Ġresults . </s> We Ġfurther Ġdefine Ġtwo Ġdense Ġvectors Ġ$ X _ u $ Ġand Ġ$ Y _ i $. ĠWe Ġcall Ġ$ X _ u $ Ġthe ĠâĢ ľ lat ent Ġfactors Ġfor Ġcharacter Ġ$ u $ ", Ġand Ġ$ Y _ i $ Ġthe ĠâĢ ľ lat ent Ġfactors Ġfor ĠH LA Ġ$ i $ ". ĠThe Ġdot Ġproduct Ġof Ġthese Ġtwo Ġvectors Ġproduces Ġa Ġvalue Ġ($ X _ u ^ TY _ i $ ) Ġthat Ġapprox imates Ġ$ P _{ u , i } $ Ġ( see ĠFigure ĠFIG REF 9 ). ĠThis Ġis Ġanalogous Ġto Ġfact oring Ġthe Ġmatrix Ġ$ P $ Ġinto Ġtwo Ġseparate Ġmat rices , Ġwhere Ġone Ġcontains Ġthe Ġlatent Ġfactors Ġfor Ġcharacters , Ġand Ġthe Ġother Ġcontains Ġthe Ġlatent Ġfactors Ġfor ĠHL As . ĠWe Ġfind Ġthat Ġ$ X _ u $ Ġand Ġ$ Y _ i $ Ġbeing Ġ36 - dimensional Ġproduces Ġdecent Ġresults . ĠTo Ġbring Ġ$ X _ u ^ TY _ i $ Ġas Ġclose Ġas Ġpossible Ġto Ġ$ P _{ u , i } $, Ġwe Ġminimize Ġthe Ġfollowing Ġloss Ġfunction Ġusing Ġthe ĠCon jug ate ĠGrad ient ĠMethod ĠB IB REF 30 : </s> The Ġfirst Ġterm Ġpenal izes Ġdifferences Ġbetween Ġthe Ġmodel 's Ġprediction Ġ($ X _ u ^ TY _ i $ ) Ġand Ġthe Ġactual Ġvalue Ġ($ P _{ u , i } $ ). ĠThe Ġsecond Ġterm Ġis Ġan ĠL 2 Ġregular izer Ġto Ġreduce Ġover fitting . ĠWe Ġfind Ġ$\ lambda Ġ= Ġ100 $ Ġprovides Ġdecent Ġresults Ġfor Ġ500 Ġiterations Ġ( see ĠSection ĠSEC REF 26 ). </s> Method ology Ġ: :: ĠCharacter ĠCommunity ĠModule Ġ( CC M ) </s> CC M Ġaims Ġto Ġdivide Ġcharacters Ġ( other Ġthan Ġ$ c _ t $ ) Ġinto Ġa Ġpositive Ġcommunity Ġand Ġa Ġnegative Ġset . ĠWe Ġdefine Ġthis Ġpositive Ġcommunity Ġas Ġcharacters Ġthat Ġare Ġdensely Ġconnected Ġinternally Ġto Ġ$ c _ t $ Ġwithin Ġthe Ġcharacter Ġspace , Ġand Ġthe Ġnegative Ġset Ġas Ġthe Ġremaining Ġcharacters . ĠWe Ġcan Ġthen Ġsample Ġdialogue Ġfrom Ġcharacters Ġin Ġthe Ġnegative Ġset Ġto Ġact Ġas Ġthe Ġdistract ors Ġ( ess entially Ġnegative Ġsamples ) Ġduring ĠL SR M Ġtraining . </s> As Ġcommunity Ġfinding Ġis Ġan Ġill - defined Ġproblem ĠB IB REF 31 , Ġwe Ġchoose Ġto Ġtreat ĠCC M Ġas Ġa Ġsimple Ġund irect ed , Ġun weight ed Ġgraph . ĠWe Ġuse Ġthe Ġvalues Ġlearned Ġin Ġthe ĠCS M Ġfor Ġ$ X _ u $ Ġand Ġ$ Y _ i $ Ġfor Ġvarious Ġvalues Ġof Ġ$ u $ Ġand Ġ$ i $, Ġwhich Ġapproximate Ġthe Ġmatrix Ġ$ P $. ĠSimilar Ġto ĠB IB REF 29 Ġh u 2008 coll abor ative , Ġwe Ġcan Ġcalculate Ġthe Ġcorrelation Ġbetween Ġtwo Ġrows Ġ( and Ġhence Ġtwo Ġcharacters ). </s> We Ġthen Ġemploy Ġa Ġtwo - level Ġconnection Ġrepresentation Ġby Ġranking Ġall Ġcharacters Ġagainst Ġeach Ġother Ġin Ġterms Ġof Ġtheir Ġcorrelation Ġwith Ġ$ c _ t $. ĠFor Ġthe Ġfirst Ġlevel , Ġthe Ġset Ġ$ S ^{ FL } $ Ġis Ġthe Ġtop Ġ10 % Ġ( 45 82 ) Ġmost Ġhighly Ġcorrelated Ġcharacters Ġwith Ġ$ c _ t $ Ġout Ġof Ġthe Ġ45 , 820 Ġtotal Ġother Ġcharacters Ġthat Ġwe Ġhave ĠH LA Ġdata Ġfor . ĠFor Ġthe Ġsecond Ġlevel , Ġfor Ġeach Ġcharacter Ġ$ s _ i $ Ġin Ġ$ S ^{ FL } $, Ġwe Ġdetermine Ġthe Ġ30 Ġmost Ġheavily Ġcorrelated Ġcharacters Ġwith Ġ$ s _ i $ Ġas Ġset Ġ$ S ^{ SL } _ i $. ĠThe Ġpositive Ġset Ġ$ S ^{ pos } $ Ġare Ġthe Ġcharacters Ġwhich Ġare Ġpresent Ġin Ġat Ġleast Ġ10 Ġ$ S ^{ SL } _ i $ Ġsets . ĠWe Ġcall Ġthis Ġvalue Ġ10 Ġthe Ġminimum Ġfrequency . ĠAll Ġother Ġcharacters Ġin Ġour Ġdialogue Ġdataset Ġmake Ġup Ġthe Ġnegative Ġset Ġ$ S ^{ neg } $. ĠThese Ġact Ġas Ġour Ġpositive Ġcommunity Ġand Ġnegative Ġset , Ġrespectively . ĠSee ĠAl gorithm Ġ1 Ġin ĠAppendix ĠA Ġfor Ġdetails , Ġand ĠFigure ĠFIG REF 11 Ġfor Ġan Ġexample . </s> Method ology Ġ: :: ĠLanguage ĠStyle ĠRecovery ĠModule Ġ( LS RM ) </s> LS RM Ġcreates Ġa Ġdialogue Ġagent Ġthat Ġalign s Ġwith Ġobserved Ġcharacteristics Ġof Ġhuman Ġcharacters Ġby Ġusing Ġthe Ġpositive Ġcharacter Ġcommunity Ġand Ġnegative Ġset Ġdetermined Ġin Ġthe ĠCC M , Ġalong Ġwith Ġa Ġdialogue Ġdataset , Ġto Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġ$ c _ t $ Ġwithout Ġits Ġdialogue . ĠWe Ġuse Ġthe ĠB ERT Ġbi - rank er Ġmodel Ġfrom Ġthe ĠFacebook ĠPar l AI Ġframework ĠB IB REF 32 , Ġwhere Ġthe Ġmodel Ġhas Ġthe Ġability Ġto Ġretrieve Ġthe Ġbest Ġresponse Ġout Ġof Ġ20 Ġcandidate Ġresponses . ĠB IB REF 12 , ĠB IB REF 19 , ĠB IB REF 0 Ġchoose Ġ20 Ġcandidate Ġresponses , Ġand Ġfor Ġcomparison Ġpurposes , Ġwe Ġdo Ġthe Ġsame . </s> Method ology Ġ: :: ĠLanguage ĠStyle ĠRecovery ĠModule Ġ( LS RM ) Ġ: :: ĠB ERT </s> BI B REF 28 Ġis Ġfirst Ġtrained Ġon Ġmassive Ġamounts Ġof Ġunl abel ed Ġtext Ġdata . ĠIt Ġjointly Ġconditions Ġon Ġtext Ġon Ġboth Ġthe Ġleft Ġand Ġright , Ġwhich Ġprovides Ġa Ġdeep Ġbi - direction al Ġrepresentation Ġof Ġsentence Ġinference . ĠB ERT Ġis Ġproven Ġto Ġperform Ġwell Ġon Ġa Ġwide Ġrange Ġof Ġtasks Ġby Ġsimply Ġfine - tun ing Ġon Ġone Ġadditional Ġlayer . ĠWe Ġare Ġinterested Ġin Ġits Ġability Ġto Ġpredict Ġthe Ġnext Ġsentence , Ġcalled ĠNext ĠSent ence ĠPrediction . ĠWe Ġperform Ġfurther Ġfine - tun ing Ġon ĠB ERT Ġfor Ġour Ġtarget Ġcharacter Ġlanguage Ġstyle Ġretrieval Ġtask Ġto Ġproduce Ġour ĠL SR M Ġmodel Ġby Ġoptimizing Ġboth Ġthe Ġencoding Ġlayers Ġand Ġthe Ġadditional Ġlayer . ĠWe Ġuse ĠB ERT Ġto Ġcreate Ġvector Ġrepresentations Ġfor Ġthe ĠO BS Ġand Ġfor Ġeach Ġcandidate Ġresponse . ĠBy Ġpassing Ġthe Ġfirst Ġoutput Ġof ĠB ERT 's Ġ12 Ġlayers Ġthrough Ġan Ġadditional Ġlinear Ġlayer , Ġthese Ġrepresentations Ġcan Ġbe Ġobtained Ġas Ġ768 - dimensional Ġsentence - level Ġembed d ings . ĠIt Ġuses Ġthe Ġdot Ġproduct Ġbetween Ġthese Ġembed d ings Ġto Ġscore Ġcandidate Ġresponses Ġand Ġis Ġtrained Ġusing Ġthe Ġranking Ġloss . </s> Method ology Ġ: :: ĠLanguage ĠStyle ĠRecovery ĠModule Ġ( LS RM ) Ġ: :: ĠCandidate Ġresponse Ġselection </s> is Ġsimilar Ġto Ġthe Ġprocedure Ġfrom Ġprevious Ġwork Ġdone Ġon Ġgrounded Ġdialogue Ġagents ĠB IB REF 0 , ĠB IB REF 19 . ĠAlong Ġwith Ġthe Ġground Ġtruth Ġresponse , Ġwe Ġrandomly Ġsample Ġ19 Ġdistract or Ġresponses Ġfrom Ġother Ġcharacters Ġfrom Ġa Ġuniform Ġdistribution Ġof Ġcharacters , Ġand Ġcall Ġthis Ġprocess Ġuniform Ġcharacter Ġsampling . ĠBased Ġon Ġour Ġobservations , Ġthis Ġrandom Ġsampling Ġprovides Ġmultiple Ġcontext Ġcorrect Ġresponses . ĠHence , Ġthe ĠB ERT Ġbi - rank er Ġmodel Ġis Ġtrained Ġby Ġlearning Ġto Ġchoose Ġcontext Ġcorrect Ġresponses , Ġand Ġthe Ġmodel Ġlearns Ġto Ġrecover Ġa Ġdomain - general Ġlanguage Ġmodel Ġthat Ġincludes Ġtraining Ġon Ġevery Ġcharacter . ĠThis Ġresults Ġin Ġa ĠUniform ĠModel Ġthat Ġcan Ġselect Ġcontext Ġcorrect Ġresponses , Ġbut Ġnot Ġresponses Ġcorresponding Ġto Ġa Ġtarget Ġcharacter Ġwith Ġspecific ĠHL As . </s> We Ġthen Ġfine - t une Ġon Ġthe Ġabove Ġmodel Ġto Ġproduce Ġour ĠL SR M Ġmodel Ġwith Ġa Ġmodification : Ġwe Ġrandomly Ġsample Ġthe Ġ19 Ġdistract or Ġresponses Ġfrom Ġonly Ġthe Ġnegative Ġcharacter Ġset Ġinstead . ĠWe Ġchoose Ġthe Ġresponses Ġthat Ġhave Ġsimilar Ġgram matical Ġstructures Ġand Ġsemantics Ġto Ġthe Ġground Ġtruth Ġresponse , Ġand Ġcall Ġthis Ġprocess Ġnegative Ġcharacter Ġsampling . ĠThis Ġguides Ġthe Ġmodel Ġaway Ġfrom Ġthe Ġlanguage Ġstyle Ġof Ġthese Ġnegative Ġcharacters Ġto Ġimprove Ġperformance Ġat Ġretrieving Ġresponses Ġfor Ġtarget Ġcharacters Ġwith Ġspecific ĠHL As . ĠOur Ġresults Ġdemonstrate Ġhigher Ġaccuracy Ġat Ġretrieving Ġthe Ġcorrect Ġresponse Ġfrom Ġcharacter Ġ$ c _ t $, Ġwhich Ġis Ġthe Ġground Ġtruth . </s> Exper iment Ġ: :: ĠDialogue ĠDat as et </s> To Ġtrain Ġthe ĠUniform ĠModel Ġand ĠL SR M , Ġwe Ġcollect Ġdialog ues Ġfrom Ġ327 Ġmajor Ġcharacters Ġ( a Ġsubset Ġof Ġthe Ġ45 , 8 21 Ġcharacters Ġwe Ġhave ĠH LA Ġdata Ġfor ) Ġin Ġ38 ĠTV Ġshows Ġfrom Ġvarious Ġexisting Ġsources Ġof Ġclean Ġdata Ġon Ġthe Ġinternet , Ġresulting Ġin Ġa Ġtotal Ġof Ġ1 , 04 2 , 647 Ġdialogue Ġlines . ĠWe Ġuse Ġa Ġsetup Ġsimilar Ġto Ġthe ĠPersona - Chat Ġdataset ĠB IB REF 0 Ġand ĠCornell ĠMovie - Dial ogs ĠCorpus ĠB IB REF 33 , Ġas Ġour Ġcollected Ġdialog ues Ġare Ġalso Ġpaired Ġin Ġterms Ġof Ġvalid Ġconversations . ĠSee ĠFigure ĠFIG REF 1 Ġfor Ġan Ġexample Ġof Ġthese Ġdialogue Ġlines . </s> Exper iment Ġ: :: ĠH LA ĠObs ervation ĠGuid ance Ġ( HL A - OG ) </s> We Ġdefine ĠH LA ĠObs ervation ĠGuid ance Ġ( HL A - OG ) Ġas Ġexplicitly Ġpassing Ġa Ġsmall Ġsubset Ġof Ġthe Ġmost Ġimportant ĠHL As Ġof Ġa Ġgiven Ġcharacter Ġas Ġpart Ġof Ġthe ĠO BS Ġrather Ġthan Ġjust Ġan Ġinitial Ġline Ġof Ġdialogue . ĠThis Ġis Ġadapted Ġfrom Ġthe Ġprocess Ġused Ġin ĠB IB REF 0 Ġz hang 2018 personal izing Ġand ĠB IB REF 10 Ġwolf 2019 trans f ert rans fo Ġwhich Ġwe Ġcall ĠPersona ĠProf iling . ĠSpecifically , Ġwe Ġpass Ġfour ĠHL As Ġthat Ġare Ġrandomly Ġdrawn Ġfrom Ġthe Ġtop Ġ40 Ġmost Ġimportant ĠHL As Ġof Ġthe Ġcharacter . ĠWe Ġuse ĠH LA - OG Ġduring Ġtraining Ġof Ġthe ĠL SR M Ġand Ġtesting Ġof Ġall Ġmodels . ĠThis Ġis Ġbecause Ġthe Ġbas elines Ġ( see ĠSection ĠSEC REF 31 ) Ġalready Ġfollow Ġa Ġsimilar Ġprocess Ġ( Person a ĠProf iling ) Ġfor Ġtraining . ĠFor Ġthe ĠUniform ĠModel , Ġwe Ġtrain Ġusing ĠNext ĠSent ence ĠPrediction Ġ( see ĠSection ĠSEC REF 12 ). ĠFor Ġtesting , ĠH LA - OG Ġis Ġnecessary Ġas Ġit Ġprovides Ġinformation Ġabout Ġwhich ĠHL As Ġthe Ġmodels Ġshould Ġattempt Ġto Ġimitate Ġin Ġtheir Ġresponse Ġselection . ĠJust Ġpassing Ġan Ġinitial Ġline Ġof Ġdialogue Ġrepl icates Ġa Ġtypical Ġdialogue Ġresponse Ġtask Ġwithout ĠHL As . ĠSee ĠTable ĠT AB REF 19 . ĠFurther , Ġwe Ġalso Ġtest Ġour ĠL SR M Ġby Ġexplicitly Ġpassing Ġfour ĠHL As Ġof Ġ` none ' Ġalong Ġwith Ġthe Ġinitial Ġline Ġof Ġdialogue Ġas Ġthe ĠO BS Ġ( No ĠH LA - OG Ġin ĠTable ĠT AB REF 19 ). </s> Exper iment Ġ: :: ĠTraining ĠDetails Ġ: :: ĠB ERT Ġbi - rank er </s> is Ġtrained Ġby Ġus Ġon Ġthe ĠPersona - Chat Ġdataset Ġfor Ġthe ĠConv AI 2 Ġchallenge . ĠSimilar Ġto ĠB IB REF 0 Ġz hang 2018 personal izing , Ġwe Ġcap Ġthe Ġlength Ġof Ġthe ĠO BS Ġat Ġ360 Ġtokens Ġand Ġthe Ġlength Ġof Ġeach Ġcandidate Ġresponse Ġat Ġ72 Ġtokens . ĠWe Ġuse Ġa Ġbatch Ġsize Ġof Ġ64 , Ġlearning Ġrate Ġof Ġ5 e - 5 , Ġand Ġperform Ġwarm - up Ġupdates Ġfor Ġ100 Ġiterations . ĠThe Ġlearning Ġrate Ġsched uler Ġuses ĠSG D Ġoptim izer Ġwith ĠN ester ov 's Ġaccelerated Ġgradient Ġdescent ĠB IB REF 34 Ġand Ġis Ġset Ġto Ġhave Ġa Ġdecay Ġof Ġ0 . 4 Ġand Ġto Ġreduce Ġon Ġplateau . </s> Exper iment Ġ: :: ĠTraining ĠDetails Ġ: :: ĠUniform ĠModel </s> is Ġproduced Ġby Ġfin et uning Ġthe ĠB ERT Ġbi - rank er Ġon Ġthe Ġdialogue Ġdata Ġdiscussed Ġin ĠSection ĠSEC REF 15 Ġusing Ġuniform Ġcharacter Ġsampling . ĠWe Ġuse Ġthe Ġsame Ġhyper param eters Ġas Ġthe ĠB ERT Ġbi - rank er Ġalong Ġwith Ġhalf - pre cision Ġoperations Ġ( i . e . Ġfloat 16 Ġoperations ) Ġto Ġincrease Ġbatch Ġsize Ġas Ġrecommended ĠB IB REF 7 . </s> Exper iment Ġ: :: ĠTraining ĠDetails Ġ: :: ĠL SR M </s> is Ġproduced Ġby Ġfin et uning Ġon Ġthe ĠUniform ĠModel Ġdiscussed Ġabove Ġusing Ġnegative Ġcharacter Ġsampling . ĠWe Ġuse Ġthe Ġsame Ġhyper param eters Ġas Ġthe ĠB ERT Ġbi - rank er Ġalong Ġwith Ġhalf - pre cision Ġoperations Ġ( i . e . Ġfloat 16 Ġoperations ) Ġto Ġincrease Ġbatch Ġsize Ġas Ġrecommended . </s> E val uation Ġ: :: ĠCS M ĠEvaluation </s> We Ġbegin Ġby Ġevaluating Ġthe Ġability Ġof Ġthe ĠCS M Ġcomponent Ġof Ġour Ġsystem Ġto Ġcorrectly Ġgenerate Ġthe Ġcharacter Ġspace . ĠTo Ġdo Ġso , Ġduring Ġtraining , Ġ30 % Ġof Ġthe Ġcharacter - HL A Ġpairs Ġ( which Ġare Ġeither Ġ0 Ġor Ġ1 ) Ġare Ġmasked , Ġand Ġthis Ġis Ġused Ġas Ġa Ġvalidation Ġset Ġ( see ĠFigure ĠFIG REF 9 ). ĠFor Ġeach Ġcharacter Ġ$ c $, Ġthe Ġmodel Ġgenerates Ġa Ġlist Ġof Ġthe Ġ12 , 815 Ġunique ĠHL As Ġranked Ġsimilarly Ġto ĠB IB REF 29 Ġh u 2008 coll abor ative Ġfor Ġ$ c $. ĠWe Ġlook Ġat Ġthe Ġrecall Ġof Ġour ĠCS M Ġmodel , Ġwhich Ġmeasures Ġthe Ġpercentage Ġof Ġtotal Ġground Ġtruth ĠHL As Ġ( over Ġall Ġcharacters Ġ$ c $ ) Ġpresent Ġwithin Ġthe Ġtop ĠN Ġranked ĠHL As Ġfor Ġall Ġ$ c $ Ġby Ġour Ġmodel . ĠThat Ġis : </s> where Ġ$ HL A _{ c } ^{ gt } $ Ġare Ġthe Ġground Ġtruth ĠHL As Ġfor Ġ$ c $, Ġand Ġ$ HL A _{ c } ^{ t N } $ Ġare Ġthe Ġtop ĠN Ġranked ĠHL As Ġby Ġthe Ġmodel Ġfor Ġ$ c $. ĠWe Ġuse Ġ$ N Ġ= Ġ100 $, Ġand Ġour Ġmodel Ġachieves Ġ25 . 08 % Ġrecall . </s> To Ġinspect Ġthe ĠCS M Ġperformance , Ġwe Ġuse Ġthe ĠT - dist ributed ĠSt och astic ĠNeighbor ĠEmb edd ing Ġ( t - S NE ) ĠB IB REF 35 Ġto Ġreduce Ġeach Ġhigh - dimension ality Ġdata Ġpoint Ġto Ġtwo - dim ensions Ġvia ĠK ull back - Le ib ler ĠDiver gence ĠB IB REF 36 . ĠThis Ġallows Ġus Ġto Ġmap Ġour Ġcharacter Ġspace Ġinto Ġtwo - dim ensions , Ġwhere Ġsimilar Ġcharacters Ġfrom Ġour Ġembed ding Ġspace Ġhave Ġhigher Ġprobability Ġof Ġbeing Ġmapped Ġclose Ġby . ĠWe Ġsampled Ġcharacters Ġfrom Ġfour Ġdifferent Ġgroups Ġor Ġregions . ĠAs Ġseen Ġin ĠFigure ĠFIG REF 4 , Ġour Ġlearned Ġcharacter Ġspace Ġeffectively Ġgroups Ġthese Ġcharacters , Ġas Ġsimilar Ġcharacters Ġare Ġadjacent Ġto Ġone Ġanother Ġin Ġfour Ġregions . </s> E val uation Ġ: :: ĠAutomatic ĠEvaluation ĠSetup Ġ: :: ĠFive - F old ĠCross ĠVal idation </s> is Ġused Ġfor Ġtraining Ġand Ġtesting Ġof Ġthe ĠUniform ĠModel Ġand ĠL SR M . ĠThe Ġfolds Ġare Ġdivided Ġrandomly Ġby Ġthe ĠTV Ġshows Ġin Ġour Ġdialogue Ġdata . ĠWe Ġuse Ġthe Ġdialogue Ġdata Ġfor Ġ80 % Ġof Ġthese Ġshows Ġas Ġthe Ġfour - f olds Ġfor Ġtraining , Ġand Ġthe Ġdialogue Ġdata Ġfor Ġthe Ġremaining Ġ20 % Ġas Ġthe Ġfifth - fold Ġfor Ġvalidation / testing . ĠThe Ġdialogue Ġdata Ġused Ġis Ġdiscussed Ġin ĠSection ĠSEC REF 15 . ĠThis Ġensures Ġno Ġmatter Ġhow Ġour Ġdata Ġis Ġdistributed , Ġeach Ġpart Ġof Ġit Ġis Ġtested , Ġallowing Ġour Ġevaluation Ġto Ġbe Ġmore Ġrobust Ġto Ġdifferent Ġcharacters . ĠSee ĠAppendix ĠC Ġfor Ġfive - fold Ġcross Ġvalidation Ġdetails Ġand Ġstatistics . </s> E val uation Ġ: :: ĠAutomatic ĠEvaluation ĠSetup Ġ: :: ĠFive ĠEvaluation ĠCharacters </s> are Ġchosen , Ġone Ġfrom Ġeach Ġof Ġthe Ġfive Ġtesting Ġsets Ġabove . ĠEach Ġis Ġa Ġwell - known Ġcharacter Ġfrom Ġa Ġseparate ĠTV Ġshow , Ġand Ġacts Ġas Ġa Ġtarget Ġcharacter Ġ$ c _ t $ Ġfor Ġevaluation Ġof Ġevery Ġmodel . ĠWe Ġchoose ĠSheldon ĠCooper Ġfrom ĠThe ĠBig ĠBang ĠTheory , ĠJean - Luc ĠPicard Ġfrom ĠStar ĠTrek , ĠMonica ĠG eller Ġfrom ĠFriends , ĠGil ĠGr iss om Ġfrom ĠCSI , Ġand ĠM arge ĠSimpson Ġfrom ĠThe ĠSimpsons . ĠWe Ġchoose Ġcharacters Ġof Ġsignificantly Ġdifferent Ġidentities Ġand Ġprofiles Ġ( int elligent Ġscientist , Ġship Ġcaptain , Ġoutgoing Ġfriend , Ġpolice Ġleader , Ġand Ġresponsible Ġmother , Ġrespectively ) Ġfrom Ġshows Ġof Ġa Ġvariety Ġof Ġgenres Ġto Ġensure Ġthat Ġwe Ġcan Ġsuccessfully Ġrecover Ġthe Ġlanguage Ġstyles Ġof Ġvarious Ġtypes Ġof Ġcharacters . ĠWe Ġchoose Ġwell - known Ġcharacters Ġbecause Ġhumans Ġrequire Ġknowledge Ġon Ġthe Ġcharacters Ġthey Ġare Ġevaluating Ġ( see ĠSection ĠSEC REF 40 ). </s> For Ġeach Ġof Ġthese Ġfive Ġevaluation Ġcharacters , Ġall Ġthe Ġdialogue Ġlines Ġfrom Ġthe Ġcharacter Ġact Ġas Ġthe Ġground Ġtruth Ġresponses . ĠThe Ġinitial Ġdialogue Ġlines Ġare Ġthe Ġcorresponding Ġdialogue Ġlines Ġto Ġwhich Ġthese Ġground Ġtruth Ġresponses Ġare Ġresponding . ĠFor Ġeach Ġinitial Ġdialogue Ġline , Ġwe Ġrandomly Ġsample Ġ19 Ġother Ġcandidate Ġresponses Ġfrom Ġthe Ġassociated Ġtesting Ġset Ġusing Ġuniform Ġcharacter Ġsampling . ĠNote Ġthat Ġthis Ġis Ġfor Ġevaluation , Ġand Ġhence Ġwe Ġuse Ġthe Ġsame Ġuniform Ġcharacter Ġsampling Ġmethod Ġfor Ġall Ġmodels Ġincluding ĠAL O HA . ĠThe Ġuse Ġof Ġnegative Ġcharacter Ġsampling Ġis Ġonly Ġin ĠAL O HA 's Ġtraining . </s> E val uation Ġ: :: ĠBas elines </s> We Ġcompare Ġagainst Ġfour Ġdialogue Ġsystem Ġbas elines : ĠK v mem nn , ĠFeed ĠYourself , ĠPoly - enc oder , Ġand Ġa ĠB ERT Ġbi - rank er Ġbaseline Ġtrained Ġon Ġthe ĠPersona - Chat Ġdataset Ġusing Ġthe Ġsame Ġtraining Ġhyper param eters Ġ( including Ġlearning Ġrate Ġsched uler Ġand Ġlength Ġca pping Ġsettings ) Ġdescribed Ġin ĠSection ĠSEC REF 20 . ĠFor Ġthe Ġfirst Ġthree Ġmodels , Ġwe Ġuse Ġthe Ġprovided Ġpret rained Ġ( on ĠPersona - Chat ) Ġmodels . ĠWe Ġevaluate Ġall Ġfour Ġon Ġour Ġfive Ġevaluation Ġcharacters Ġdiscussed Ġin ĠSection ĠSEC REF 28 . </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: ĠHits @ n / N </s> is Ġthe Ġaccuracy Ġof Ġthe Ġcorrect Ġground Ġtruth Ġresponse Ġbeing Ġwithin Ġthe Ġtop Ġ$ n $ Ġranked Ġcandidate Ġresponses Ġout Ġof Ġ$ N $ Ġtotal Ġcandidates . ĠWe Ġmeasure ĠHits @ 1 / 20 , ĠHits @ 5 / 20 , Ġand ĠHits @ 10 / 20 . </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: ĠMean ĠRank </s> is Ġthe Ġaverage Ġrank Ġthat Ġa Ġmodel Ġassigns Ġthe Ġground Ġtruth Ġresponse Ġamong Ġthe Ġ20 Ġtotal Ġcandidates . </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: ĠMean ĠRec ip rocal ĠRank Ġ( MR R ) </s> BI B REF 37 Ġlooks Ġat Ġthe Ġmean Ġof Ġthe Ġmultipl icative Ġin vers es Ġof Ġthe Ġrank Ġof Ġeach Ġcorrect Ġanswer Ġout Ġof Ġa Ġsample Ġof Ġqueries Ġ$ Q $ : </s> where Ġ$ rank _ i $ Ġrefers Ġto Ġthe Ġrank Ġposition Ġof Ġthe Ġcorrect Ġresponse Ġfor Ġthe Ġ$ i $ - th Ġquery , Ġand Ġ$ | Q | $ Ġrefers Ġto Ġthe Ġtotal Ġnumber Ġof Ġqueries Ġin Ġ$ Q $. </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: Ġ@ ! ST ART @ $ F _ 1 $ @ ! END @ - score </s> equ als Ġ$ 2 Ġ* Ġ\ frac { pre cision * rec all }{ pre cision + rec all } $. ĠFor Ġdialogue , Ġprecision Ġis Ġthe Ġfraction Ġof Ġwords Ġin Ġthe Ġchosen Ġresponse Ġcontained Ġin Ġthe Ġground Ġtruth Ġresponse , Ġand Ġrecall Ġis Ġthe Ġfraction Ġof Ġwords Ġin Ġthe Ġground Ġtruth Ġresponse Ġcontained Ġin Ġthe Ġchosen Ġresponse . </s> E val uation Ġ: :: ĠKey ĠEvaluation ĠMet rics Ġ: :: ĠB LE U </s> BI B REF 38 Ġgenerally Ġindicates Ġhow Ġclose Ġtwo Ġpieces Ġof Ġtext Ġare Ġin Ġcontent Ġand Ġstructure , Ġwith Ġhigher Ġvalues Ġindicating Ġgreater Ġsimilarity . ĠWe Ġreport Ġour Ġfinal ĠB LE U Ġscores Ġas Ġthe Ġaverage Ġscores Ġof Ġ1 Ġto Ġ4 - gram s . </s> E val uation Ġ: :: ĠHuman ĠEvaluation ĠSetup </s> We Ġconduct Ġa Ġhuman Ġevaluation Ġwith Ġ12 Ġparticipants , Ġ8 Ġmale Ġand Ġ4 Ġfemale , Ġwho Ġare Ġaffiliated Ġproject Ġresearchers Ġaged Ġ20 - 39 Ġat Ġthe ĠUniversity Ġof Ġ[ AN ON ]. ĠWe Ġchoose Ġthe Ġsame Ġfive Ġevaluation Ġcharacters Ġas Ġin ĠSection ĠSEC REF 28 . ĠTo Ġcontrol Ġbias , Ġeach Ġparticipant Ġevaluates Ġone Ġor Ġtwo Ġcharacters . ĠFor Ġeach Ġcharacter , Ġwe Ġrandomly Ġselect Ġ10 Ġtesting Ġsamples Ġ( each Ġincludes Ġan Ġinitial Ġline Ġof Ġdialogue Ġalong Ġwith Ġ20 Ġcandidate Ġresponses , Ġone Ġof Ġwhich Ġis Ġthe Ġground Ġtruth ) Ġfrom Ġthe Ġsame Ġtesting Ġdata Ġfor Ġthe Ġautomatic Ġevaluation Ġdiscussed Ġin ĠSection ĠSEC REF 28 . </s> These Ġten Ġsamples Ġmake Ġup Ġa Ġsingle Ġquestionnaire Ġpresented Ġin Ġfull Ġto Ġeach Ġparticipant Ġevaluating Ġthe Ġcorresponding Ġcharacter , Ġand Ġthe Ġparticipant Ġis Ġasked Ġto Ġselect Ġthe Ġsingle Ġtop Ġresponse Ġthey Ġthink Ġthe Ġcharacter Ġwould Ġmost Ġlikely Ġrespond Ġwith Ġfor Ġeach Ġof Ġthe Ġten Ġinitial Ġdialogue Ġlines . ĠSee ĠFigure ĠFIG REF 41 Ġfor Ġan Ġexample . ĠWe Ġmask Ġany Ġcharacter Ġnames Ġwithin Ġthe Ġcandidate Ġresponses Ġto Ġprevent Ġhuman Ġparticipants Ġfrom Ġusing Ġnames Ġto Ġidentify Ġwhich Ġshow Ġthe Ġresponse Ġis Ġfrom . </s> Each Ġcandidate Ġis Ġpresc reen ed Ġto Ġensure Ġthey Ġhave Ġsufficient Ġknowledge Ġof Ġthe Ġcharacter Ġto Ġbe Ġa Ġparticipant . ĠWe Ġask Ġthree Ġpresc reen ing Ġquestions Ġwhere Ġthe Ġparticipant Ġhas Ġto Ġidentify Ġan Ġimage , Ġrelationship , Ġand Ġoccupation Ġof Ġthe Ġcharacter . ĠAll Ġ12 Ġof Ġour Ġparticipants Ġpassed Ġthe Ġthe Ġpresc reen ing . </s> Results Ġand ĠAnalysis Ġ: :: ĠEvaluation ĠResults </s> Table ĠT AB REF 44 Ġshows Ġaverage Ġresults Ġof Ġour Ġautomatic Ġand Ġhuman Ġevaluations . ĠTable ĠT AB REF 45 Ġshows Ġaverage ĠHits @ 1 / 20 Ġscores Ġby Ġevaluation Ġcharacter . ĠSee ĠAppendix ĠF Ġfor Ġdetailed Ġevaluation Ġresults . ĠAL O HA Ġis Ġthe Ġmodel Ġwith ĠH LA - OG Ġduring Ġtraining Ġand Ġtesting , Ġand ĠAL O HA Ġ( No ĠH LA - OG ) Ġis Ġthe Ġmodel Ġwith ĠH LA - OG Ġduring Ġtraining Ġbut Ġtested Ġwith Ġthe Ġfour ĠHL As Ġin Ġthe ĠO BS Ġmarked Ġas Ġ` none ' Ġ( see ĠSection ĠSEC REF 17 ). ĠSee ĠAppendix ĠG Ġfor Ġdemo Ġinteractions Ġbetween Ġa Ġhuman , ĠB ERT Ġbi - rank er Ġbaseline , Ġand ĠAL O HA Ġfor Ġall Ġfive Ġevaluation Ġcharacters . </s> Results Ġand ĠAnalysis Ġ: :: ĠEvaluation ĠChallenges </s> The Ġevaluation Ġof Ġour Ġtask Ġ( ret rieving Ġthe Ġlanguage Ġstyle Ġof Ġa Ġspecific Ġcharacter ) Ġis Ġchallenging Ġand Ġhence Ġthe Ġfive - fold Ġcross Ġvalidation Ġis Ġnecessary Ġfor Ġthe Ġfollowing Ġreasons : </s> The Ġability Ġto Ġchoose Ġa Ġcontext Ġcorrect Ġresponse Ġwithout Ġattributes Ġof Ġspecific Ġcharacters Ġmay Ġbe Ġhard Ġto Ġseparate Ġfrom Ġour Ġtarget Ġmetric , Ġwhich Ġis Ġthe Ġability Ġto Ġretrieve Ġthe Ġcorrect Ġresponse Ġof Ġa Ġtarget Ġcharacter Ġby Ġits ĠHL As . ĠHowever , Ġfrom Ġmanual Ġobservation , Ġwe Ġnoticed Ġthat Ġin Ġthe Ġ20 Ġchosen Ġcandidate Ġresponses , Ġthere Ġare Ġtypically Ġnumerous Ġcontext Ġcorrect Ġresponses , Ġbut Ġonly Ġone Ġground Ġtruth Ġfor Ġthe Ġtarget Ġcharacter Ġ( for Ġan Ġexample , Ġsee ĠFigure ĠFIG REF 41 ). ĠHence , Ġa Ġmodel Ġthat Ġonly Ġchooses Ġdialogue Ġbased Ġon Ġcontext Ġis Ġdistinguish able Ġfrom Ġone Ġthat Ġlearns ĠHL As . </s> Ret rieving Ġresponses Ġfor Ġthe Ġtarget Ġcharacter Ġdepends Ġon Ġthe Ġother Ġcandidate Ġresponses . ĠFor Ġexample , Ġdialogue Ġretrieval Ġperformance Ġfor ĠGr iss om Ġfrom ĠCSI , Ġwhich Ġis Ġa Ġcrime / police Ġcontext , Ġis Ġhigher Ġthan Ġother Ġevaluation Ġcharacters Ġ( see ĠTable ĠT AB REF 45 ), Ġpotentially Ġdue Ġto Ġother Ġcandidate Ġresponses Ġnot Ġfalling Ġwithin Ġthe Ġsame Ġcrime / police Ġcontext . </s> Results Ġand ĠAnalysis Ġ: :: ĠPerformance : ĠAL O HA Ġvs . ĠHumans </s> As Ġobserved Ġfrom ĠTable ĠT AB REF 44 , ĠAL O HA Ġhas Ġa Ġperformance Ġrelatively Ġclose Ġto Ġhumans . ĠHuman ĠHits @ 1 / 20 Ġscores Ġhave Ġa Ġmean Ġof Ġ40 . 67 % Ġand Ġa Ġmedian Ġover Ġcharacters Ġof Ġ40 %. ĠThe Ġlimited Ġhuman Ġevaluation Ġsample Ġsize Ġlimits Ġwhat Ġcan Ġbe Ġinferred , Ġbut Ġit Ġindicates Ġthat Ġthe Ġproblem Ġis Ġsolved Ġto Ġthe Ġextent Ġthat ĠAL O HA Ġis Ġable Ġperform Ġrelatively Ġclose Ġto Ġhumans Ġon Ġaverage . ĠNotice Ġthat Ġeven Ġhumans Ġdo Ġnot Ġperform Ġextremely Ġwell , Ġdemonstrating Ġthat Ġthis Ġtask Ġof Ġcharacter Ġbased Ġdialogue Ġretrieval Ġis Ġmore Ġdifficult Ġthan Ġtypical Ġdialogue Ġretrieval Ġtasks ĠB IB REF 19 , ĠB IB REF 12 . </s> Looking Ġmore Ġclosely Ġat Ġeach Ġcharacter Ġfrom ĠTable ĠT AB REF 45 , Ġwe Ġcan Ġsee Ġthat Ġhuman Ġevaluation Ġscores Ġare Ġhigher Ġfor ĠSheldon Ġand ĠGr iss om . ĠThis Ġmay Ġbe Ġdue Ġto Ġthese Ġcharacters Ġhaving Ġmore Ġdistinct Ġpersonalities , Ġmaking Ġthem Ġmore Ġmemorable . </s> We Ġalso Ġlook Ġat ĠPearson Ġcorrelation Ġvalues Ġof Ġthe ĠHits @ 1 / 20 Ġscores Ġacross Ġthe Ġfive Ġevaluation Ġcharacters . ĠFor Ġhuman Ġversus ĠUniform ĠModel , Ġthis Ġis Ġ- 0 . 469 4 , Ġdemonstrating Ġthat Ġthe ĠUniform ĠModel , Ġwithout Ġknowledge Ġof ĠHL As , Ġfails Ġto Ġimitate Ġhuman Ġimpressions . ĠFor Ġhuman Ġversus ĠAL O HA , Ġthis Ġis Ġ0 . 42 50 , Ġdemonstrating Ġthat Ġour Ġsystem Ġis Ġable Ġto Ġretrieve Ġcharacter Ġresponses Ġsomewhat Ġsimilarly Ġto Ġhuman Ġimpressions . ĠLastly , Ġfor Ġhuman Ġversus Ġthe Ġdifference Ġin Ġscores Ġbetween ĠAL O HA Ġand ĠUniform ĠModel , Ġthis Ġis Ġ0 . 78 15 . ĠThe Ġdifference Ġbetween ĠAL O HA Ġand Ġthe ĠUniform ĠModel , Ġwhich Ġis Ġbased Ġon Ġthe Ġadditional Ġknowledge Ġof Ġthe ĠHL As , Ġis Ġhence Ġshown Ġto Ġimprove Ġupon Ġthe ĠUniform ĠModel Ġsimilarly Ġto Ġhuman Ġimpressions . ĠThis Ġdemonstrates Ġthat ĠHL As Ġare Ġindeed Ġan Ġaccurate Ġmethod Ġof Ġmodeling Ġhuman Ġimpressions Ġof Ġcharacter Ġattributes , Ġand Ġalso Ġdemonstrates Ġthat Ġour Ġsystem , ĠAL O HA , Ġis Ġable Ġto Ġeffectively Ġuse Ġthese ĠHL As Ġto Ġimprove Ġupon Ġdialogue Ġretrieval Ġperformance . </s> Results Ġand ĠAnalysis Ġ: :: ĠPerformance : ĠAL O HA Ġvs . ĠBas elines </s> AL O HA , Ġcombined Ġwith Ġthe ĠHL As Ġand Ġdialogue Ġdataset , Ġachieves Ġa Ġsignificant Ġimprovement Ġon Ġthe Ġtarget Ġcharacter Ġlanguage Ġstyle Ġretrieval Ġtask Ġcompared Ġto Ġthe Ġbaseline Ġopen - domain Ġchat bot Ġmodels . ĠAs Ġobserved Ġfrom ĠTable ĠT AB REF 44 , ĠAL O HA Ġachieves Ġa Ġsignificant Ġboost Ġin ĠHits @ n / N Ġaccuracy Ġand Ġother Ġmetrics Ġfor Ġretrieving Ġthe Ġcorrect Ġresponse Ġof Ġfive Ġdiverse Ġcharacters Ġwith Ġdifferent Ġidentities Ġ( see ĠSection ĠSEC REF 28 ). </s> Results Ġand ĠAnalysis Ġ: :: ĠPerformance : ĠAL O HA Ġvs . ĠUniform ĠModel </s> We Ġobserve Ġa Ġnoticeable Ġimprovement Ġin Ġperformance Ġbetween ĠAL O HA Ġand Ġthe ĠUniform ĠModel Ġin Ġrecovering Ġthe Ġlanguage Ġstyles Ġof Ġspecific Ġcharacters Ġthat Ġis Ġconsistent Ġacross Ġall Ġfive Ġfolds Ġ( see ĠTables ĠT AB REF 44 Ġand ĠT AB REF 45 ), Ġindicating Ġthat Ġlack Ġof Ġknowledge Ġof ĠHL As Ġlimits Ġthe Ġability Ġof Ġthe Ġmodel Ġto Ġsuccessfully Ġrecover Ġthe Ġlanguage Ġstyle Ġof Ġspecific Ġcharacters . ĠWe Ġclaim Ġthat , Ġto Ġthe Ġbest Ġof Ġour Ġknowledge , Ġwe Ġhave Ġmade Ġthe Ġfirst Ġstep Ġin Ġusing ĠH LA - based Ġcharacter Ġdialogue Ġclust ering Ġto Ġimprove Ġupon Ġpersonality Ġlearning Ġfor Ġchat bots . </s> AL O HA Ġdemonstrates Ġan Ġaccuracy Ġboost Ġfor Ġall Ġfive Ġevaluation Ġcharacters , Ġshowing Ġthat Ġthe Ġsystem Ġis Ġrobust Ġand Ġstable Ġand Ġhas Ġthe Ġability Ġto Ġrecover Ġthe Ġdialogue Ġstyles Ġof Ġfictional Ġcharacters Ġregardless Ġof Ġthe Ġcharacter 's Ġprofile Ġand Ġidentity , Ġgenre Ġof Ġthe Ġshow , Ġand Ġcontext Ġof Ġthe Ġdialogue . </s> Results Ġand ĠAnalysis Ġ: :: ĠPerformance : ĠH LA - OG </s> As Ġobserved Ġfrom ĠTable ĠT AB REF 44 , ĠAL O HA Ġperforms Ġslightly Ġbetter Ġoverall Ġcompared Ġto ĠAL O HA Ġ( No ĠH LA - OG ). ĠTable ĠT AB REF 45 Ġshows Ġthat Ġthis Ġslight Ġperformance Ġincrease Ġis Ġconsistent Ġacross Ġfour Ġof Ġthe Ġfive Ġevaluation Ġcharacters . ĠIn Ġthe Ġcase Ġof ĠSheldon , Ġthe ĠH LA - OG Ġmodel Ġperforms Ġa Ġbit Ġworse . ĠThis Ġis Ġpossibly Ġdue Ġto Ġthe Ġlarge Ġnumber Ġof ĠSheldon 's ĠHL As Ġ( 217 ) Ġcompared Ġto Ġthe Ġother Ġfour Ġevaluation Ġcharacters Ġ( average Ġof Ġ93 . 75 ), Ġalong Ġwith Ġthe Ġlimited Ġamount Ġof ĠHL As Ġwe Ġare Ġusing Ġfor Ġguidance Ġdue Ġto Ġthe Ġmodels ' Ġlimited Ġmemory . ĠIn Ġgeneral , ĠH LA ĠObs ervation ĠGuid ance Ġduring Ġtesting Ġappears Ġto Ġimprove Ġupon Ġthe Ġperformance Ġof ĠAL O HA , Ġbut Ġthis Ġimprovement Ġis Ġminimal . </s> Conclusion Ġand ĠFuture ĠWork </s> We Ġproposed ĠHuman ĠLevel ĠAttributes Ġ( HL As ) Ġas Ġa Ġnovel Ġapproach Ġto Ġmodel Ġhuman - like Ġattributes Ġof Ġcharacters , Ġand Ġcollected Ġa Ġlarge Ġvolume Ġof Ġdialogue Ġdata Ġfor Ġvarious Ġcharacters Ġwith Ġcomplete Ġand Ġrobust Ġprofiles . ĠWe Ġalso Ġproposed Ġand Ġevaluated Ġa Ġsystem , ĠAL O HA , Ġthat Ġuses ĠHL As Ġto Ġrecommend Ġtailored Ġresponses Ġtrace able Ġto Ġspecific Ġcharacters , Ġand Ġdemonstrated Ġits Ġout performance Ġof Ġthe Ġbas elines Ġand Ġability Ġto Ġeffectively Ġrecover Ġlanguage Ġstyles Ġof Ġvarious Ġcharacters , Ġshowing Ġpromise Ġfor Ġlearning Ġcharacter Ġor Ġpersonality Ġstyles . ĠAL O HA Ġwas Ġalso Ġshown Ġto Ġbe Ġstable Ġregardless Ġof Ġthe Ġcharacter 's Ġidentity , Ġgenre Ġof Ġshow , Ġand Ġcontext Ġof Ġdialogue . </s> Pot ential Ġdirections Ġfor Ġfuture Ġwork Ġinclude Ġtraining ĠAL O HA Ġwith Ġa Ġmulti - turn Ġresponse Ġapproach ĠB IB REF 0 Ġthat Ġtracks Ġdialogue Ġover Ġmultiple Ġresponses , Ġas Ġwe Ġcould Ġnot Ġacquire Ġmulti - turn Ġdialogue Ġdata Ġfor ĠTV Ġshows . ĠAnother Ġpotential Ġis Ġthe Ġmodeling Ġof Ġthe Ġdialog Ġcounterpart Ġ( e . g . Ġthe Ġdialogue Ġof Ġother Ġcharacters Ġspeaking Ġto Ġthe Ġtarget Ġcharacter ). ĠFurther , Ġperforming Ġsemantic Ġtext Ġexchange Ġon Ġthe Ġchosen Ġresponse Ġwith Ġa Ġmodel Ġsuch Ġas ĠSM ERT I ĠB IB REF 39 Ġmay Ġimprove Ġthe Ġability Ġof ĠAL O HA Ġto Ġcon verse Ġwith Ġhumans . ĠThis Ġis Ġbecause Ġthe Ġresponse Ġmay Ġbe Ġcontext Ġand ĠH LA Ġcorrect , Ġbut Ġincorrect Ġsem antically Ġ( e . g . Ġthe Ġresponse Ġmay Ġsay Ġthe Ġweather Ġis Ġsunny Ġwhen Ġit Ġis Ġactually Ġrainy ). ĠH LA - aligned Ġgener ative Ġmodels Ġis Ġanother Ġarea Ġof Ġexploration . ĠTypically , Ġgener ative Ġmodels Ġproduce Ġtext Ġthat Ġis Ġless Ġfluent , Ġbut Ġfurther Ġwork Ġin Ġthis Ġarea Ġmay Ġlead Ġto Ġbetter Ġresults . ĠLastly , Ġa Ġmore Ġdiverse Ġand Ġlarger Ġparticipant Ġpool Ġis Ġrequired Ġdue Ġto Ġthe Ġlimited Ġsize Ġof Ġour Ġhuman Ġevaluation .