No
BOOLEAN
Unanswerable
NONE
That said, these results, though they do show a marginal increase in dev accuracy and a decrease in CE loss, suggest that perhaps listing description is not too predictive of occupancy rate given our parameterizations. While the listing description is surely an influential metric in determining the quality of a listing, other factors such as location, amenities, and home type might play a larger role in the consumer's decision. We were hopeful that these factors would be represented in the price per bedroom of the listing – our control variable – but the relationship may not have been strong enough. However, should a strong relationship actually exist and there be instead a problem with our method, there are a few possibilities of what went wrong. We assumed that listings with similar occupancy rates would have similar listing descriptions regardless of price, which is not necessarily a strong assumption. This is coupled with an unexpected sparseness of clean data. With over 40,000 listings, we did not expect to see such poor attention to orthography in what are essentially public advertisements of the properties. In this way, our decision to use a window size of 5, a minimum occurrence count of 2, and a dimensionality of 50 when training our GloVe vectors was ad hoc. FLOAT SELECTED: Table 2: GAN Model, Keywords = [parking], Varying Gamma Parameter
1806.11432
Introduction </s> The Ġdevelopment Ġof Ġonline Ġpeer - to - peer Ġmarkets Ġin Ġthe Ġ1990 s , Ġgalvan ized Ġby Ġthe Ġlaunch Ġof Ġsites Ġlike ĠeBay , Ġfundamentally Ġshifted Ġthe Ġway Ġbuyers Ġand Ġsellers Ġcould Ġconnect Ġ[ 4 ]. ĠThese Ġnew Ġmarkets Ġnot Ġonly Ġlever aged Ġtechnology Ġto Ġallow Ġfor Ġfaster Ġtransaction Ġspeeds , Ġbut Ġin Ġthe Ġprocess Ġalso Ġexposed Ġa Ġvariety Ġof Ġunprecedented Ġmarket - design s Ġ[ 4 ]. </s> Today , Ġmany Ġof Ġthe Ġmost Ġwell - known Ġpeer - to - peer Ġmarkets Ġlike ĠUber Ġand ĠInst ac art Ġuse Ġa Ġcentralized Ġsystem Ġthat Ġmatches Ġworkers Ġwith Ġassigned Ġtasks Ġvia Ġa Ġseries Ġof Ġcomplex Ġalgorithms Ġ[ 4 ]. ĠStill , Ġa Ġnumber Ġof Ġother Ġwebsites Ġlike ĠAirbnb Ġand ĠeBay Ġrely Ġon Ġsellers Ġand Ġbuyers Ġto Ġorgan ically Ġfind Ġone - another Ġin Ġa Ġdecentralized Ġfashion . ĠIn Ġthe Ġcase Ġof Ġthese Ġdecentralized Ġsystems , Ġsellers Ġare Ġasked Ġto Ġprice Ġand Ġmarket Ġtheir Ġproducts Ġin Ġorder Ġto Ġattract Ġpotential Ġbuyers . ĠWithout Ġa Ġlarge Ġmarketing Ġteam Ġat Ġtheir Ġdisposal , Ġhowever , Ġsellers Ġmost Ġoften Ġrely Ġon Ġtheir Ġintu itions Ġfor Ġhow Ġto Ġpresent Ġtheir Ġarticles Ġor Ġlistings Ġin Ġthe Ġmost Ġappealing Ġmanner . ĠNaturally , Ġthis Ġleads Ġto Ġmarket Ġin effic iencies , Ġwhere Ġwilling Ġsellers Ġand Ġbuyers Ġoften Ġfail Ġto Ġconnect Ġdue Ġto Ġan Ġinadequate Ġpresentation Ġof Ġthe Ġproduct Ġor Ġservice Ġoffered . </s> Background </s> Fortunately , Ġwe Ġbelieve Ġthat Ġthe Ġintroduction Ġof Ġun super vised Ġgener ative Ġlanguage Ġmodels Ġpresents Ġa Ġway Ġin Ġwhich Ġto Ġtackle Ġthis Ġparticular Ġshort coming Ġof Ġpeer - to - peer Ġmarkets . ĠIn Ġ2014 , ĠIan ĠGood f ellow Ġet . Ġal Ġproposed Ġthe Ġgeneral Ġadvers arial Ġnetwork Ġ( GAN ) Ġ[ 5 ]. ĠThe Ġgroup Ġshowcased Ġhow Ġthis Ġgener ative Ġmodel Ġcould Ġlearn Ġto Ġartificially Ġreplicate Ġdata Ġpatterns Ġto Ġan Ġunprecedented Ġrealistic Ġdegree Ġ[ 5 ]. ĠSince Ġthen , Ġthese Ġmodels Ġhave Ġshown Ġtremendous Ġpotential Ġin Ġtheir Ġability Ġto Ġgenerate Ġphoto - real istic Ġimages Ġand Ġcoherent Ġtext Ġsamples Ġ[ 5 ]. </s> The Ġframework Ġthat ĠG AN s Ġuse Ġfor Ġgenerating Ġnew Ġdata Ġpoints Ġemploys Ġan Ġend - to - end Ġneural Ġnetwork Ġcomprised Ġof Ġtwo Ġmodels : Ġa Ġgenerator Ġand Ġa Ġdiscrim inator Ġ[ 5 ]. ĠThe Ġgenerator Ġis Ġtasked Ġwith Ġrepl icating Ġthe Ġdata Ġthat Ġis Ġfed Ġinto Ġthe Ġmodel , Ġwithout Ġever Ġbeing Ġdirectly Ġexposed Ġto Ġthe Ġreal Ġsamples . ĠInstead , Ġthis Ġmodel Ġlearns Ġto Ġreproduce Ġthe Ġgeneral Ġpatterns Ġof Ġthe Ġinput Ġvia Ġits Ġinteraction Ġwith Ġthe Ġdiscrim inator . </s> The Ġrole Ġof Ġthe Ġdiscrim inator , Ġin Ġturn , Ġis Ġto Ġtell Ġapart Ġwhich Ġdata Ġpoints Ġare ĠâĢ ĺ real âĢ Ļ Ġand Ġwhich Ġhave Ġbeen Ġcreated Ġby Ġthe Ġgenerator . ĠOn Ġeach Ġrun Ġthrough Ġthe Ġmodel , Ġthe Ġgenerator Ġthen Ġadapt s Ġits Ġconstructed Ġoutput Ġso Ġas Ġto Ġmore Ġeffectively ĠâĢ ĺ tr ick âĢ Ļ Ġthe Ġdiscrim inator Ġinto Ġnot Ġbeing Ġable Ġto Ġdistinguish Ġthe Ġreal Ġfrom Ġthe Ġgenerated Ġdata . ĠThe Ġend - to - end Ġnature Ġof Ġthe Ġmodel Ġthen Ġforces Ġboth Ġthe Ġgenerator Ġand Ġdiscrim inator Ġto Ġlearn Ġin Ġparallel Ġ[ 7 ]. ĠWhile ĠG AN Ġmodels Ġhave Ġshown Ġgreat Ġpotential Ġin Ġtheir Ġability Ġto Ġgenerate Ġrealistic Ġdata Ġsamples , Ġthey Ġare Ġnotoriously Ġdifficult Ġto Ġtrain . ĠThis Ġdifficulty Ġarises Ġfrom Ġtwo - parts : Ġ1 ) ĠFirst , Ġit Ġis Ġdifficult Ġto Ġtune Ġthe Ġhyper - param eters Ġcorrectly Ġfor Ġthe Ġadvers arial Ġmodel Ġto Ġcontinue Ġlearning Ġthroughout Ġall Ġof Ġthe Ġtraining Ġepoch s Ġ[ 5 ]. ĠSince Ġboth Ġthe Ġdiscrim inator Ġand Ġgenerator Ġare Ġupdated Ġvia Ġthe Ġsame Ġgradient , Ġit Ġis Ġvery Ġcommon Ġfor Ġthe Ġmodel Ġto Ġfall Ġinto Ġa Ġlocal Ġmin ima Ġbefore Ġcompleting Ġall Ġof Ġthe Ġdefined Ġtraining Ġcycles . Ġ2 ) ĠG AN s Ġare Ġcomput ationally Ġexpensive Ġto Ġtrain , Ġgiven Ġthat Ġboth Ġmodels Ġare Ġupdated Ġon Ġeach Ġcycle Ġin Ġparallel Ġ[ 5 ]. ĠThis Ġcompounds Ġthe Ġdifficulty Ġof Ġtuning Ġthe Ġmodel âĢ Ļ s Ġparameters . </s> Nonetheless , ĠG AN s Ġhave Ġcontinued Ġto Ġshow Ġtheir Ġvalue Ġparticularly Ġin Ġthe Ġdomain Ġof Ġtext - generation . ĠOf Ġparticular Ġinterest Ġfor Ġour Ġpurposes , ĠRad ford Ġet Ġal . Ġpropose Ġsynthes izing Ġimages Ġfrom Ġtext Ġdescriptions Ġ[ 3 ]. ĠThe Ġgroup Ġdemonstrates Ġhow ĠG AN s Ġcan Ġproduce Ġimages Ġthat Ġcorrespond Ġto Ġa Ġuser - defined Ġtext Ġdescription . ĠIt Ġthus Ġseems Ġfeasible Ġthat Ġby Ġusing Ġa Ġsimilar Ġmodel , Ġwe Ġcan Ġproduce Ġtext Ġsamples Ġthat Ġare Ġconditioned Ġupon Ġa Ġset Ġof Ġuser - specified Ġkeywords . </s> We Ġwere Ġsimilarly Ġinfluenced Ġby Ġthe Ġwork Ġof ĠRad ford Ġet . Ġal , Ġwho Ġargue Ġfor Ġthe Ġimportance Ġof Ġlayer Ġnormal ization Ġand Ġdata - specific Ġtrained Ġword Ġembed d ings Ġfor Ġtext Ġgeneration Ġ[ 9 ] Ġand Ġsentiment Ġanalysis Ġcategor ization . ĠThese Ġfindings Ġlead Ġus Ġto Ġquestion Ġwhether Ġit Ġis Ġpossible Ġto Ġemploy Ġrecurrent Ġneural Ġnetworks Ġwith Ġlong Ġshort - term Ġmemory Ġgates , Ġas Ġdefined Ġby ĠMik ol ov Ġet Ġal ., Ġto Ġcategor ize Ġproduct Ġdescriptions Ġinto Ġcategories Ġbased Ġon Ġthe Ġproduct 's Ġpopularity Ġ[ 6 ]. </s> Data </s> The Ġdata Ġfor Ġthe Ġproject Ġwas Ġacquired Ġfrom ĠA ird na , Ġa Ġdata Ġprocessing Ġservice Ġthat Ġcollabor ates Ġwith ĠAirbnb Ġto Ġproduce Ġhigh - acc uracy Ġdata Ġsumm aries Ġfor Ġlistings Ġin Ġgeographic Ġregions Ġof Ġthe ĠUnited ĠStates . ĠFor Ġthe Ġsake Ġof Ġsimplicity , Ġwe Ġfocus Ġour Ġanalysis Ġon ĠAirbnb Ġlistings Ġfrom ĠManhattan , ĠNY , Ġduring Ġthe Ġtime Ġperiod Ġof ĠJanuary Ġ1 , Ġ2016 , Ġto ĠJanuary Ġ1 , Ġ2017 . ĠThe Ġdata Ġprovided Ġto Ġus Ġcontained Ġinformation Ġfor Ġroughly Ġ40 , 000 ĠManhattan Ġlistings Ġthat Ġwere Ġposted Ġon ĠAirbnb Ġduring Ġthis Ġdefined Ġtime Ġperiod . ĠFor Ġeach Ġlisting , Ġwe Ġwere Ġgiven Ġinformation Ġof Ġthe Ġamenities Ġof Ġthe Ġlisting Ġ( number Ġof Ġbathrooms , Ġnumber Ġof Ġbedrooms ĠâĢ¦ ), Ġthe Ġlisting âĢ Ļ s Ġzip Ġcode , Ġthe Ġhost âĢ Ļ s Ġdescription Ġof Ġthe Ġlisting , Ġthe Ġprice Ġof Ġthe Ġlisting , Ġand Ġthe Ġoccupancy Ġrate Ġof Ġthe Ġlisting . ĠAirbnb Ġdefines Ġa Ġhome 's Ġoccupancy Ġrate , Ġas Ġthe Ġpercentage Ġof Ġtime Ġthat Ġa Ġlisting Ġis Ġoccupied Ġover Ġthe Ġtime Ġperiod Ġthat Ġthe Ġlisting Ġis Ġavailable . ĠThis Ġgives Ġus Ġa Ġreasonable Ġmetric Ġfor Ġdefining Ġpopular Ġversus Ġless Ġpopular Ġlistings . </s> App roach </s> Prior Ġto Ġbuilding Ġour Ġgener ative Ġmodel , Ġwe Ġsought Ġto Ġgain Ġa Ġbetter Ġunderstanding Ġof Ġhow Ġless Ġand Ġmore Ġpopular Ġlisting Ġdescriptions Ġdiffered Ġin Ġtheir Ġwriting Ġstyle . ĠWe Ġdefined Ġa Ġhome âĢ Ļ s Ġpopularity Ġvia Ġits Ġoccupancy Ġrate Ġmetric , Ġwhich Ġwe Ġdescribe Ġin Ġthe ĠData Ġsection . ĠUsing Ġthis Ġpopularity Ġhe uristic , Ġwe Ġfirst Ġstrat ified Ġour Ġdataset Ġinto Ġgroup ings Ġof Ġlistings Ġat Ġsimilar Ġprice Ġpoints Ġ( i . e . Ġ$ 0 -$ 30 , Ġ$ 30 -$ 60 , Ġ... ). ĠImport antly , Ġrather Ġthan Ġusing Ġthe Ġhome âĢ Ļ s Ġquoted Ġprice , Ġwe Ġrelied Ġon Ġthe Ġprice Ġper Ġbedroom Ġas Ġa Ġbetter Ġmetric Ġfor Ġthe Ġcost Ġof Ġthe Ġlisting . ĠHaving Ġclustered Ġour Ġlistings Ġinto Ġthese Ġgroup ings , Ġwe Ġthen Ġselected Ġthe Ġtop Ġthird Ġof Ġlistings Ġby Ġoccupancy Ġrate , Ġas Ġpart Ġof Ġthe ĠâĢ ĺ high Ġpopularity âĢ Ļ Ġgroup . ĠList ings Ġin Ġthe Ġmiddle Ġand Ġlowest Ġthirds Ġby Ġoccupancy Ġrate Ġwere Ġlabeled ĠâĢ ĺ medium Ġpopularity âĢ Ļ Ġand ĠâĢ ĺ low Ġpopularity âĢ Ļ Ġrespectively . ĠWe Ġthen Ġcombined Ġall Ġof Ġthe Ġlistings Ġwith Ġhigh / medium / low Ġpopularity Ġtogether Ġfor Ġour Ġfinal Ġdata Ġset . </s> Rec urrent ĠNeural ĠNetwork Ġwith ĠLong ĠShort - Term ĠMemory ĠGates </s> Using Ġour Ġcleaned Ġdata Ġset , Ġwe Ġnow Ġbuilt Ġa Ġrecurrent Ġneural Ġnetwork Ġ( R NN ) Ġwith Ġlong Ġshort - term Ġmemory Ġgates Ġ( L ST M ). ĠOur ĠR NN / L ST M Ġis Ġtrained Ġto Ġpredict , Ġgiven Ġa Ġdescription , Ġwhether Ġa Ġhome Ġcorresponds Ġto Ġa Ġhigh / medium / low Ġpopularity Ġlisting . ĠThe Ġarchitecture Ġof Ġthe ĠR NN / L ST M Ġemploys ĠT ensor flow âĢ Ļ s ĠDynamic ĠR NN Ġpackage . ĠEach Ġsentence Ġinput Ġis Ġfirst Ġfed Ġinto Ġan Ġembed ding Ġlayer , Ġwhere Ġthe Ġinput âĢ Ļ s Ġtext Ġis Ġconverted Ġto Ġa ĠGl o Ve Ġvector . ĠThese ĠGl o Ve Ġvectors Ġare Ġlearned Ġvia Ġa Ġglobal Ġword - word Ġco - occ urrence Ġmatrix Ġusing Ġour Ġcorpus Ġof ĠAirbnb Ġlisting Ġdescriptions Ġ[ 8 ]. ĠAt Ġeach Ġtime Ġstep , Ġthe ĠGl o Ve Ġvectors Ġare Ġthen Ġfed Ġinto Ġan ĠL ST M Ġlayer . ĠFor Ġeach Ġlayer , Ġthe Ġmodel Ġforward Ġpropag ates Ġthe Ġoutput Ġof Ġthe ĠL ST M Ġlayer Ġto Ġthe Ġnext Ġtime - step âĢ Ļ s ĠL ST M Ġlayer Ġvia Ġa Ġrect ified Ġlinear Ġunit Ġ( RL U ) Ġactivation Ġfunction . ĠEach Ġlayer Ġalso Ġpipes Ġthe Ġoutput Ġof Ġthe ĠL ST M Ġthrough Ġa Ġcross - ent ropy Ġoperation , Ġto Ġpredict , Ġfor Ġeach Ġtime - step , Ġthe Ġcategory Ġof Ġthe Ġinput Ġsequence . ĠWe Ġfinally Ġensemble Ġthese Ġpredictions , Ġto Ġcreate Ġthe Ġmodel âĢ Ļ s Ġcomplete Ġoutput Ġprediction . </s> Gener ative ĠAd vers arial ĠNetwork </s> Having Ġobserved Ġan Ġun ide al Ġperformance Ġon Ġthis Ġtask Ġ( see ĠExper iments Ġbelow ), Ġwe Ġturned Ġour Ġattention Ġto Ġbuilding Ġa Ġmodel Ġthat Ġcan Ġreplicate Ġthe Ġwriting Ġstyle Ġof Ġhigh Ġpopularity Ġlisting Ġdescriptions . ĠTo Ġsolve Ġthis Ġtask , Ġwe Ġdesigned Ġa Ġframework Ġfor Ġa Ġgeneral Ġadvers arial Ġnetwork . ĠThis Ġmodel Ġemploys Ġthe Ġstandard Ġset Ġup Ġof Ġa Ġgenerator Ġand Ġa Ġdiscrim inator , Ġbut Ġextends Ġthe Ġframework Ġwith Ġthe Ġadoption Ġof Ġthe ĠDie hl - Mart inez - K am al u Ġloss . </s> The Ġgenerator Ġis Ġdesigned Ġas Ġa Ġfeed - forward Ġneural Ġnetwork Ġwith Ġthree Ġlayers Ġof Ġdepth . ĠThe Ġinput Ġto Ġthe Ġgenerator Ġis Ġsimply Ġa Ġvector Ġof Ġrandom Ġnoise . ĠThis Ġinput Ġis Ġthen Ġfed Ġdirectly Ġto Ġthe Ġfirst Ġhidden Ġlayer Ġvia Ġa Ġlinear Ġtransformation . ĠBetween Ġthe Ġfirst Ġand Ġsecond Ġlayer Ġwe Ġapply Ġan Ġexponential Ġlinear Ġunit Ġ( EL U ) Ġas Ġa Ġnon - linear Ġactivation Ġfunction . ĠOur Ġreasoning Ġfor Ġdoing Ġso Ġis Ġbased Ġon Ġfindings Ġby ĠDash Ġet Ġal . Ġthat Ġthe Ġexperimental Ġaccuracy Ġof ĠEL Us Ġover Ġrect ified Ġlinear Ġunits Ġ( RL U ) Ġtends Ġto Ġbe Ġsomewhat Ġhigher Ġfor Ġgener ative Ġtasks Ġ[ 3 ]. ĠThen , Ġto Ġscale Ġthe Ġgenerator âĢ Ļ s Ġoutput Ġto Ġbe Ġin Ġthe Ġrange Ġ0 - 1 , Ġwe Ġapply Ġa Ġs igm oid Ġnon - linear ity Ġbetween Ġthe Ġsecond Ġand Ġthe Ġthird Ġlayer Ġof Ġthe Ġmodel . </s> The Ġdiscrim inator Ġsimilarly Ġused Ġa Ġfeed - forward Ġstructure Ġwith Ġthree Ġlayers Ġof Ġdepth . ĠThe Ġinput Ġto Ġthe Ġdiscrim inator Ġcomes Ġfrom Ġtwo Ġsources : Ġreal Ġdata Ġfed Ġdirectly Ġinto Ġthe Ġdiscrim inator Ġand Ġthe Ġdata Ġgenerated Ġby Ġthe Ġgenerator . ĠThis Ġinput Ġis Ġthen Ġpip ed Ġinto Ġthe Ġfirst Ġhidden Ġlayer . ĠAs Ġbefore , Ġan ĠEL U Ġtransformation Ġis Ġthen Ġapplied Ġbetween Ġthe Ġfirst Ġand Ġsecond Ġlayer , Ġas Ġwell Ġas Ġbetween Ġthe Ġsecond Ġand Ġthird Ġhidden Ġlayers . ĠFinally , Ġa Ġs igm oid Ġactivation Ġis Ġused Ġon Ġthe Ġoutput Ġof Ġthe Ġlast Ġhidden Ġlayer . ĠThis Ġs igm oid Ġactivation Ġis Ġimportant Ġsince Ġthe Ġoutput Ġof Ġour Ġdiscrim inator Ġis Ġa Ġbinary Ġboolean Ġthat Ġindicates Ġwhether Ġthe Ġdiscrim inator Ġbelieves Ġthe Ġinput Ġto Ġhave Ġbeen Ġreal Ġdata Ġor Ġdata Ġproduced Ġby Ġthe Ġgenerator . ĠThis Ġdiscrim inator Ġis Ġthus Ġtrained Ġto Ġminimize Ġthe Ġbinary Ġcross - ent ropy Ġloss Ġof Ġits Ġprediction Ġ( whether Ġthe Ġdata Ġwas Ġreal Ġor Ġfake ) Ġand Ġthe Ġreal Ġground - truth Ġof Ġeach Ġdata Ġpoint . </s> The Ġgeneral Ġframework Ġdefined Ġabove Ġwas Ġinspired Ġlargely Ġby Ġthe Ġopen - source Ġcode Ġof ĠNag ĠDev , Ġand Ġwas Ġbuilt Ġusing ĠPy tor ch Ġ[ 7 ]. ĠOne Ġkey Ġextension Ġto Ġthe Ġbasic ĠG AN Ġmodel , Ġhowever , Ġis Ġthe Ġloss Ġfunction Ġthat Ġwe Ġapply Ġto Ġthe Ġgenerator , Ġnamely Ġthe ĠDie hl - Mart inez - K am al u Ġ( DM K ) ĠLoss Ġwhich Ġwe Ġdefine Ġbelow . </s> The ĠDie hl - Mart inez - K am al u ĠLoss Ġis Ġa Ġweighted Ġcombination Ġof Ġa Ġbinary Ġcross Ġentropy Ġloss Ġwith Ġa Ġdot - product Ġattention Ġmetric Ġof Ġeach Ġuser - defined Ġkeyword Ġwith Ġthe Ġmodel âĢ Ļ s Ġgenerated Ġoutput . ĠForm ally , Ġthe Ġbinary Ġcross Ġentropy Ġ( BC E ) Ġloss Ġfor Ġone Ġexample Ġis Ġdefined Ġas : Ġ$ ĠBCE ( x , y ) Ġ= Ġy Ġ\ cd ot Ġlog x Ġ+ Ġ( 1 - y ) Ġ\ cd ot Ġlog ( 1 - x ), Ġ$ </s> where Ġx Ġis Ġdefined Ġas Ġthe Ġpredicted Ġlabel Ġfor Ġeach Ġsample Ġand Ġy Ġis Ġthe Ġtrue Ġlabel Ġ( i . e . Ġreal Ġversus Ġfake Ġdata ). ĠThe ĠDM K Ġloss Ġthen Ġcalculates Ġan Ġadditional Ġterm , Ġwhich Ġcorresponds Ġto Ġthe Ġdot - product Ġattention Ġof Ġeach Ġword Ġin Ġthe Ġgenerated Ġoutput Ġwith Ġeach Ġkeyword Ġspecified Ġby Ġthe Ġuser . ĠTo Ġillustrate Ġby Ġexample , Ġsay Ġa Ġuser Ġdesires Ġthe Ġgenerated Ġoutput Ġto Ġcontain Ġthe Ġkeywords , Ġ$\ l brace Ġsubway , Ġman hattan \ r brace Ġ$ Ġ. ĠThe Ġmodel Ġthen Ġconverts Ġeach Ġof Ġthese Ġkeywords Ġto Ġtheir Ġcorresponding Ġglove Ġvectors . ĠLet Ġus Ġdefine Ġthe Ġfollowing Ġnotation Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ Ġis Ġthe ĠGl o Ve Ġrepresentation Ġof Ġthe Ġword Ġapple , Ġand Ġlet Ġus Ġsuppose Ġthat Ġ$ g $ Ġis Ġthe Ġvector Ġof Ġword Ġembed d ings Ġgenerated Ġby Ġthe Ġgenerator . ĠThat Ġis , Ġ$ g _ 1 $ Ġis Ġthe Ġfirst Ġword Ġembed ding Ġof Ġthe Ġgenerator âĢ Ļ s Ġoutput . ĠLet Ġus Ġalso Ġsuppose Ġ$ k $ Ġis Ġa Ġvector Ġof Ġthe Ġkeywords Ġspecified Ġby Ġthe Ġuser . ĠIn Ġour Ġexamples , Ġ$ k $ Ġis Ġalways Ġin Ġ$ R ^{ 1 } $ Ġwith Ġ$ k _ 1 $ Ġone Ġequ aling Ġof Ġ$ âĢ ĺ sub way âĢ Ļ $ Ġor Ġ$ âĢ ĺ park ing âĢ Ļ $ Ġ. ĠThe Ġdot - product Ġterm Ġof Ġthe ĠDM K Ġloss Ġthen Ġcalculates Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ 0 Ġ. ĠWe igh ing Ġthis Ġterm Ġby Ġsome Ġhyper - param eter , Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ 1 Ġ, Ġthen Ġgives Ġus Ġthe Ġentire Ġdefinition Ġof Ġthe ĠDM K Ġloss : Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ 2 Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ 3 </s> Exper iments </s> In Ġseeking Ġto Ġanswer Ġthe Ġquestion Ġof Ġwhether Ġthe Ġoccupancy Ġrate Ġof Ġa Ġlisting Ġcould Ġbe Ġextracted Ġfrom Ġthe Ġlisting âĢ Ļ s Ġsummary , Ġwe Ġran Ġa Ġnumber Ġof Ġexperiments Ġon Ġour Ġfirst Ġmodel . ĠTwo Ġparameter izations Ġwhich Ġwe Ġpresent Ġhere Ġare Ġ( 1 ) Ġwhether Ġthe Ġword Ġvectors Ġused Ġin Ġthe Ġembed ding Ġlayer Ġare Ġtrained Ġon Ġour Ġcorpus Ġor Ġcome Ġpret rained Ġfrom ĠWikipedia Ġand ĠGig aw ord Ġand Ġ( 2 ) Ġwhether Ġen semb ling Ġor Ġthe Ġfinal Ġhidden Ġstate Ġin Ġisolation Ġare Ġused Ġto Ġmake Ġa Ġprediction Ġfor Ġthe Ġsequence . ĠCommon Ġto Ġall Ġexperiments Ġwas Ġour Ġdecision Ġto Ġuse Ġan ĠAdam Ġoptim izer , Ġ16 ĠL ST M Ġunits , Ġ50 - dimensional ĠGl o Ve Ġvectors , Ġand Ġa Ġ70 - 30 Ġsplit Ġin Ġtrain Ġand Ġtest Ġdata . </s> Over Ġten Ġepoch s , Ġthe Ġmodel Ġparameter ization Ġwhich Ġperforms Ġthe Ġbest Ġuses ĠGl o Ve Ġvectors Ġtrained Ġon Ġa Ġcorpus Ġconsisting Ġof Ġall Ġlisting Ġdescriptions Ġand Ġen semb ling Ġto Ġmake Ġits Ġclass Ġprediction . ĠAs Ġa Ġresult , Ġour Ġfindings Ġare Ġwell Ġin - line Ġwith Ġthose Ġpresented Ġby ĠRad ford Ġet . Ġal Ġwho Ġunderscore Ġthe Ġimportance Ġof Ġtraining Ġword Ġembed d ings Ġon Ġa Ġdata - specific Ġcorpus Ġfor Ġbest Ġresults Ġon Ġgener ative Ġtasks Ġ[ 9 ]. </s> That Ġsaid , Ġthese Ġresults , Ġthough Ġthey Ġdo Ġshow Ġa Ġmarginal Ġincrease Ġin Ġdev Ġaccuracy Ġand Ġa Ġdecrease Ġin ĠCE Ġloss , Ġsuggest Ġthat Ġperhaps Ġlisting Ġdescription Ġis Ġnot Ġtoo Ġpredictive Ġof Ġoccupancy Ġrate Ġgiven Ġour Ġparameter izations . ĠWhile Ġthe Ġlisting Ġdescription Ġis Ġsurely Ġan Ġinfluential Ġmetric Ġin Ġdetermining Ġthe Ġquality Ġof Ġa Ġlisting , Ġother Ġfactors Ġsuch Ġas Ġlocation , Ġamenities , Ġand Ġhome Ġtype Ġmight Ġplay Ġa Ġlarger Ġrole Ġin Ġthe Ġconsumer 's Ġdecision . ĠWe Ġwere Ġhopeful Ġthat Ġthese Ġfactors Ġwould Ġbe Ġrepresented Ġin Ġthe Ġprice Ġper Ġbedroom Ġof Ġthe Ġlisting ĠâĢĵ Ġour Ġcontrol Ġvariable ĠâĢĵ Ġbut Ġthe Ġrelationship Ġmay Ġnot Ġhave Ġbeen Ġstrong Ġenough . </s> However , Ġshould Ġa Ġstrong Ġrelationship Ġactually Ġexist Ġand Ġthere Ġbe Ġinstead Ġa Ġproblem Ġwith Ġour Ġmethod , Ġthere Ġare Ġa Ġfew Ġpossibilities Ġof Ġwhat Ġwent Ġwrong . ĠWe Ġassumed Ġthat Ġlistings Ġwith Ġsimilar Ġoccupancy Ġrates Ġwould Ġhave Ġsimilar Ġlisting Ġdescriptions Ġregardless Ġof Ġprice , Ġwhich Ġis Ġnot Ġnecessarily Ġa Ġstrong Ġassumption . ĠThis Ġis Ġcoupled Ġwith Ġan Ġunexpected Ġsp ars eness Ġof Ġclean Ġdata . ĠWith Ġover Ġ40 , 000 Ġlistings , Ġwe Ġdid Ġnot Ġexpect Ġto Ġsee Ġsuch Ġpoor Ġattention Ġto Ġorth ography Ġin Ġwhat Ġare Ġessentially Ġpublic Ġadvertisements Ġof Ġthe Ġproperties . ĠIn Ġthis Ġway , Ġour Ġdecision Ġto Ġuse Ġa Ġwindow Ġsize Ġof Ġ5 , Ġa Ġminimum Ġoccurrence Ġcount Ġof Ġ2 , Ġand Ġa Ġdimension ality Ġof Ġ50 Ġwhen Ġtraining Ġour ĠGl o Ve Ġvectors Ġwas Ġad Ġhoc . </s> Se eking Ġto Ġcreate Ġa Ġmodel Ġwhich Ġcould Ġgenerate Ġand Ġdiscriminate Ġa ĠâĢ ľ high - occup ancy Ġlisting Ġdescription âĢ Ŀ , Ġwe Ġwanted Ġto Ġevaluate Ġthe Ġcapabilities Ġof Ġa Ġgener ative Ġadvers arial Ġnetwork Ġtrained Ġon Ġeither Ġthe Ġstandard Ġbinary Ġcross - ent ropy Ġloss Ġor Ġthe ĠDM K Ġloss Ġproposed Ġabove . ĠCommon Ġto Ġboth Ġmodels Ġwas Ġthe Ġdecision Ġto Ġalternate Ġbetween Ġtraining Ġthe Ġgenerator Ġfor Ġ50 Ġsteps Ġand Ġthe Ġdiscrim inator Ġfor Ġ2000 Ġsteps . ĠWe Ġleave Ġfurther Ġtuning Ġof Ġthe Ġmodels Ġto Ġfuture Ġresearch Ġas Ġeach Ġoccasionally Ġfalls Ġinto Ġun ide al Ġlocal Ġoptim a Ġwithin Ġ20 Ġiterations . ĠOne Ġpotential Ġculprit Ġis Ġthe Ġstep Ġimbalance Ġbetween Ġgenerator Ġand Ġdiscrim inator ĠâĢĵ Ġshould Ġeither Ġlearn Ġat Ġa Ġmuch Ġfaster Ġrate Ġthan Ġthe Ġother , Ġone Ġcomponent Ġis Ġliable Ġto Ġbe ĠâĢ ľ def e ated " Ġand Ġcease Ġto Ġlearn Ġthe Ġtraining Ġdata . </s> Qual itatively Ġthe Ġnetwork Ġtrained Ġon Ġthe ĠDM K Ġloss Ġshows Ġgreat Ġpromise . ĠWith Ġrespect Ġto Ġthe Ġtwo Ġexperiments Ġpresented Ġhere , Ġwe Ġhave Ġshown Ġthat Ġit Ġis Ġpossible Ġto Ġintroduce Ġa Ġmeasure Ġof Ġsuggestion Ġin Ġthe Ġtext Ġproduced Ġby Ġthe Ġgenerator . ĠWhile Ġthis Ġmodel Ġis Ġalso Ġsubject Ġto Ġa Ġrapid Ġdead lock Ġbetween Ġgenerator Ġand Ġdiscrim inator , Ġit Ġis Ġinteresting Ġto Ġsee Ġhow Ġthe Ġintroduction Ġof Ġkeywords Ġis Ġgradual Ġand Ġaffects Ġthe Ġprox imal Ġtokens Ġincluded Ġin Ġthe Ġoutput . ĠThis Ġbehavior Ġwas Ġmade Ġpossible Ġby Ġpaying Ġclose Ġattention Ġto Ġthe Ġhyper param eter Ġ$\ gam ma Ġ$ Ġ, Ġthe Ġweight Ġgiven Ġto Ġthe Ġdot Ġproduct Ġattention Ġterm Ġof Ġthe ĠDM K Ġloss . ĠAfter Ġmanual Ġtuning , Ġwe Ġsettle Ġon Ġ$\ gam ma Ġ= 0 . 000 45 $ Ġfor Ġthis Ġweight . ĠBelow , Ġwe Ġillustrate Ġmodel Ġoutputs Ġusing Ġdifferent Ġvalues Ġof ĠGamma . ĠAs Ġis Ġapparent , Ġfor Ġa Ġhyper - param eter Ġvalue Ġless Ġthan Ġroughly Ġ$\ gam ma Ġ= Ġ0 . 000 4 $ Ġ, Ġthe Ġmodel Ġtends Ġto Ġignore Ġthe Ġimportance Ġof Ġthe Ġkeyword Ġweights . ĠConversely , Ġwith Ġa Ġ$\ gam ma Ġ$ Ġvalue Ġhigher Ġthan Ġ$ 0 . 000 5 $ Ġ, Ġthe Ġmodel Ġtends Ġtowards Ġoverweight ing Ġthe Ġrepresentation Ġof Ġthe Ġkeywords Ġin Ġthe Ġmodel Ġoutput . </s> Conclusion </s> We Ġhope Ġthat Ġthis Ġresearch Ġpaper Ġestablishes Ġa Ġfirst Ġattempt Ġat Ġusing Ġgener ative Ġmachine Ġlearning Ġmodels Ġfor Ġthe Ġpurposes Ġof Ġmarketing Ġon Ġpeer - to - peer Ġplatforms . ĠAs Ġthe Ġuse Ġof Ġthese Ġmarkets Ġbecomes Ġmore Ġwidespread , Ġthe Ġuse Ġof Ġself - brand ing Ġand Ġmarketing Ġtools Ġwill Ġgrow Ġin Ġimportance . ĠThe Ġdevelopment Ġof Ġtools Ġlike Ġthe ĠDM K Ġloss Ġin Ġcombination Ġwith ĠG AN s Ġdemonstrates Ġthe Ġenormous Ġpotential Ġthese Ġframeworks Ġcan Ġhave Ġin Ġsolving Ġproblems Ġthat Ġinevitably Ġarise Ġon Ġpeer - to - peer Ġplatforms . ĠCertainly , Ġhowever , Ġmore Ġworks Ġremains Ġto Ġbe Ġdone Ġin Ġthis Ġfield , Ġand Ġrecent Ġdevelopments Ġin Ġun super vised Ġgener ative Ġmodels Ġalready Ġpromise Ġpossible Ġextensions Ġto Ġour Ġanalysis . </s> For Ġexample , Ġsince Ġthe Ġintroduction Ġof ĠG AN s , Ġresearch Ġgroups Ġhave Ġstudied Ġhow Ġvari ational Ġaut oen cod ers Ġcan Ġbe Ġincorporated Ġinto Ġthese Ġmodels , Ġto Ġincrease Ġthe Ġclarity Ġand Ġaccuracy Ġof Ġthe Ġmodels âĢ Ļ Ġoutputs . ĠWang Ġet Ġal . Ġin Ġparticular Ġdemonstrate Ġhow Ġusing Ġa Ġvari ational Ġaut oen c oder Ġin Ġthe Ġgenerator Ġmodel Ġcan Ġincrease Ġthe Ġaccuracy Ġof Ġgenerated Ġtext Ġsamples Ġ[ 11 ]. ĠMost Ġpromising ly , Ġthe Ġdata Ġused Ġby Ġthe Ġgroup Ġwas Ġa Ġlarge Ġdata Ġset Ġof ĠAmazon Ġcustomer Ġreviews , Ġwhich Ġin Ġmany Ġways Ġparallels Ġthe Ġtone Ġand Ġsemantic Ġstructure Ġof ĠAirbnb Ġlisting Ġdescriptions . ĠMore Ġgenerally , ĠBowman Ġet Ġal . Ġdemonstrate Ġhow Ġthe Ġuse Ġof Ġvari ational Ġaut oen cod ers Ġpresents Ġa Ġmore Ġaccurate Ġmodel Ġfor Ġtext Ġgeneration , Ġas Ġcompared Ġto Ġstandard Ġrecurrent Ġneural Ġnetwork Ġlanguage Ġmodels Ġ[ 1 ]. </s> For Ġfuture Ġwork , Ġwe Ġmay Ġalso Ġconsider Ġexperimenting Ġwith Ġdifferent Ġforms Ġof Ġword Ġembed d ings , Ġsuch Ġas Ġimproved Ġword Ġvectors Ġ( I W V ) Ġwhich Ġwere Ġsuggested Ġby ĠRez ae inia Ġet Ġal Ġ[ 10 ]. ĠThese ĠIM V Ġword Ġembed d ings Ġare Ġtrained Ġin Ġa Ġsimilar Ġfashion Ġto ĠGl o Ve Ġvectors , Ġbut Ġalso Ġencode Ġadditional Ġinformation Ġabout Ġeach Ġword , Ġlike Ġthe Ġword âĢ Ļ s Ġpart Ġof Ġspeech . ĠFor Ġour Ġmodel , Ġwe Ġmay Ġconsider Ġencoding Ġsimilar Ġinformation Ġin Ġorder Ġfor Ġthe Ġgenerator Ġto Ġmore Ġeasily Ġlearn Ġcommon Ġsemantic Ġpatterns Ġinherent Ġto Ġthe Ġmarketing Ġdata Ġbeing Ġanalyzed .
Introduction The development of online peer-to-peer markets in the 1990s, galvanized by the launch of sites like eBay, fundamentally shifted the way buyers and sellers could connect [4]. These new markets not only leveraged technology to allow for faster transaction speeds, but in the process also exposed a variety of unprecedented market-designs [4]. Today, many of the most well-known peer-to-peer markets like Uber and Instacart use a centralized system that matches workers with assigned tasks via a series of complex algorithms [4]. Still, a number of other websites like Airbnb and eBay rely on sellers and buyers to organically find one-another in a decentralized fashion. In the case of these decentralized systems, sellers are asked to price and market their products in order to attract potential buyers. Without a large marketing team at their disposal, however, sellers most often rely on their intuitions for how to present their articles or listings in the most appealing manner. Naturally, this leads to market inefficiencies, where willing sellers and buyers often fail to connect due to an inadequate presentation of the product or service offered. Background Fortunately, we believe that the introduction of unsupervised generative language models presents a way in which to tackle this particular shortcoming of peer-to-peer markets. In 2014, Ian Goodfellow et. al proposed the general adversarial network (GAN) [5]. The group showcased how this generative model could learn to artificially replicate data patterns to an unprecedented realistic degree [5]. Since then, these models have shown tremendous potential in their ability to generate photo-realistic images and coherent text samples [5]. The framework that GANs use for generating new data points employs an end-to-end neural network comprised of two models: a generator and a discriminator [5]. The generator is tasked with replicating the data that is fed into the model, without ever being directly exposed to the real samples. Instead, this model learns to reproduce the general patterns of the input via its interaction with the discriminator. The role of the discriminator, in turn, is to tell apart which data points are ‘real’ and which have been created by the generator. On each run through the model, the generator then adapts its constructed output so as to more effectively ‘trick’ the discriminator into not being able to distinguish the real from the generated data. The end-to-end nature of the model then forces both the generator and discriminator to learn in parallel [7]. While GAN models have shown great potential in their ability to generate realistic data samples, they are notoriously difficult to train. This difficulty arises from two-parts: 1) First, it is difficult to tune the hyper-parameters correctly for the adversarial model to continue learning throughout all of the training epochs [5]. Since both the discriminator and generator are updated via the same gradient, it is very common for the model to fall into a local minima before completing all of the defined training cycles. 2) GANs are computationally expensive to train, given that both models are updated on each cycle in parallel [5]. This compounds the difficulty of tuning the model’s parameters. Nonetheless, GANs have continued to show their value particularly in the domain of text-generation. Of particular interest for our purposes, Radford et al. propose synthesizing images from text descriptions [3]. The group demonstrates how GANs can produce images that correspond to a user-defined text description. It thus seems feasible that by using a similar model, we can produce text samples that are conditioned upon a set of user-specified keywords. We were similarly influenced by the work of Radford et. al, who argue for the importance of layer normalization and data-specific trained word embeddings for text generation [9] and sentiment analysis categorization. These findings lead us to question whether it is possible to employ recurrent neural networks with long short-term memory gates, as defined by Mikolov et al., to categorize product descriptions into categories based on the product's popularity [6]. Data The data for the project was acquired from Airdna, a data processing service that collaborates with Airbnb to produce high-accuracy data summaries for listings in geographic regions of the United States. For the sake of simplicity, we focus our analysis on Airbnb listings from Manhattan, NY, during the time period of January 1, 2016, to January 1, 2017. The data provided to us contained information for roughly 40,000 Manhattan listings that were posted on Airbnb during this defined time period. For each listing, we were given information of the amenities of the listing (number of bathrooms, number of bedrooms …), the listing’s zip code, the host’s description of the listing, the price of the listing, and the occupancy rate of the listing. Airbnb defines a home's occupancy rate, as the percentage of time that a listing is occupied over the time period that the listing is available. This gives us a reasonable metric for defining popular versus less popular listings. Approach Prior to building our generative model, we sought to gain a better understanding of how less and more popular listing descriptions differed in their writing style. We defined a home’s popularity via its occupancy rate metric, which we describe in the Data section. Using this popularity heuristic, we first stratified our dataset into groupings of listings at similar price points (i.e. $0-$30, $30-$60, ...). Importantly, rather than using the home’s quoted price, we relied on the price per bedroom as a better metric for the cost of the listing. Having clustered our listings into these groupings, we then selected the top third of listings by occupancy rate, as part of the ‘high popularity’ group. Listings in the middle and lowest thirds by occupancy rate were labeled ‘medium popularity’ and ‘low popularity’ respectively. We then combined all of the listings with high/medium/low popularity together for our final data set. Recurrent Neural Network with Long Short-Term Memory Gates Using our cleaned data set, we now built a recurrent neural network (RNN) with long short-term memory gates (LSTM). Our RNN/LSTM is trained to predict, given a description, whether a home corresponds to a high/medium/low popularity listing. The architecture of the RNN/LSTM employs Tensorflow’s Dynamic RNN package. Each sentence input is first fed into an embedding layer, where the input’s text is converted to a GloVe vector. These GloVe vectors are learned via a global word-word co-occurrence matrix using our corpus of Airbnb listing descriptions [8]. At each time step, the GloVe vectors are then fed into an LSTM layer. For each layer, the model forward propagates the output of the LSTM layer to the next time-step’s LSTM layer via a rectified linear unit (RLU) activation function. Each layer also pipes the output of the LSTM through a cross-entropy operation, to predict, for each time-step, the category of the input sequence. We finally ensemble these predictions, to create the model’s complete output prediction. Generative Adversarial Network Having observed an unideal performance on this task (see Experiments below), we turned our attention to building a model that can replicate the writing style of high popularity listing descriptions. To solve this task, we designed a framework for a general adversarial network. This model employs the standard set up of a generator and a discriminator, but extends the framework with the adoption of the Diehl-Martinez-Kamalu loss. The generator is designed as a feed-forward neural network with three layers of depth. The input to the generator is simply a vector of random noise. This input is then fed directly to the first hidden layer via a linear transformation. Between the first and second layer we apply an exponential linear unit (ELU) as a non-linear activation function. Our reasoning for doing so is based on findings by Dash et al. that the experimental accuracy of ELUs over rectified linear units (RLU) tends to be somewhat higher for generative tasks [3]. Then, to scale the generator’s output to be in the range 0-1, we apply a sigmoid non-linearity between the second and the third layer of the model. The discriminator similarly used a feed-forward structure with three layers of depth. The input to the discriminator comes from two sources: real data fed directly into the discriminator and the data generated by the generator. This input is then piped into the first hidden layer. As before, an ELU transformation is then applied between the first and second layer, as well as between the second and third hidden layers. Finally, a sigmoid activation is used on the output of the last hidden layer. This sigmoid activation is important since the output of our discriminator is a binary boolean that indicates whether the discriminator believes the input to have been real data or data produced by the generator. This discriminator is thus trained to minimize the binary cross-entropy loss of its prediction (whether the data was real or fake) and the real ground-truth of each data point. The general framework defined above was inspired largely by the open-source code of Nag Dev, and was built using Pytorch [7]. One key extension to the basic GAN model, however, is the loss function that we apply to the generator, namely the Diehl-Martinez-Kamalu (DMK) Loss which we define below. The Diehl-Martinez-Kamalu Loss is a weighted combination of a binary cross entropy loss with a dot-product attention metric of each user-defined keyword with the model’s generated output. Formally, the binary cross entropy (BCE) loss for one example is defined as: $ BCE(x,y) = y \cdot logx + (1-y) \cdot log(1-x), $ where x is defined as the predicted label for each sample and y is the true label (i.e. real versus fake data). The DMK loss then calculates an additional term, which corresponds to the dot-product attention of each word in the generated output with each keyword specified by the user. To illustrate by example, say a user desires the generated output to contain the keywords, $\lbrace subway, manhattan\rbrace $ . The model then converts each of these keywords to their corresponding glove vectors. Let us define the following notation $e(‘apple’)$ is the GloVe representation of the word apple, and let us suppose that $g$ is the vector of word embeddings generated by the generator. That is, $g_1$ is the first word embedding of the generator’s output. Let us also suppose $k$ is a vector of the keywords specified by the user. In our examples, $k$ is always in $R^{1}$ with $k_1$ one equaling of $‘subway’$ or $‘parking’$ . The dot-product term of the DMK loss then calculates $e(‘apple’)$0 . Weighing this term by some hyper-parameter, $e(‘apple’)$1 , then gives us the entire definition of the DMK loss: $e(‘apple’)$2 $e(‘apple’)$3 Experiments In seeking to answer the question of whether the occupancy rate of a listing could be extracted from the listing’s summary, we ran a number of experiments on our first model. Two parameterizations which we present here are (1) whether the word vectors used in the embedding layer are trained on our corpus or come pretrained from Wikipedia and Gigaword and (2) whether ensembling or the final hidden state in isolation are used to make a prediction for the sequence. Common to all experiments was our decision to use an Adam optimizer, 16 LSTM units, 50-dimensional GloVe vectors, and a 70-30 split in train and test data. Over ten epochs, the model parameterization which performs the best uses GloVe vectors trained on a corpus consisting of all listing descriptions and ensembling to make its class prediction. As a result, our findings are well in-line with those presented by Radford et. al who underscore the importance of training word embeddings on a data-specific corpus for best results on generative tasks [9]. That said, these results, though they do show a marginal increase in dev accuracy and a decrease in CE loss, suggest that perhaps listing description is not too predictive of occupancy rate given our parameterizations. While the listing description is surely an influential metric in determining the quality of a listing, other factors such as location, amenities, and home type might play a larger role in the consumer's decision. We were hopeful that these factors would be represented in the price per bedroom of the listing – our control variable – but the relationship may not have been strong enough. However, should a strong relationship actually exist and there be instead a problem with our method, there are a few possibilities of what went wrong. We assumed that listings with similar occupancy rates would have similar listing descriptions regardless of price, which is not necessarily a strong assumption. This is coupled with an unexpected sparseness of clean data. With over 40,000 listings, we did not expect to see such poor attention to orthography in what are essentially public advertisements of the properties. In this way, our decision to use a window size of 5, a minimum occurrence count of 2, and a dimensionality of 50 when training our GloVe vectors was ad hoc. Seeking to create a model which could generate and discriminate a “high-occupancy listing description”, we wanted to evaluate the capabilities of a generative adversarial network trained on either the standard binary cross-entropy loss or the DMK loss proposed above. Common to both models was the decision to alternate between training the generator for 50 steps and the discriminator for 2000 steps. We leave further tuning of the models to future research as each occasionally falls into unideal local optima within 20 iterations. One potential culprit is the step imbalance between generator and discriminator – should either learn at a much faster rate than the other, one component is liable to be “defeated" and cease to learn the training data. Qualitatively the network trained on the DMK loss shows great promise. With respect to the two experiments presented here, we have shown that it is possible to introduce a measure of suggestion in the text produced by the generator. While this model is also subject to a rapid deadlock between generator and discriminator, it is interesting to see how the introduction of keywords is gradual and affects the proximal tokens included in the output. This behavior was made possible by paying close attention to the hyperparameter $\gamma $ , the weight given to the dot product attention term of the DMK loss. After manual tuning, we settle on $\gamma =0.00045$ for this weight. Below, we illustrate model outputs using different values of Gamma. As is apparent, for a hyper-parameter value less than roughly $\gamma = 0.0004$ , the model tends to ignore the importance of the keyword weights. Conversely, with a $\gamma $ value higher than $0.0005$ , the model tends towards overweighting the representation of the keywords in the model output. Conclusion We hope that this research paper establishes a first attempt at using generative machine learning models for the purposes of marketing on peer-to-peer platforms. As the use of these markets becomes more widespread, the use of self-branding and marketing tools will grow in importance. The development of tools like the DMK loss in combination with GANs demonstrates the enormous potential these frameworks can have in solving problems that inevitably arise on peer-to-peer platforms. Certainly, however, more works remains to be done in this field, and recent developments in unsupervised generative models already promise possible extensions to our analysis. For example, since the introduction of GANs, research groups have studied how variational autoencoders can be incorporated into these models, to increase the clarity and accuracy of the models’ outputs. Wang et al. in particular demonstrate how using a variational autoencoder in the generator model can increase the accuracy of generated text samples [11]. Most promisingly, the data used by the group was a large data set of Amazon customer reviews, which in many ways parallels the tone and semantic structure of Airbnb listing descriptions. More generally, Bowman et al. demonstrate how the use of variational autoencoders presents a more accurate model for text generation, as compared to standard recurrent neural network language models [1]. For future work, we may also consider experimenting with different forms of word embeddings, such as improved word vectors (IWV) which were suggested by Rezaeinia et al [10]. These IMV word embeddings are trained in a similar fashion to GloVe vectors, but also encode additional information about each word, like the word’s part of speech. For our model, we may consider encoding similar information in order for the generator to more easily learn common semantic patterns inherent to the marketing data being analyzed.
Does the method achieve sota performance on this dataset?
701571680724c05ca70c11bc267fb1160ea1460a
Does Ġthe Ġmethod Ġachieve Ġs ota Ġperformance Ġon Ġthis Ġdataset ?
No
<s> Does Ġthe Ġmethod Ġachieve Ġs ota Ġperformance Ġon Ġthis Ġdataset ? </s> Introduction </s> The Ġdevelopment Ġof Ġonline Ġpeer - to - peer Ġmarkets Ġin Ġthe Ġ1990 s , Ġgalvan ized Ġby Ġthe Ġlaunch Ġof Ġsites Ġlike ĠeBay , Ġfundamentally Ġshifted Ġthe Ġway Ġbuyers Ġand Ġsellers Ġcould Ġconnect Ġ[ 4 ]. ĠThese Ġnew Ġmarkets Ġnot Ġonly Ġlever aged Ġtechnology Ġto Ġallow Ġfor Ġfaster Ġtransaction Ġspeeds , Ġbut Ġin Ġthe Ġprocess Ġalso Ġexposed Ġa Ġvariety Ġof Ġunprecedented Ġmarket - design s Ġ[ 4 ]. </s> Today , Ġmany Ġof Ġthe Ġmost Ġwell - known Ġpeer - to - peer Ġmarkets Ġlike ĠUber Ġand ĠInst ac art Ġuse Ġa Ġcentralized Ġsystem Ġthat Ġmatches Ġworkers Ġwith Ġassigned Ġtasks Ġvia Ġa Ġseries Ġof Ġcomplex Ġalgorithms Ġ[ 4 ]. ĠStill , Ġa Ġnumber Ġof Ġother Ġwebsites Ġlike ĠAirbnb Ġand ĠeBay Ġrely Ġon Ġsellers Ġand Ġbuyers Ġto Ġorgan ically Ġfind Ġone - another Ġin Ġa Ġdecentralized Ġfashion . ĠIn Ġthe Ġcase Ġof Ġthese Ġdecentralized Ġsystems , Ġsellers Ġare Ġasked Ġto Ġprice Ġand Ġmarket Ġtheir Ġproducts Ġin Ġorder Ġto Ġattract Ġpotential Ġbuyers . ĠWithout Ġa Ġlarge Ġmarketing Ġteam Ġat Ġtheir Ġdisposal , Ġhowever , Ġsellers Ġmost Ġoften Ġrely Ġon Ġtheir Ġintu itions Ġfor Ġhow Ġto Ġpresent Ġtheir Ġarticles Ġor Ġlistings Ġin Ġthe Ġmost Ġappealing Ġmanner . ĠNaturally , Ġthis Ġleads Ġto Ġmarket Ġin effic iencies , Ġwhere Ġwilling Ġsellers Ġand Ġbuyers Ġoften Ġfail Ġto Ġconnect Ġdue Ġto Ġan Ġinadequate Ġpresentation Ġof Ġthe Ġproduct Ġor Ġservice Ġoffered . </s> Background </s> Fortunately , Ġwe Ġbelieve Ġthat Ġthe Ġintroduction Ġof Ġun super vised Ġgener ative Ġlanguage Ġmodels Ġpresents Ġa Ġway Ġin Ġwhich Ġto Ġtackle Ġthis Ġparticular Ġshort coming Ġof Ġpeer - to - peer Ġmarkets . ĠIn Ġ2014 , ĠIan ĠGood f ellow Ġet . Ġal Ġproposed Ġthe Ġgeneral Ġadvers arial Ġnetwork Ġ( GAN ) Ġ[ 5 ]. ĠThe Ġgroup Ġshowcased Ġhow Ġthis Ġgener ative Ġmodel Ġcould Ġlearn Ġto Ġartificially Ġreplicate Ġdata Ġpatterns Ġto Ġan Ġunprecedented Ġrealistic Ġdegree Ġ[ 5 ]. ĠSince Ġthen , Ġthese Ġmodels Ġhave Ġshown Ġtremendous Ġpotential Ġin Ġtheir Ġability Ġto Ġgenerate Ġphoto - real istic Ġimages Ġand Ġcoherent Ġtext Ġsamples Ġ[ 5 ]. </s> The Ġframework Ġthat ĠG AN s Ġuse Ġfor Ġgenerating Ġnew Ġdata Ġpoints Ġemploys Ġan Ġend - to - end Ġneural Ġnetwork Ġcomprised Ġof Ġtwo Ġmodels : Ġa Ġgenerator Ġand Ġa Ġdiscrim inator Ġ[ 5 ]. ĠThe Ġgenerator Ġis Ġtasked Ġwith Ġrepl icating Ġthe Ġdata Ġthat Ġis Ġfed Ġinto Ġthe Ġmodel , Ġwithout Ġever Ġbeing Ġdirectly Ġexposed Ġto Ġthe Ġreal Ġsamples . ĠInstead , Ġthis Ġmodel Ġlearns Ġto Ġreproduce Ġthe Ġgeneral Ġpatterns Ġof Ġthe Ġinput Ġvia Ġits Ġinteraction Ġwith Ġthe Ġdiscrim inator . </s> The Ġrole Ġof Ġthe Ġdiscrim inator , Ġin Ġturn , Ġis Ġto Ġtell Ġapart Ġwhich Ġdata Ġpoints Ġare ĠâĢ ĺ real âĢ Ļ Ġand Ġwhich Ġhave Ġbeen Ġcreated Ġby Ġthe Ġgenerator . ĠOn Ġeach Ġrun Ġthrough Ġthe Ġmodel , Ġthe Ġgenerator Ġthen Ġadapt s Ġits Ġconstructed Ġoutput Ġso Ġas Ġto Ġmore Ġeffectively ĠâĢ ĺ tr ick âĢ Ļ Ġthe Ġdiscrim inator Ġinto Ġnot Ġbeing Ġable Ġto Ġdistinguish Ġthe Ġreal Ġfrom Ġthe Ġgenerated Ġdata . ĠThe Ġend - to - end Ġnature Ġof Ġthe Ġmodel Ġthen Ġforces Ġboth Ġthe Ġgenerator Ġand Ġdiscrim inator Ġto Ġlearn Ġin Ġparallel Ġ[ 7 ]. ĠWhile ĠG AN Ġmodels Ġhave Ġshown Ġgreat Ġpotential Ġin Ġtheir Ġability Ġto Ġgenerate Ġrealistic Ġdata Ġsamples , Ġthey Ġare Ġnotoriously Ġdifficult Ġto Ġtrain . ĠThis Ġdifficulty Ġarises Ġfrom Ġtwo - parts : Ġ1 ) ĠFirst , Ġit Ġis Ġdifficult Ġto Ġtune Ġthe Ġhyper - param eters Ġcorrectly Ġfor Ġthe Ġadvers arial Ġmodel Ġto Ġcontinue Ġlearning Ġthroughout Ġall Ġof Ġthe Ġtraining Ġepoch s Ġ[ 5 ]. ĠSince Ġboth Ġthe Ġdiscrim inator Ġand Ġgenerator Ġare Ġupdated Ġvia Ġthe Ġsame Ġgradient , Ġit Ġis Ġvery Ġcommon Ġfor Ġthe Ġmodel Ġto Ġfall Ġinto Ġa Ġlocal Ġmin ima Ġbefore Ġcompleting Ġall Ġof Ġthe Ġdefined Ġtraining Ġcycles . Ġ2 ) ĠG AN s Ġare Ġcomput ationally Ġexpensive Ġto Ġtrain , Ġgiven Ġthat Ġboth Ġmodels Ġare Ġupdated Ġon Ġeach Ġcycle Ġin Ġparallel Ġ[ 5 ]. ĠThis Ġcompounds Ġthe Ġdifficulty Ġof Ġtuning Ġthe Ġmodel âĢ Ļ s Ġparameters . </s> Nonetheless , ĠG AN s Ġhave Ġcontinued Ġto Ġshow Ġtheir Ġvalue Ġparticularly Ġin Ġthe Ġdomain Ġof Ġtext - generation . ĠOf Ġparticular Ġinterest Ġfor Ġour Ġpurposes , ĠRad ford Ġet Ġal . Ġpropose Ġsynthes izing Ġimages Ġfrom Ġtext Ġdescriptions Ġ[ 3 ]. ĠThe Ġgroup Ġdemonstrates Ġhow ĠG AN s Ġcan Ġproduce Ġimages Ġthat Ġcorrespond Ġto Ġa Ġuser - defined Ġtext Ġdescription . ĠIt Ġthus Ġseems Ġfeasible Ġthat Ġby Ġusing Ġa Ġsimilar Ġmodel , Ġwe Ġcan Ġproduce Ġtext Ġsamples Ġthat Ġare Ġconditioned Ġupon Ġa Ġset Ġof Ġuser - specified Ġkeywords . </s> We Ġwere Ġsimilarly Ġinfluenced Ġby Ġthe Ġwork Ġof ĠRad ford Ġet . Ġal , Ġwho Ġargue Ġfor Ġthe Ġimportance Ġof Ġlayer Ġnormal ization Ġand Ġdata - specific Ġtrained Ġword Ġembed d ings Ġfor Ġtext Ġgeneration Ġ[ 9 ] Ġand Ġsentiment Ġanalysis Ġcategor ization . ĠThese Ġfindings Ġlead Ġus Ġto Ġquestion Ġwhether Ġit Ġis Ġpossible Ġto Ġemploy Ġrecurrent Ġneural Ġnetworks Ġwith Ġlong Ġshort - term Ġmemory Ġgates , Ġas Ġdefined Ġby ĠMik ol ov Ġet Ġal ., Ġto Ġcategor ize Ġproduct Ġdescriptions Ġinto Ġcategories Ġbased Ġon Ġthe Ġproduct 's Ġpopularity Ġ[ 6 ]. </s> Data </s> The Ġdata Ġfor Ġthe Ġproject Ġwas Ġacquired Ġfrom ĠA ird na , Ġa Ġdata Ġprocessing Ġservice Ġthat Ġcollabor ates Ġwith ĠAirbnb Ġto Ġproduce Ġhigh - acc uracy Ġdata Ġsumm aries Ġfor Ġlistings Ġin Ġgeographic Ġregions Ġof Ġthe ĠUnited ĠStates . ĠFor Ġthe Ġsake Ġof Ġsimplicity , Ġwe Ġfocus Ġour Ġanalysis Ġon ĠAirbnb Ġlistings Ġfrom ĠManhattan , ĠNY , Ġduring Ġthe Ġtime Ġperiod Ġof ĠJanuary Ġ1 , Ġ2016 , Ġto ĠJanuary Ġ1 , Ġ2017 . ĠThe Ġdata Ġprovided Ġto Ġus Ġcontained Ġinformation Ġfor Ġroughly Ġ40 , 000 ĠManhattan Ġlistings Ġthat Ġwere Ġposted Ġon ĠAirbnb Ġduring Ġthis Ġdefined Ġtime Ġperiod . ĠFor Ġeach Ġlisting , Ġwe Ġwere Ġgiven Ġinformation Ġof Ġthe Ġamenities Ġof Ġthe Ġlisting Ġ( number Ġof Ġbathrooms , Ġnumber Ġof Ġbedrooms ĠâĢ¦ ), Ġthe Ġlisting âĢ Ļ s Ġzip Ġcode , Ġthe Ġhost âĢ Ļ s Ġdescription Ġof Ġthe Ġlisting , Ġthe Ġprice Ġof Ġthe Ġlisting , Ġand Ġthe Ġoccupancy Ġrate Ġof Ġthe Ġlisting . ĠAirbnb Ġdefines Ġa Ġhome 's Ġoccupancy Ġrate , Ġas Ġthe Ġpercentage Ġof Ġtime Ġthat Ġa Ġlisting Ġis Ġoccupied Ġover Ġthe Ġtime Ġperiod Ġthat Ġthe Ġlisting Ġis Ġavailable . ĠThis Ġgives Ġus Ġa Ġreasonable Ġmetric Ġfor Ġdefining Ġpopular Ġversus Ġless Ġpopular Ġlistings . </s> App roach </s> Prior Ġto Ġbuilding Ġour Ġgener ative Ġmodel , Ġwe Ġsought Ġto Ġgain Ġa Ġbetter Ġunderstanding Ġof Ġhow Ġless Ġand Ġmore Ġpopular Ġlisting Ġdescriptions Ġdiffered Ġin Ġtheir Ġwriting Ġstyle . ĠWe Ġdefined Ġa Ġhome âĢ Ļ s Ġpopularity Ġvia Ġits Ġoccupancy Ġrate Ġmetric , Ġwhich Ġwe Ġdescribe Ġin Ġthe ĠData Ġsection . ĠUsing Ġthis Ġpopularity Ġhe uristic , Ġwe Ġfirst Ġstrat ified Ġour Ġdataset Ġinto Ġgroup ings Ġof Ġlistings Ġat Ġsimilar Ġprice Ġpoints Ġ( i . e . Ġ$ 0 -$ 30 , Ġ$ 30 -$ 60 , Ġ... ). ĠImport antly , Ġrather Ġthan Ġusing Ġthe Ġhome âĢ Ļ s Ġquoted Ġprice , Ġwe Ġrelied Ġon Ġthe Ġprice Ġper Ġbedroom Ġas Ġa Ġbetter Ġmetric Ġfor Ġthe Ġcost Ġof Ġthe Ġlisting . ĠHaving Ġclustered Ġour Ġlistings Ġinto Ġthese Ġgroup ings , Ġwe Ġthen Ġselected Ġthe Ġtop Ġthird Ġof Ġlistings Ġby Ġoccupancy Ġrate , Ġas Ġpart Ġof Ġthe ĠâĢ ĺ high Ġpopularity âĢ Ļ Ġgroup . ĠList ings Ġin Ġthe Ġmiddle Ġand Ġlowest Ġthirds Ġby Ġoccupancy Ġrate Ġwere Ġlabeled ĠâĢ ĺ medium Ġpopularity âĢ Ļ Ġand ĠâĢ ĺ low Ġpopularity âĢ Ļ Ġrespectively . ĠWe Ġthen Ġcombined Ġall Ġof Ġthe Ġlistings Ġwith Ġhigh / medium / low Ġpopularity Ġtogether Ġfor Ġour Ġfinal Ġdata Ġset . </s> Rec urrent ĠNeural ĠNetwork Ġwith ĠLong ĠShort - Term ĠMemory ĠGates </s> Using Ġour Ġcleaned Ġdata Ġset , Ġwe Ġnow Ġbuilt Ġa Ġrecurrent Ġneural Ġnetwork Ġ( R NN ) Ġwith Ġlong Ġshort - term Ġmemory Ġgates Ġ( L ST M ). ĠOur ĠR NN / L ST M Ġis Ġtrained Ġto Ġpredict , Ġgiven Ġa Ġdescription , Ġwhether Ġa Ġhome Ġcorresponds Ġto Ġa Ġhigh / medium / low Ġpopularity Ġlisting . ĠThe Ġarchitecture Ġof Ġthe ĠR NN / L ST M Ġemploys ĠT ensor flow âĢ Ļ s ĠDynamic ĠR NN Ġpackage . ĠEach Ġsentence Ġinput Ġis Ġfirst Ġfed Ġinto Ġan Ġembed ding Ġlayer , Ġwhere Ġthe Ġinput âĢ Ļ s Ġtext Ġis Ġconverted Ġto Ġa ĠGl o Ve Ġvector . ĠThese ĠGl o Ve Ġvectors Ġare Ġlearned Ġvia Ġa Ġglobal Ġword - word Ġco - occ urrence Ġmatrix Ġusing Ġour Ġcorpus Ġof ĠAirbnb Ġlisting Ġdescriptions Ġ[ 8 ]. ĠAt Ġeach Ġtime Ġstep , Ġthe ĠGl o Ve Ġvectors Ġare Ġthen Ġfed Ġinto Ġan ĠL ST M Ġlayer . ĠFor Ġeach Ġlayer , Ġthe Ġmodel Ġforward Ġpropag ates Ġthe Ġoutput Ġof Ġthe ĠL ST M Ġlayer Ġto Ġthe Ġnext Ġtime - step âĢ Ļ s ĠL ST M Ġlayer Ġvia Ġa Ġrect ified Ġlinear Ġunit Ġ( RL U ) Ġactivation Ġfunction . ĠEach Ġlayer Ġalso Ġpipes Ġthe Ġoutput Ġof Ġthe ĠL ST M Ġthrough Ġa Ġcross - ent ropy Ġoperation , Ġto Ġpredict , Ġfor Ġeach Ġtime - step , Ġthe Ġcategory Ġof Ġthe Ġinput Ġsequence . ĠWe Ġfinally Ġensemble Ġthese Ġpredictions , Ġto Ġcreate Ġthe Ġmodel âĢ Ļ s Ġcomplete Ġoutput Ġprediction . </s> Gener ative ĠAd vers arial ĠNetwork </s> Having Ġobserved Ġan Ġun ide al Ġperformance Ġon Ġthis Ġtask Ġ( see ĠExper iments Ġbelow ), Ġwe Ġturned Ġour Ġattention Ġto Ġbuilding Ġa Ġmodel Ġthat Ġcan Ġreplicate Ġthe Ġwriting Ġstyle Ġof Ġhigh Ġpopularity Ġlisting Ġdescriptions . ĠTo Ġsolve Ġthis Ġtask , Ġwe Ġdesigned Ġa Ġframework Ġfor Ġa Ġgeneral Ġadvers arial Ġnetwork . ĠThis Ġmodel Ġemploys Ġthe Ġstandard Ġset Ġup Ġof Ġa Ġgenerator Ġand Ġa Ġdiscrim inator , Ġbut Ġextends Ġthe Ġframework Ġwith Ġthe Ġadoption Ġof Ġthe ĠDie hl - Mart inez - K am al u Ġloss . </s> The Ġgenerator Ġis Ġdesigned Ġas Ġa Ġfeed - forward Ġneural Ġnetwork Ġwith Ġthree Ġlayers Ġof Ġdepth . ĠThe Ġinput Ġto Ġthe Ġgenerator Ġis Ġsimply Ġa Ġvector Ġof Ġrandom Ġnoise . ĠThis Ġinput Ġis Ġthen Ġfed Ġdirectly Ġto Ġthe Ġfirst Ġhidden Ġlayer Ġvia Ġa Ġlinear Ġtransformation . ĠBetween Ġthe Ġfirst Ġand Ġsecond Ġlayer Ġwe Ġapply Ġan Ġexponential Ġlinear Ġunit Ġ( EL U ) Ġas Ġa Ġnon - linear Ġactivation Ġfunction . ĠOur Ġreasoning Ġfor Ġdoing Ġso Ġis Ġbased Ġon Ġfindings Ġby ĠDash Ġet Ġal . Ġthat Ġthe Ġexperimental Ġaccuracy Ġof ĠEL Us Ġover Ġrect ified Ġlinear Ġunits Ġ( RL U ) Ġtends Ġto Ġbe Ġsomewhat Ġhigher Ġfor Ġgener ative Ġtasks Ġ[ 3 ]. ĠThen , Ġto Ġscale Ġthe Ġgenerator âĢ Ļ s Ġoutput Ġto Ġbe Ġin Ġthe Ġrange Ġ0 - 1 , Ġwe Ġapply Ġa Ġs igm oid Ġnon - linear ity Ġbetween Ġthe Ġsecond Ġand Ġthe Ġthird Ġlayer Ġof Ġthe Ġmodel . </s> The Ġdiscrim inator Ġsimilarly Ġused Ġa Ġfeed - forward Ġstructure Ġwith Ġthree Ġlayers Ġof Ġdepth . ĠThe Ġinput Ġto Ġthe Ġdiscrim inator Ġcomes Ġfrom Ġtwo Ġsources : Ġreal Ġdata Ġfed Ġdirectly Ġinto Ġthe Ġdiscrim inator Ġand Ġthe Ġdata Ġgenerated Ġby Ġthe Ġgenerator . ĠThis Ġinput Ġis Ġthen Ġpip ed Ġinto Ġthe Ġfirst Ġhidden Ġlayer . ĠAs Ġbefore , Ġan ĠEL U Ġtransformation Ġis Ġthen Ġapplied Ġbetween Ġthe Ġfirst Ġand Ġsecond Ġlayer , Ġas Ġwell Ġas Ġbetween Ġthe Ġsecond Ġand Ġthird Ġhidden Ġlayers . ĠFinally , Ġa Ġs igm oid Ġactivation Ġis Ġused Ġon Ġthe Ġoutput Ġof Ġthe Ġlast Ġhidden Ġlayer . ĠThis Ġs igm oid Ġactivation Ġis Ġimportant Ġsince Ġthe Ġoutput Ġof Ġour Ġdiscrim inator Ġis Ġa Ġbinary Ġboolean Ġthat Ġindicates Ġwhether Ġthe Ġdiscrim inator Ġbelieves Ġthe Ġinput Ġto Ġhave Ġbeen Ġreal Ġdata Ġor Ġdata Ġproduced Ġby Ġthe Ġgenerator . ĠThis Ġdiscrim inator Ġis Ġthus Ġtrained Ġto Ġminimize Ġthe Ġbinary Ġcross - ent ropy Ġloss Ġof Ġits Ġprediction Ġ( whether Ġthe Ġdata Ġwas Ġreal Ġor Ġfake ) Ġand Ġthe Ġreal Ġground - truth Ġof Ġeach Ġdata Ġpoint . </s> The Ġgeneral Ġframework Ġdefined Ġabove Ġwas Ġinspired Ġlargely Ġby Ġthe Ġopen - source Ġcode Ġof ĠNag ĠDev , Ġand Ġwas Ġbuilt Ġusing ĠPy tor ch Ġ[ 7 ]. ĠOne Ġkey Ġextension Ġto Ġthe Ġbasic ĠG AN Ġmodel , Ġhowever , Ġis Ġthe Ġloss Ġfunction Ġthat Ġwe Ġapply Ġto Ġthe Ġgenerator , Ġnamely Ġthe ĠDie hl - Mart inez - K am al u Ġ( DM K ) ĠLoss Ġwhich Ġwe Ġdefine Ġbelow . </s> The ĠDie hl - Mart inez - K am al u ĠLoss Ġis Ġa Ġweighted Ġcombination Ġof Ġa Ġbinary Ġcross Ġentropy Ġloss Ġwith Ġa Ġdot - product Ġattention Ġmetric Ġof Ġeach Ġuser - defined Ġkeyword Ġwith Ġthe Ġmodel âĢ Ļ s Ġgenerated Ġoutput . ĠForm ally , Ġthe Ġbinary Ġcross Ġentropy Ġ( BC E ) Ġloss Ġfor Ġone Ġexample Ġis Ġdefined Ġas : Ġ$ ĠBCE ( x , y ) Ġ= Ġy Ġ\ cd ot Ġlog x Ġ+ Ġ( 1 - y ) Ġ\ cd ot Ġlog ( 1 - x ), Ġ$ </s> where Ġx Ġis Ġdefined Ġas Ġthe Ġpredicted Ġlabel Ġfor Ġeach Ġsample Ġand Ġy Ġis Ġthe Ġtrue Ġlabel Ġ( i . e . Ġreal Ġversus Ġfake Ġdata ). ĠThe ĠDM K Ġloss Ġthen Ġcalculates Ġan Ġadditional Ġterm , Ġwhich Ġcorresponds Ġto Ġthe Ġdot - product Ġattention Ġof Ġeach Ġword Ġin Ġthe Ġgenerated Ġoutput Ġwith Ġeach Ġkeyword Ġspecified Ġby Ġthe Ġuser . ĠTo Ġillustrate Ġby Ġexample , Ġsay Ġa Ġuser Ġdesires Ġthe Ġgenerated Ġoutput Ġto Ġcontain Ġthe Ġkeywords , Ġ$\ l brace Ġsubway , Ġman hattan \ r brace Ġ$ Ġ. ĠThe Ġmodel Ġthen Ġconverts Ġeach Ġof Ġthese Ġkeywords Ġto Ġtheir Ġcorresponding Ġglove Ġvectors . ĠLet Ġus Ġdefine Ġthe Ġfollowing Ġnotation Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ Ġis Ġthe ĠGl o Ve Ġrepresentation Ġof Ġthe Ġword Ġapple , Ġand Ġlet Ġus Ġsuppose Ġthat Ġ$ g $ Ġis Ġthe Ġvector Ġof Ġword Ġembed d ings Ġgenerated Ġby Ġthe Ġgenerator . ĠThat Ġis , Ġ$ g _ 1 $ Ġis Ġthe Ġfirst Ġword Ġembed ding Ġof Ġthe Ġgenerator âĢ Ļ s Ġoutput . ĠLet Ġus Ġalso Ġsuppose Ġ$ k $ Ġis Ġa Ġvector Ġof Ġthe Ġkeywords Ġspecified Ġby Ġthe Ġuser . ĠIn Ġour Ġexamples , Ġ$ k $ Ġis Ġalways Ġin Ġ$ R ^{ 1 } $ Ġwith Ġ$ k _ 1 $ Ġone Ġequ aling Ġof Ġ$ âĢ ĺ sub way âĢ Ļ $ Ġor Ġ$ âĢ ĺ park ing âĢ Ļ $ Ġ. ĠThe Ġdot - product Ġterm Ġof Ġthe ĠDM K Ġloss Ġthen Ġcalculates Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ 0 Ġ. ĠWe igh ing Ġthis Ġterm Ġby Ġsome Ġhyper - param eter , Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ 1 Ġ, Ġthen Ġgives Ġus Ġthe Ġentire Ġdefinition Ġof Ġthe ĠDM K Ġloss : Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ 2 Ġ$ e ( âĢ ĺ apple âĢ Ļ ) $ 3 </s> Exper iments </s> In Ġseeking Ġto Ġanswer Ġthe Ġquestion Ġof Ġwhether Ġthe Ġoccupancy Ġrate Ġof Ġa Ġlisting Ġcould Ġbe Ġextracted Ġfrom Ġthe Ġlisting âĢ Ļ s Ġsummary , Ġwe Ġran Ġa Ġnumber Ġof Ġexperiments Ġon Ġour Ġfirst Ġmodel . ĠTwo Ġparameter izations Ġwhich Ġwe Ġpresent Ġhere Ġare Ġ( 1 ) Ġwhether Ġthe Ġword Ġvectors Ġused Ġin Ġthe Ġembed ding Ġlayer Ġare Ġtrained Ġon Ġour Ġcorpus Ġor Ġcome Ġpret rained Ġfrom ĠWikipedia Ġand ĠGig aw ord Ġand Ġ( 2 ) Ġwhether Ġen semb ling Ġor Ġthe Ġfinal Ġhidden Ġstate Ġin Ġisolation Ġare Ġused Ġto Ġmake Ġa Ġprediction Ġfor Ġthe Ġsequence . ĠCommon Ġto Ġall Ġexperiments Ġwas Ġour Ġdecision Ġto Ġuse Ġan ĠAdam Ġoptim izer , Ġ16 ĠL ST M Ġunits , Ġ50 - dimensional ĠGl o Ve Ġvectors , Ġand Ġa Ġ70 - 30 Ġsplit Ġin Ġtrain Ġand Ġtest Ġdata . </s> Over Ġten Ġepoch s , Ġthe Ġmodel Ġparameter ization Ġwhich Ġperforms Ġthe Ġbest Ġuses ĠGl o Ve Ġvectors Ġtrained Ġon Ġa Ġcorpus Ġconsisting Ġof Ġall Ġlisting Ġdescriptions Ġand Ġen semb ling Ġto Ġmake Ġits Ġclass Ġprediction . ĠAs Ġa Ġresult , Ġour Ġfindings Ġare Ġwell Ġin - line Ġwith Ġthose Ġpresented Ġby ĠRad ford Ġet . Ġal Ġwho Ġunderscore Ġthe Ġimportance Ġof Ġtraining Ġword Ġembed d ings Ġon Ġa Ġdata - specific Ġcorpus Ġfor Ġbest Ġresults Ġon Ġgener ative Ġtasks Ġ[ 9 ]. </s> That Ġsaid , Ġthese Ġresults , Ġthough Ġthey Ġdo Ġshow Ġa Ġmarginal Ġincrease Ġin Ġdev Ġaccuracy Ġand Ġa Ġdecrease Ġin ĠCE Ġloss , Ġsuggest Ġthat Ġperhaps Ġlisting Ġdescription Ġis Ġnot Ġtoo Ġpredictive Ġof Ġoccupancy Ġrate Ġgiven Ġour Ġparameter izations . ĠWhile Ġthe Ġlisting Ġdescription Ġis Ġsurely Ġan Ġinfluential Ġmetric Ġin Ġdetermining Ġthe Ġquality Ġof Ġa Ġlisting , Ġother Ġfactors Ġsuch Ġas Ġlocation , Ġamenities , Ġand Ġhome Ġtype Ġmight Ġplay Ġa Ġlarger Ġrole Ġin Ġthe Ġconsumer 's Ġdecision . ĠWe Ġwere Ġhopeful Ġthat Ġthese Ġfactors Ġwould Ġbe Ġrepresented Ġin Ġthe Ġprice Ġper Ġbedroom Ġof Ġthe Ġlisting ĠâĢĵ Ġour Ġcontrol Ġvariable ĠâĢĵ Ġbut Ġthe Ġrelationship Ġmay Ġnot Ġhave Ġbeen Ġstrong Ġenough . </s> However , Ġshould Ġa Ġstrong Ġrelationship Ġactually Ġexist Ġand Ġthere Ġbe Ġinstead Ġa Ġproblem Ġwith Ġour Ġmethod , Ġthere Ġare Ġa Ġfew Ġpossibilities Ġof Ġwhat Ġwent Ġwrong . ĠWe Ġassumed Ġthat Ġlistings Ġwith Ġsimilar Ġoccupancy Ġrates Ġwould Ġhave Ġsimilar Ġlisting Ġdescriptions Ġregardless Ġof Ġprice , Ġwhich Ġis Ġnot Ġnecessarily Ġa Ġstrong Ġassumption . ĠThis Ġis Ġcoupled Ġwith Ġan Ġunexpected Ġsp ars eness Ġof Ġclean Ġdata . ĠWith Ġover Ġ40 , 000 Ġlistings , Ġwe Ġdid Ġnot Ġexpect Ġto Ġsee Ġsuch Ġpoor Ġattention Ġto Ġorth ography Ġin Ġwhat Ġare Ġessentially Ġpublic Ġadvertisements Ġof Ġthe Ġproperties . ĠIn Ġthis Ġway , Ġour Ġdecision Ġto Ġuse Ġa Ġwindow Ġsize Ġof Ġ5 , Ġa Ġminimum Ġoccurrence Ġcount Ġof Ġ2 , Ġand Ġa Ġdimension ality Ġof Ġ50 Ġwhen Ġtraining Ġour ĠGl o Ve Ġvectors Ġwas Ġad Ġhoc . </s> Se eking Ġto Ġcreate Ġa Ġmodel Ġwhich Ġcould Ġgenerate Ġand Ġdiscriminate Ġa ĠâĢ ľ high - occup ancy Ġlisting Ġdescription âĢ Ŀ , Ġwe Ġwanted Ġto Ġevaluate Ġthe Ġcapabilities Ġof Ġa Ġgener ative Ġadvers arial Ġnetwork Ġtrained Ġon Ġeither Ġthe Ġstandard Ġbinary Ġcross - ent ropy Ġloss Ġor Ġthe ĠDM K Ġloss Ġproposed Ġabove . ĠCommon Ġto Ġboth Ġmodels Ġwas Ġthe Ġdecision Ġto Ġalternate Ġbetween Ġtraining Ġthe Ġgenerator Ġfor Ġ50 Ġsteps Ġand Ġthe Ġdiscrim inator Ġfor Ġ2000 Ġsteps . ĠWe Ġleave Ġfurther Ġtuning Ġof Ġthe Ġmodels Ġto Ġfuture Ġresearch Ġas Ġeach Ġoccasionally Ġfalls Ġinto Ġun ide al Ġlocal Ġoptim a Ġwithin Ġ20 Ġiterations . ĠOne Ġpotential Ġculprit Ġis Ġthe Ġstep Ġimbalance Ġbetween Ġgenerator Ġand Ġdiscrim inator ĠâĢĵ Ġshould Ġeither Ġlearn Ġat Ġa Ġmuch Ġfaster Ġrate Ġthan Ġthe Ġother , Ġone Ġcomponent Ġis Ġliable Ġto Ġbe ĠâĢ ľ def e ated " Ġand Ġcease Ġto Ġlearn Ġthe Ġtraining Ġdata . </s> Qual itatively Ġthe Ġnetwork Ġtrained Ġon Ġthe ĠDM K Ġloss Ġshows Ġgreat Ġpromise . ĠWith Ġrespect Ġto Ġthe Ġtwo Ġexperiments Ġpresented Ġhere , Ġwe Ġhave Ġshown Ġthat Ġit Ġis Ġpossible Ġto Ġintroduce Ġa Ġmeasure Ġof Ġsuggestion Ġin Ġthe Ġtext Ġproduced Ġby Ġthe Ġgenerator . ĠWhile Ġthis Ġmodel Ġis Ġalso Ġsubject Ġto Ġa Ġrapid Ġdead lock Ġbetween Ġgenerator Ġand Ġdiscrim inator , Ġit Ġis Ġinteresting Ġto Ġsee Ġhow Ġthe Ġintroduction Ġof Ġkeywords Ġis Ġgradual Ġand Ġaffects Ġthe Ġprox imal Ġtokens Ġincluded Ġin Ġthe Ġoutput . ĠThis Ġbehavior Ġwas Ġmade Ġpossible Ġby Ġpaying Ġclose Ġattention Ġto Ġthe Ġhyper param eter Ġ$\ gam ma Ġ$ Ġ, Ġthe Ġweight Ġgiven Ġto Ġthe Ġdot Ġproduct Ġattention Ġterm Ġof Ġthe ĠDM K Ġloss . ĠAfter Ġmanual Ġtuning , Ġwe Ġsettle Ġon Ġ$\ gam ma Ġ= 0 . 000 45 $ Ġfor Ġthis Ġweight . ĠBelow , Ġwe Ġillustrate Ġmodel Ġoutputs Ġusing Ġdifferent Ġvalues Ġof ĠGamma . ĠAs Ġis Ġapparent , Ġfor Ġa Ġhyper - param eter Ġvalue Ġless Ġthan Ġroughly Ġ$\ gam ma Ġ= Ġ0 . 000 4 $ Ġ, Ġthe Ġmodel Ġtends Ġto Ġignore Ġthe Ġimportance Ġof Ġthe Ġkeyword Ġweights . ĠConversely , Ġwith Ġa Ġ$\ gam ma Ġ$ Ġvalue Ġhigher Ġthan Ġ$ 0 . 000 5 $ Ġ, Ġthe Ġmodel Ġtends Ġtowards Ġoverweight ing Ġthe Ġrepresentation Ġof Ġthe Ġkeywords Ġin Ġthe Ġmodel Ġoutput . </s> Conclusion </s> We Ġhope Ġthat Ġthis Ġresearch Ġpaper Ġestablishes Ġa Ġfirst Ġattempt Ġat Ġusing Ġgener ative Ġmachine Ġlearning Ġmodels Ġfor Ġthe Ġpurposes Ġof Ġmarketing Ġon Ġpeer - to - peer Ġplatforms . ĠAs Ġthe Ġuse Ġof Ġthese Ġmarkets Ġbecomes Ġmore Ġwidespread , Ġthe Ġuse Ġof Ġself - brand ing Ġand Ġmarketing Ġtools Ġwill Ġgrow Ġin Ġimportance . ĠThe Ġdevelopment Ġof Ġtools Ġlike Ġthe ĠDM K Ġloss Ġin Ġcombination Ġwith ĠG AN s Ġdemonstrates Ġthe Ġenormous Ġpotential Ġthese Ġframeworks Ġcan Ġhave Ġin Ġsolving Ġproblems Ġthat Ġinevitably Ġarise Ġon Ġpeer - to - peer Ġplatforms . ĠCertainly , Ġhowever , Ġmore Ġworks Ġremains Ġto Ġbe Ġdone Ġin Ġthis Ġfield , Ġand Ġrecent Ġdevelopments Ġin Ġun super vised Ġgener ative Ġmodels Ġalready Ġpromise Ġpossible Ġextensions Ġto Ġour Ġanalysis . </s> For Ġexample , Ġsince Ġthe Ġintroduction Ġof ĠG AN s , Ġresearch Ġgroups Ġhave Ġstudied Ġhow Ġvari ational Ġaut oen cod ers Ġcan Ġbe Ġincorporated Ġinto Ġthese Ġmodels , Ġto Ġincrease Ġthe Ġclarity Ġand Ġaccuracy Ġof Ġthe Ġmodels âĢ Ļ Ġoutputs . ĠWang Ġet Ġal . Ġin Ġparticular Ġdemonstrate Ġhow Ġusing Ġa Ġvari ational Ġaut oen c oder Ġin Ġthe Ġgenerator Ġmodel Ġcan Ġincrease Ġthe Ġaccuracy Ġof Ġgenerated Ġtext Ġsamples Ġ[ 11 ]. ĠMost Ġpromising ly , Ġthe Ġdata Ġused Ġby Ġthe Ġgroup Ġwas Ġa Ġlarge Ġdata Ġset Ġof ĠAmazon Ġcustomer Ġreviews , Ġwhich Ġin Ġmany Ġways Ġparallels Ġthe Ġtone Ġand Ġsemantic Ġstructure Ġof ĠAirbnb Ġlisting Ġdescriptions . ĠMore Ġgenerally , ĠBowman Ġet Ġal . Ġdemonstrate Ġhow Ġthe Ġuse Ġof Ġvari ational Ġaut oen cod ers Ġpresents Ġa Ġmore Ġaccurate Ġmodel Ġfor Ġtext Ġgeneration , Ġas Ġcompared Ġto Ġstandard Ġrecurrent Ġneural Ġnetwork Ġlanguage Ġmodels Ġ[ 1 ]. </s> For Ġfuture Ġwork , Ġwe Ġmay Ġalso Ġconsider Ġexperimenting Ġwith Ġdifferent Ġforms Ġof Ġword Ġembed d ings , Ġsuch Ġas Ġimproved Ġword Ġvectors Ġ( I W V ) Ġwhich Ġwere Ġsuggested Ġby ĠRez ae inia Ġet Ġal Ġ[ 10 ]. ĠThese ĠIM V Ġword Ġembed d ings Ġare Ġtrained Ġin Ġa Ġsimilar Ġfashion Ġto ĠGl o Ve Ġvectors , Ġbut Ġalso Ġencode Ġadditional Ġinformation Ġabout Ġeach Ġword , Ġlike Ġthe Ġword âĢ Ļ s Ġpart Ġof Ġspeech . ĠFor Ġour Ġmodel , Ġwe Ġmay Ġconsider Ġencoding Ġsimilar Ġinformation Ġin Ġorder Ġfor Ġthe Ġgenerator Ġto Ġmore Ġeasily Ġlearn Ġcommon Ġsemantic Ġpatterns Ġinherent Ġto Ġthe Ġmarketing Ġdata Ġbeing Ġanalyzed .