Hypertext documents , such as web pages and academic papers , are of great importance in delivering information in our daily life . Although being effective on plain documents , conventional text embedding methods suffer from information loss if directly adapted to hyper-documents . In this paper , we propose a general embedding approach for hyper-documents , namely , hyperdoc2vec , along with four criteria characterizing necessary information that hyper-document embedding models should preserve . Systematic comparisons are conducted between hyperdoc2vec and several competitors on two tasks , i.e. , paper classification and citation recommendation , in the academic paper domain . Analyses and experiments both validate the superiority of hyperdoc2vec to other models w.r.t . the four criteria . The ubiquitous World Wide Web has boosted research interests on hypertext documents , e.g. , personal webpages ( Lu and Getoor , 2003 ) , Wikipedia pages ( Gabrilovich and Markovitch , 2007 ) , as well as academic papers ( Sugiyama and Kan , 2010 ) . Unlike independent plain documents , a hypertext document ( hyper-doc for short ) links to another hyper-doc by a hyperlink or citation mark in its textual content . Given this essential distinction , hyperlinks or citations are worth specific modeling in many tasks such as link-based classification ( Lu and Getoor , 2003 ) , web retrieval ( Page et al . , 1999 ) , entity linking ( Cucerzan , 2007 ) , and citation recommendation ( He et al . , 2010 ) . To model hypertext documents , various efforts ( Cohn and Hofmann , 2000 ; Kataria et al . , 2010 ; Perozzi et al . , 2014 ; Zwicklbauer et al . , 2016 ; Wang et al . , 2016 ) have been made to depict networks of hyper-docs as well as their content . Among potential techniques , distributed representation ( Mikolov et al . , 2013 ; Le and Mikolov , 2014 ) tends to be promising since its validity and effectiveness are proven for plain documents on many natural language processing ( NLP ) tasks . Conventional attempts on utilizing embedding techniques in hyper-doc-related tasks generally fall into two types . The first type ( Berger et al . , 2017 ; Zwicklbauer et al . , 2016 ) simply downcasts hyper-docs to plain documents and feeds them into word2vec ( Mikolov et al . , 2013 ) ( w2v for short ) or doc2vec ( Le and Mikolov , 2014 ) ( d2v for short ) . These approaches involve downgrading hyperlinks and inevitably omit certain information in hyper-docs . However , no previous work investigates the information loss , and how it affects the performance of such downcasting-based adaptations . The second type designs sophisticated embedding models to fulfill certain tasks , e.g. , citation recommendation ( Huang et al . , 2015b ) , paper classification ( Wang et al . , 2016 ) , and entity linking ( Yamada et al . , 2016 ) , etc . These models are limited to specific tasks , and it is yet unknown whether embeddings learned for those particular tasks can generalize to others . Based on the above facts , we are interested in two questions : • What information should hyper-doc embedding models preserve , and what nice property should they possess ? • Is there a general approach to learning taskindependent embeddings of hyper-docs ? To answer the two questions , we formalize the hyper-doc embedding task , and propose four criteria , i.e. , content awareness , context awareness , newcomer friendliness , and context intent aware-ness , to assess different models . Then we discuss simple downcasting-based adaptations of existing approaches w.r.t . the above criteria , and demonstrate that none of them satisfy all four . To this end , we propose hyperdoc2vec ( h-d2v for short ) , a general embedding approach for hyperdocs . Different from most existing approaches , h-d2v learns two vectors for each hyper-doc to characterize its roles of citing others and being cited . Owning to this , h-d2v is able to directly model hyperlinks or citations without downgrading them . To evaluate the learned embeddings , we employ two tasks in the academic paper domain1 , i.e. , paper classification and citation recommendation . Experimental results demonstrate the superiority of h-d2v . Comparative studies and controlled experiments also confirm that h-d2v benefits from satisfying the above four criteria . We summarize our contributions as follows : • We propose four criteria to assess different hyper-document embedding models . • We propose hyperdoc2vec , a general embedding approach for hyper-documents . • We systematically conduct comparisons with competing approaches , validating the superiority of h-d2v in terms of the four criteria . We focus on the hyper-doc embedding problem . We propose that hyper-doc embedding algorithms should be content aware , context aware , newcomer friendly , and context intent aware . To meet all four criteria , we propose a general approach , hyperdoc2vec , which assigns two vectors to each hyper-doc and models citations in a straightforward manner . In doing so , the learned embeddings satisfy all criteria , which no existing model is able to . For evaluation , paper classification and citation recommendation are conducted on three academic paper datasets . Results confirm the effectiveness of our approach . Further analyses also demonstrate that possessing the four properties helps h-d2v outperform other models .
Existing text embedding methods do not take structures of hyper-documents into account losing useful properties for downstream tasks.
They propose an embedding method for hyper-documents that learns citation information along with four criteria to assess the properties the models should preserve.
The proposed model satisfies all of the introduced criteria and performs two tasks in the academic domain better than existing models.