A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> Dictionaries and phrase tables are the basis of modern statistical machine translation systems. This paper develops a method that can automate the process of generating and extending dictionaries and phrase tables. Our method can translate missing word and phrase entries by learning language structures based on large monolingual data and mapping between languages from small bilingual data. It uses distributed representation of words and learns a linear mapping between vector spaces of languages. Despite its simplicity, our method is surprisingly effective: we can achieve almost 90% precision@5 for translation of words between English and Spanish. This method makes little assumption about the languages, so it can be used to extend and refine dictionaries and translation tables for any language pairs. <s> BIB001 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities. <s> BIB002 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the training speed. By subsampling of the frequent words we obtain significant speedup and also learn more regular word representations. We also describe a simple alternative to the hierarchical softmax called negative sampling. ::: ::: An inherent limitation of word representations is their indifference to word order and their inability to represent idiomatic phrases. For example, the meanings of "Canada" and "Air" cannot be easily combined to obtain "Air Canada". Motivated by this example, we present a simple method for finding phrases in text, and show that learning good vector representations for millions of phrases is possible. <s> BIB003 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> Recent methods for learning vector space representations of words have succeeded in capturing fine-grained semantic and syntactic regularities using vector arithmetic, but the origin of these regularities has remained opaque. We analyze and make explicit the model properties needed for such regularities to emerge in word vectors. The result is a new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods. Our model efficiently leverages statistical information by training only on the nonzero elements in a word-word cooccurrence matrix, rather than on the entire sparse matrix or on individual context windows in a large corpus. The model produces a vector space with meaningful substructure, as evidenced by its performance of 75% on a recent word analogy task. It also outperforms related models on similarity tasks and named entity recognition. <s> BIB004 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> The word2vec software of Tomas Mikolov and colleagues (this https URL ) has gained a lot of traction lately, and provides state-of-the-art word embeddings. The learning models behind the software are described in two research papers. We found the description of the models in these papers to be somewhat cryptic and hard to follow. While the motivations and presentation may be obvious to the neural-networks language-modeling crowd, we had to struggle quite a bit to figure out the rationale behind the equations. ::: This note is an attempt to explain equation (4) (negative sampling) in "Distributed Representations of Words and Phrases and their Compositionality" by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean. <s> BIB005 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> Next basket recommendation is a crucial task in market basket analysis. Given a user's purchase history, usually a sequence of transaction data, one attempts to build a recommender that can predict the next few items that the user most probably would like. Ideally, a good recommender should be able to explore the sequential behavior (i.e., buying one item leads to buying another next), as well as account for users' general taste (i.e., what items a user is typically interested in) for recommendation. Moreover, these two factors may interact with each other to influence users' next purchase. To tackle the above problems, in this paper, we introduce a novel recommendation approach, namely hierarchical representation model (HRM). HRM can well capture both sequential behavior and users' general taste by involving transaction and user representations in prediction. Meanwhile, the flexibility of applying different aggregation operations, especially nonlinear operations, on representations allows us to model complicated interactions among different factors. Theoretically, we show that our model subsumes several existing methods when choosing proper aggregation operations. Empirically, we demonstrate that our model can consistently outperform the state-of-the-art baselines under different evaluation metrics on real-world transaction data. <s> BIB006 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> One crucial task in recommendation is to predict what a user will buy next given her shopping history. In this paper, we propose a novel neural network to complete this task. The model consists of an embedding layer, a hidden layer and an output layer. Firstly, the distributed representations of the user and the items bought before are obtained and used to form a feature vector by the embedding layer. Then the hidden layer transforms the feature vector to another space by a non-linear operator. Finally, the softmax operator is adopted to output the probabilities of next items. We can see that the model elegantly involves both the user's general interest and the sequential dependencies between items for prediction. Experimental results on two real datasets prove the effectiveness of our model. <s> BIB007 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> We propose Meta-Prod2vec, a novel method to compute item similarities for recommendation that leverages existing item metadata. Such scenarios are frequently encountered in applications such as content recommendation, ad targeting and web search. Our method leverages past user interactions with items and their attributes to compute low-dimensional embeddings of items. Specifically, the item metadata is injected into the model as side information to regularize the item embeddings. We show that the new item representations lead to better performance on recommendation tasks on an open music dataset. <s> BIB008 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> Contextual factors can benefit music recommendation and retrieval tasks remarkably. However, how to acquire and utilize the contextual information still need to be studied. In this paper, we propose a context aware music recommendation approach, which can recommend music appropriate for users' contextual preference for music. In analogy to matrix factorization methods for collaborative filtering, the proposed approach does not require songs to be described by features beforehand, but it learns music pieces' embeddings (vectors in low-dimensional continuous space) from music playing records and corresponding metadata and infer users' general and contextual preference for music from their playing records with the learned embedding. Then, our approach can recommend appropriate music pieces. Experimental evaluations on a real world dataset show that the proposed approach outperforms baseline methods. <s> BIB009 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> We present a method of generating item recommendations by learning item feature vector embeddings. Our work is analogous to approaches like Word2Vec or Glove used to generate a good vector representation of words in a natural language corpus. We treat the items that a user interacted with as analogous to words and the string of items interacted with in a session as sentences. Our embedding generates semantically related clusters and the item vectors generated can be used to compute item similarity which can be used to drive product recommendations. Our method also allows us to use the feature vectors in other machine learning systems. We validate our method on the MovieLens dataset. <s> BIB010 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> Social network platforms can use the data produced by their users to serve them better. One of the services these platforms provide is recommendation service. Recommendation systems can predict the future preferences of users using their past preferences. In the recommendation systems literature there are various techniques, such as neighborhood based methods, machine-learning based methods and matrix-factorization based methods. In this work, a set of well known methods from natural language processing domain, namely Word2Vec, is applied to recommendation systems domain. Unlike previous works that use Word2Vec for recommendation, this work uses non-textual features, the check-ins, and it recommends venues to visit/check-in to the target users. For the experiments, a Foursquare check-in dataset is used. The results show that use of continuous vector space representations of items modeled by techniques of Word2Vec is promising for making recommendations. <s> BIB011 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> In this paper, we present a novel perspective to address recommendation tasks by utilizing the network representation learning techniques. Our idea is based on the observation that the input of typical recommendation tasks can be formulated as graphs. Thus, we propose to use the k-partite adoption graph to characterize various kinds of information in recommendation tasks. Once the historical adoption records have been transformed into a graph, we can apply the network embedding approach to learn vertex embeddings on the k-partite adoption network. Embeddings for different kinds of information are projected into the same latent space, where we can easily measure the relatedness between multiple vertices on the graph using some similarity measurements. In this way, the recommendation task has been casted into a similarity evaluation process using embedding vectors. The proposed approach is both general and scalable. To evaluate the effectiveness of the proposed approach, we construct extensive experiments on two different recommendation tasks using real-world datasets. The experimental results have shown the superiority of our approach. To the best of our knowledge, it is the first time that a network representation learning approach has been applied to recommendation tasks. <s> BIB012 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> To predict customer’s next choice in the context of what he/she has bought in a session is interesting and critical in the transaction domain especially for online shopping. Precise prediction leads to high quality recommendations and thus high benefit. Such kind of recommendation is usually formalized as transaction-based recommender systems (TBRS). Existing TBRS either tend to recommend popular items while ignore infrequent and newly-released ones (e.g., pattern-based RS) or assume a rigid order between items within a transaction (e.g., Markov Chain-based RS) which does not satisfy real-world cases in most time. In this paper, we propose a neural network-based comprehensive transaction embedding model (NTEM) which can effectively perceive the next choice in a transaction context. Specifically, we learn these comprehensive embeddings of both items and their features from relaxed ordered transactions. The relevance between items revealed by the transactions is encoded into such embeddings. With rich information embedded, such embeddings are powerful to predict the next choices given those already bought items. NTEM is a shallow wide-in-wide-out network, which is more efficient than deep networks considering large numbers of items and transactions. Experimental results on real-world datasets show that NTEM outperforms three typical TBRS models FPMC, PRME and GRU4Rec in terms of recommendation accuracy and novelty. Our implementation is available at https://github.com/shoujin88/NTEM-model. <s> BIB013 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> Recent methods for learning vector space representations of words, word embedding, such as GloVe and Word2Vec have succeeded in capturing fine-grained semantic and syntactic regularities. We analyzed the effectiveness of these methods for e-commerce recommender systems by transferring the sequence of items generated by users' browsing journey in an e-commerce website into a sentence of words. We examined the prediction of fine-grained item similarity (such as item most similar to iPhone 6 64GB smart phone) and item analogy (such as iPhone 5 is to iPhone 6 as Samsung S5 is to Samsung S6) using real life users' browsing history of an online European department store. Our results reveal that such methods outperform related models such as singular value decomposition (SVD) with respect to item similarity and analogy tasks across different product categories. Furthermore, these methods produce a highly condensed item vector space representation, item embedding, with behavioral meaning sub-structure. These vectors can be used as features in a variety of recommender system applications. In particular, we used these vectors as features in a neural network based models for anonymous user recommendation based on session's first few clicks. It is found that recurrent neural network that preserves the order of user's clicks outperforms standard neural network, item-to-item similarity and SVD (recall@10 value of 42% based on first three clicks) for this task. <s> BIB014 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> In this paper, we propose a novel Product Graph Embedding (PGE) model to investigate time-aware product recommendation by leveraging the network representation learning technique. Our model captures the sequential influences of products by transforming the historical purchase records into a product graph. Then the product can be transformed into a low dimensional vector by the network embedding model. Once products are projected into the latent space, we present a novel method to compute user's latest preferences, which projects users into the same latent space as products. This method is based on time-decay functions and the embedding of sequential products that the user purchased. Thus, relatedness between a product and a user can be measured by the similarity between the embedding vectors which represent the product and the user's preferences. The experimental results on purchase records crawled from JINGDONG, show the superiority of our proposed framework for personalized product recommendation. <s> BIB015 </s> A Survey on Session-based Recommender Systems <s> Neural Model-based Approaches <s> Contextual factors greatly affect users' preferences for music, so they can benefit music recommendation and music retrieval. However, how to acquire and utilize the contextual information is still facing challenges. This paper proposes a novel approach for context-aware music recommendation, which infers users' preferences for music, and then recommends music pieces that fit their real-time requirements. Specifically, the proposed approach first learns the low dimensional representations of music pieces from users' music listening sequences using neural network models. Based on the learned representations, it then infers and models users' general and contextual preferences for music from users' historical listening records. Finally, music pieces in accordance with user's preferences are recommended to the target user. Extensive experiments are conducted on real world datasets to compare the proposed method with other state-of-the-art recommendation methods. The results demonstrate that the proposed method significantly outperforms those baselines, especially on sparse data. <s> BIB016
Derived by the powerful generalization and representation capability of neural networks, a series of neural model-based approaches have been developed to model the comprehensive relations between item features, items and sessions for recommendations. Generally, neural model-based SBRS can be divided into shallow neural models and deep neural models according to the number of layers incorporated into the neural networks. Next, we introduce these two kinds of recommendation models respectively. 7.3.1 Shallow Neural Models. Shallow neural models, sometimes called embedding models, usually contain a shallow network structure  which maps the items within sessions into a latent space and then capture inter-item relations more easily in this latent space. The mechanism behind is that, when mapping items into a multi-dimensional latent space, their positions in such a space reflect their relations. For instance, the items within a shorter distance have higher similarity and are more relevant to each other. To this end, the latent numerical vector representation of each item contain much richer information than the original item ID. This is mainly inspired by the great success of the recently proposed word embedding techniques including Skip-gram BIB004 and the CBOW model BIB001 in natural language processing BIB002 BIB003 . A representative shallow neural model-based SBRS is  . It designed a shallow network with a wide-in-wideout structure to firstly map user IDs and the corresponding item IDs into latent vector representations and then combine them together as the given context representation, which is lastly fed into the output layer (a softmax layer) to predict the corresponding next item. To be specific, the shallow network embeds a user u and an item i into a latent vector using the logistic function δ (·) for nonlinear transformation: where W 1 ∈ R K × |U | and W 2 ∈ R L× |I | are the weight matrix to connect user input and session context input to the hidden layer respectively. In this way, each user and each item are represented by a numerical vector v u ∈ [0, 1] K and v i ∈ [0, 1] L respectively, similar to the vector representation of each word in the Word2Vec model BIB005 . In addition, Wang etc. incorporated item features into the network to embed an item ID and its features simultaneously to tackle the cold-start recommendation issues BIB013 . The work in BIB008 BIB009 also incorporated side information to help with building a more informative item embedding. To attentively learn the really relevance scale of different contextual items to the target item, the attention mechanism is adapted into the embedding process of items  . The work in BIB006 and BIB007 learned a hierarchical representation for the next basket recommendations. Furthermore, there are more similar SBRS built on item embedding inspired by the Word2Vec model like  BIB014 BIB010 BIB015 BIB011 BIB016 BIB012 .