In this work, we address the problem of recommending jobs to university students. For this, we explore the utilization of neural item embeddings for the task of content-based recommendation, and we propose to integrate the factors of frequency and recency of interactions with job postings to combine these item embeddings. We evaluate our job recommendation system on a dataset of the Austrian student job portal Studo using prediction accuracy, diversity and an adapted novelty metric. This paper demonstrates that utilizing frequency and recency of interactions with job postings for combining item embeddings results in a robust model with respect to accuracy and diversity, which also provides the best adapted novelty results.
Models that train item embeddings, such as Word2Vec @cite_8 , have been shown to provide good results in many natural language processing tasks. As such, research on improving the recommendation performance by employing word embedding vectors is gaining more momentum. As an example, the authors of @cite_13 used the learned word embeddings trained on Wikipedia for content-based recommender systems. Additionally, in @cite_17 , word embeddings were employed to represent job postings. In order to achieve a high accuracy, a large number of embedding dimensions were required. The authors of @cite_14 utilized item embeddings on top of Collaborative Filtering to tackle the cold-start problem for job recommendations. In addition to that, in the RecSys Challenge of 2016, @cite_16 suggested that time-dependent models can enhance the job recommendation performance, which further motivates the applicability of our BLL-based approach.
@cite_14 @cite_8 @cite_16 @cite_13 @cite_17
2752172973 2613830894 2546915671 2786148476
The success of neural network methods for computing word embeddings has motivated methods for generating semantic embeddings of longer pieces of text, such as sentences and paragraphs. Surprisingly, (ICLR'16) showed that such complicated methods are outperformed, especially in out-of-domain (transfer learning) settings, by simpler methods involving mild retraining of word embeddings and basic linear regression. The method of requires retraining with a substantial labeled dataset such as Paraphrase Database (, 2013). @PARASPLIT The current paper goes further, showing that the following completely unsupervised sentence embedding is a formidable baseline: Use word embeddings computed using one of the popular methods on unlabeled corpus like Wikipedia, represent the sentence by a weighted average of the word vectors, and then modify them a bit using PCA SVD. This weighting improves performance by about 10 to 30 in textual similarity tasks, and beats sophisticated supervised methods including RNN's and LSTM's. It even improves 's embeddings. This simple method should be used as the baseline to beat in future, especially when labeled training data is scarce or nonexistent. @PARASPLIT The paper also gives a theoretical explanation of the success of the above unsupervised method using a latent variable generative model for sentences, which is a simple extension of the model in (TACL'16) with new "smoothing" terms that allow for words occurring out of context, as well as high probabilities for words like and, not in all contexts. XING is a leading career-oriented social networking site in Europe, which usually recommend job ads to their customers. One of the widely used methods in Recomender Systems is content-based filtering, which analyzes the description of item characteristics and the user profile illustrating user's preferences. Due to the sparsity of its dataset, i.e. many job postings are rarely interacted with, XING has been using content-based recommender system to promote the quality of the recommendations. Recent word embedding technique learns semantically meaningful representations for words from co-occurrence in sentences, which enables the effective comparison between words. Based on the Word2Vec technique, XING represents job postings by the average embedding over words they contain. This study explores three alternative methods to represent job postings for the task of recommending jobs to users. In the first experiment, we explore whether the use of a subset of words is more effective to represent the job postings. In the second experiment, instead of averaging over word embeddings, we directly learn document embeddings using Paragraph2Vec. And finally, the third experiment uses Word Mover's Distance to estimate the similarity between job postings. Our experiments show that the embeddings that are learned with Paragraph2Vec result in a better estimation of which job postings are similar, but only when high-dimensional settings are used. The Word Mover's Distance algorithm is computationally expensive, therefore we use existing lower-bounds that allowed us to complete a small-scale experiment within the available time. The results indicate that Word Mover's Distance is not as effective as the average over word embeddings and Paragraph2Vec. In the final part of this thesis, we present the Link2Vec, a novel item representation method based on Word2Vec, which learns semantic representations for items based on the context surrounding the hyperlinks that refer to the item, e.g. hyperlinks to the item's Wikipedia page. Our experiments show that the effectiveness of the embeddings learned with Link2Vec improves with the amount of training data. For the evaluation on the MovieLens dataset, we only obtained a limited set of hyperlinks, which resulted in results that approximate a baseline that uses the average over word embeddings. Recurrent neural networks (RNNs) have achieved state-of-the-art performances in many natural language processing tasks, such as language modeling and machine translation. However, when the vocabulary is large, the RNN model will become very big (e.g., possibly beyond the memory capacity of a GPU device) and its training will become very inefficient. In this work, we propose a novel technique to tackle this challenge. The key idea is to use 2-Component (2C) shared embedding for word representations. We allocate every word in the vocabulary into a table, each row of which is associated with a vector, and each column associated with another vector. Depending on its position in the table, a word is jointly represented by two components: a row vector and a column vector. Since the words in the same row share the row vector and the words in the same column share the column vector, we only need @math vectors to represent a vocabulary of @math unique words, which are far less than the @math vectors required by existing approaches. Based on the 2-Component shared embedding, we design a new RNN algorithm and evaluate it using the language modeling task on several benchmark datasets. The results show that our algorithm significantly reduces the model size and speeds up the training process, without sacrifice of accuracy (it achieves similar, if not better, perplexity as compared to state-of-the-art language models). Remarkably, on the One-Billion-Word benchmark Dataset, our algorithm achieves comparable perplexity to previous language models, whilst reducing the model size by a factor of 40-100, and speeding up the training process by a factor of 2. We name our proposed algorithm to reflect its very small model size and very high training speed. Learning effective text representations is a key foundation for numerous machine learning and NLP applications. While the celebrated Word2Vec technique yields semantically rich word representations, it is less clear whether sentence or document representations should be built upon word representations or from scratch. Recent work has demonstrated that a distance measure between documents called (WMD) that aligns semantically similar words, yields unprecedented KNN classification accuracy. However, WMD is very expensive to compute, and is harder to apply beyond simple KNN than feature embeddings. In this paper, we propose the (WME), a novel approach to building an unsupervised document (sentence) embedding from pre-trained word embeddings. Our technique extends the theory of to show convergence of the inner product between WMEs to a positive-definite kernel that can be interpreted as a soft version of (inverse) WMD. The proposed embedding is more efficient and flexible than WMD in many situations. As an example, WME with a simple linear classifier reduces the computational cost of WMD-based KNN in document length and in number of samples, while simultaneously improving accuracy. In experiments on 9 benchmark text classification datasets and 22 textual similarity tasks the proposed technique consistently matches or outperforms state-of-the-art techniques, with significantly higher accuracy on problems of short length.