The topic modeling discovers the latent topic probability of given the text documents. To generate the more meaningful topic that better represents the given document, we proposed a universal method which can be used in the data preprocessing stage. The method consists of three steps. First, it generates the word/word-pair from every single document. Second, it applies a two way parallel TF-IDF algorithm to word/word-pair for semantic filtering. Third, it uses the k-means algorithm to merge the word pairs that have the similar semantic meaning.

 Experiments are carried out on the Open Movie Database (OMDb), Reuters Dataset and 20NewsGroup Dataset and use the mean Average Precision score as the evaluation metric. Comparing our results with other state-of-the-art topic models, such as Latent Dirichlet allocation and traditional Restricted Boltzmann Machines. Our proposed data preprocessing can improve the generated topic accuracy by up to 12.99\%. How the number of clusters and the number of word pairs should be adjusted for different type of text document is also discussed.
 After millennium, most collective information are digitized to form an immense database distributed across the Internet. Among all, text-based knowledge is dominant because of its vast availability and numerous forms of existence. For example, news, articles, or even Twitter posts are various kinds of text documents. For the human, it is difficult to locate one's searching target in the sea of countless texts without a well-defined computational model to organize the information. On the other hand, in this big data era, the e-commerce industry takes huge advantages of machine learning techniques to discover customers' preference. For example, notifying a customer of the release of "Star Wars: The Last Jedi" if he/she has ever purchased the tickets for "Star Trek Beyond"; recommending a reader "A Brief History of Time" from Stephen Hawking in case there is a "Relativity: The Special and General Theory" from Albert Einstein in the shopping cart on Amazon. The content based recommendation is achieved by analyzing the theme of the items extracted from its text description.Topic modeling is a collection of algorithms that aim to discover and annotate large archives of documents with thematic information BID0 . Usually, general topic modeling algorithms do not require any prior annotations or labeling of the document while the abstraction is the output of the algorithms. Topic modeling enables us to convert a collection of large documents into a set of topic vectors. Each entry in this concise representation is a probability of the latent topic distribution. By comparing the topic distributions, we can easily calculate the similarity between two different documents. BID25 Some topic modeling algorithms are highly frequently used in text-mining BID13 , preference recommendation BID27 and computer vision BID28 . BID0 Many of the traditional topic models focus on latent semantic analysis with unsupervised learning. Latent Semantic Indexing (LSI) BID11 applies Singular-Value Decomposition (SVD) BID6 to transform the term-document matrix to a lower dimension where semantically similar terms are merged. It can be used to report the semantic distance between two documents, however, it does not explicitly provide the topic information. The Probabilistic Latent Semantic Analysis (PLSA) BID9 model uses maximum likelihood estimation to extract latent topics and topic word distribution, while the Latent Dirichlet Allocation (LDA) BID1 model performs iterative sampling and characterization to search for the same information.The availability of many manually categorized online documents, such as Internet Movie Database (IMDb) movie review Inc. (1990), Wikipedia articles, makes the training and testing of topics models possible. All of the existing workds are based on the bag-of-words model, where a document is considered as a collection of words. The semantic information of words and interaction among objects are assumed to be unknown during the model construction. Such simple representation can be improved by recent research advances in natural language processing and word embedding. In this paper, we will explore the existing knowledge and build a topic model using explicit semantic analysis.The work studies the best data processing and feature extraction algorithms for topic modeling and information retrieval. We investigate how the available semantic knowledge, which can be obtained from language analysis or from existing dictionary such as WordNet, can assist in the topic modeling.Our main contributions are:• We redesign a new topic model which combines two types of text features to be the model input.• We apply the numerical statistic algorithm to determine the key elements for each document dynamically.• We apply a vector quantization method to merge and filter text unit based on the semantic meaning.• We significantly improve the accuracy of the prediction using our proposed model. The rest of the paper is structured as follows: In Section 2, we review the existing methods, from which we got the inspirations. This is followed in Section 3 by details about our topic models. Section 4 describes our experimental steps and evaluate the results. Finally , Section 5 concludes this work. In this paper, we proposed a few techniques to processes the dataset and optimized the original RBM model. During the dataset processing part, first, we used a semantic dependency parser to extract the word pairs from each sentence of the text document. Then, by applying a two way parallel TF-IDF processing, we filtered the data in word level and word pair level. Finally, Kmeans clustering algorithm helped us merge the similar word pairs and remove the noise from the feature dictionary. We replaced the original word only RBM model by introducing word pairs. At the end, we showed that proper selection of K value and word pair generation techniques can significantly improve the topic prediction accuracy and the document retrieval performance. With our improvement, experimental results have verified that, compared to original word only RBM model, our proposed word/word pair combined model can improve the mAP score up to 10.48% in OMDb dataset, up to 1.11% in Reuters dataset and up to 12.99% in the 20NewsGroup dataset.
We proposed a universal method which can be used in the data preprocessing stage to generate the more meaningful topic that better represents the given document
{"src": "scitldr-AIC_train", "paper_id": "Byni8NLHf"}