A Survey of Sentiment Analysis Based on Transfer Learning <s> 4) SUMMARY <s> Sentiment classification aims to automatically predict sentiment polarity (e.g., positive or negative) of users publishing sentiment data (e.g., reviews, blogs). Although traditional classification algorithms can be used to train sentiment classifiers from manually labeled text data, the labeling work can be time-consuming and expensive. Meanwhile, users often use some different words when they express sentiment in different domains. If we directly apply a classifier trained in one domain to other domains, the performance will be very low due to the differences between these domains. In this work, we develop a general solution to sentiment classification when we do not have any labels in a target domain but have some labeled data in a different domain, regarded as source domain. In this cross-domain sentiment classification setting, to bridge the gap between the domains, we propose a spectral feature alignment (SFA) algorithm to align domain-specific words from different domains into unified clusters, with the help of domain-independent words as a bridge. In this way, the clusters can be used to reduce the gap between domain-specific words of the two domains, which can be used to train sentiment classifiers in the target domain accurately. Compared to previous approaches, SFA can discover a robust representation for cross-domain data by fully exploiting the relationship between the domain-specific and domain-independent words via simultaneously co-clustering them in a common latent space. We perform extensive experiments on two real world datasets, and demonstrate that SFA significantly outperforms previous approaches to cross-domain sentiment classification. <s> BIB001 </s> A Survey of Sentiment Analysis Based on Transfer Learning <s> 4) SUMMARY <s> Sentiment classification aims to automatically predict sentiment polarity (e.g., positive or negative) of user generated sentiment data (e.g., reviews, blogs). In real applications, these users generated sentiment data can span so many different domains that it is difficult to manually label training data for all of them. In this article, we develop a general solution to cross-domain sentiment classification when we do not have any labeled data in a target domain but have some labeled data in a source domain. To bridge the gap between domains, we propose a novel algorithm, called topical correspondence transfer (TCT). This is achieved by learning the domain-specific information from different domains into unified topics, with the help of shared topics across all domains. In this way, the topical correspondences behind the shared topics can be used as a bridge to reduce the gap between domains. We conduct experiments on a benchmark composed of reviews of 4 types of Amazon products. Experimental results show that our proposed TCT significantly outperforms the baseline method, and achieves an accuracy which is competitive with the state-of-the-art methods for cross-domain sentiment classification. HighlightsWe propose a novel algorithm, called topical correspondence transfer (TCT) for cross-domain sentiment classification.The proposed framework is achieved by learning the domain-specific information from different domains into unified topics, with the help of shared topics across all domains.An objective function is defined to simultaneously learn a topical representation and document sentiment prediction with the model. <s> BIB002 </s> A Survey of Sentiment Analysis Based on Transfer Learning <s> 4) SUMMARY <s> One-hot CNN (convolutional neural network) has been shown to be effective for text categorization (Johnson&Zhang, 2015). We view it as a special case of a general framework which jointly trains a linear model with a non-linear feature generator consisting of `text region embedding + pooling'. Under this framework, we explore a more sophisticated region embedding method using Long Short-Term Memory (LSTM). LSTM can embed text regions of variable (and possibly large) sizes, whereas the region size needs to be fixed in a CNN. We seek effective and efficient use of LSTM for this purpose in the supervised and semi-supervised settings. The best results were obtained by combining region embeddings in the form of LSTM and convolution layers trained on unlabeled data. The results indicate that on this task, embeddings of text regions, which can convey complex concepts, are more useful than embeddings of single words in isolation. We report performances exceeding the previous best results on four benchmark datasets. <s> BIB003 </s> A Survey of Sentiment Analysis Based on Transfer Learning <s> 4) SUMMARY <s> The dominant approach for many NLP tasks are recurrent neural networks, in particular LSTMs, and convolutional neural networks. However, these architectures are rather shallow in comparison to the deep convolutional networks which have pushed the state-of-the-art in computer vision. We present a new architecture (VDCNN) for text processing which operates directly at the character level and uses only small convolutions and pooling operations. We are able to show that the performance of this model increases with depth: using up to 29 convolutional layers, we report improvements over the state-of-the-art on several public text classification tasks. To the best of our knowledge, this is the first time that very deep convolutional nets have been applied to text processing. <s> BIB004 </s> A Survey of Sentiment Analysis Based on Transfer Learning <s> 4) SUMMARY <s> Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch. We propose Universal Language Model Fine-tuning (ULMFiT), an effective transfer learning method that can be applied to any task in NLP, and introduce techniques that are key for fine-tuning a language model. Our method significantly outperforms the state-of-the-art on six text classification tasks, reducing the error by 18-24% on the majority of datasets. Furthermore, with only 100 labeled examples, it matches the performance of training from scratch on 100x more data. We open-source our pretrained models and code. <s> BIB005
The above is an overview of sentiment analysis algorithms and applications based on transfer learning. People can have a more comprehensive understanding about the application scenes of the three types of methods. For the parametertransfer method, it can utilize the trained parameters and the rich semantic features of large corpus, and at the same time reduce the computational overhead of model training so that users can reduce the computational cost. However, the method has the disadvantage that the parameters are not easy to converge. For the instance-transfer method, its application scope is affected by the similarity between the two domains. The analysis result is poor when the two domains have large differences. However, this method is easy to apply in an unlabeled dataset. For the feature-representationtransfer method, the application scope is more extensive than the instance-transfer method. When the two domains have partial cross features, the transformation can be used to obtain a great analysis effect, but it is difficult to solve the optimization problem. According to the above analysis, transfer learning methods can solve some problems such as data labeling, computational overhead and personalized demand brought by big data. In order to better compare the differences between different algorithms, we will show the experimental performance between the different methods. As shown in Table 1 , it shows the main information of different algorithms, including four aspects: research content, basic model/core idea, transfer learning methods, datasets and accuracy. By comparing the basic models and experimental performance of different methods, users can be provided with more detailed sentiment analysis resources to compare different methods. We will make a detailed analysis of the methods in Table 1 . It can be seen in Table 1 that the models of BIB004 and BIB003 have achieved higher accuracy without using transfer learning. Although, LSTM and its derivative models are widely used in transfer learning as the basic model, BIB004 uses the deeper CNN in which max-pooling performs better than other pooling types and the result shows higher performance, which indicate that in transfer learning deeper CNN is also worth further exploration. The better effect of LSTM combined with one-hot CNN in BIB003 proves the advantages of the regional embedding method, and further proves that CNN has a good application in text processing. Most of the parameter-transfer methods are based on LSTM, and  and BIB005 get better results. In  , the parameters obtained by the model trained by the unlabeled data are used as the initialization parameters for the training of the next stage model. This pre-training approach enhances the stability of training about LSTM and effectively combines the advantages of both unsupervised and supervised learning. What is worth learning in BIB005 is the versatility of the model, which can be used for various NLP tasks and achieve great results. Its fine-tuning tips are also used for reference. One is discriminative fine-tuning: different layers VOLUME 7, 2019 use different learning rates. The other is slanted triangular learning rates: a new way to correct learning rates. The core of the instance-transfer method is weight reuse. The current method does not use the neural network model, mainly in the improvement and optimization of machine learning algorithms. It can be seen from Table 1 that the datasets selected by experiments of different algorithms have certain differences. The disadvantage of this is that they cannot be compared with other methods of the same type. On the other hand, this type of method is also mainly applied to text tasks in specific fields such as cross-language sentiment analysis, small-field text analysis, etc., which may be convenient to apply in practice. However, due to the limitations of its own application scenarios (higher similarity between source and target domains), most scholars have studied more on modeltransfer methods and feature-representation transfer methods. Most feature-representation transfer methods use Amazon Product Reviews as an experimental dataset, which makes it easier for users to compare different algorithms and visualize the pros and cons of them. The core of this type of method is the optimization of the feature space, and the neural network model is rarely used. There will be some deviations in the accuracy of the experimental results due to different experimental settings or different selection of source and target domains. In Table 1 , we show the optimal and worst results for the three algorithms in BIB001 - BIB002 . It can be seen that the accuracy of the three algorithms in the experiment is not much different, and the highest accuracy is BIB001 , but from the worst results, BIB002 is more stable. It is worth considering that the domain of experimental data is same in the best performance of the three methods. The source domain is all the Electronics (E) dataset, and the target domain is all the Kitchen (K) dataset, which means that the text data features have the same impact on different algorithms. In addition, the combination of different source and target domains can produce different results with large differences in accuracy, and also proves the impact of the similarity between source and target domains on transfer learning. From Table 1 , we can see that not all transfer learning methods are more accurate than normal methods. This is mainly because of two reasons. First, the difference between multi-task and single tasks. Due to the complexity of the text tasks, when the transfer learning method is applied to NLP, different target tasks need to adopt different fine-tuning methods, so many transfer learning methods are difficult to apply to multiple NLP tasks. Therefore, in terms of text sentiment analysis tasks, the performance of some transfer methods may be slightly worse than the single model for sentiment analysis tasks. Second, the limitations of the transfer learning itself and the lack of fine-tuning. In the process of transfer, due to the difference in data or tasks between the target domain and the source domain, it is necessary to pay attention to the negative effects that negative transfer may have and the catastrophic information forgetting or over-fitting when fine-tuning. Although the performance of transfer learning is not best, it can solve the problems encountered in current sentiment analysis such as label consumption, time consumption and computer performance problems. Therefore, the application of transfer learning is more extensive, and it can solve multi-angle text tasks, which is a hot research topic in the future.