Research on Quality Analysis Model of Additional Comments Based on Text Analysis

Based on the idea of text analysis, this study analyzes the relevance of text topics, the emotional consistency of fist comments and additional comments, and the information contained in the text, then get out of the online quality assessment model. The calculation results can reflect the quality of additional comments, helping consumers and businesses to identify important information and make correct judgments.


Introduction
In recent years, e-commerce has been developing rapidly, and more and more consumers would like to choose online shopping. The 34th "Statistical Report on the Development of China's Internet Network" showed that users' reviews are the main reference factor for online shopping users to make purchasing decisions. As an important form of online word-of-mouth, the comment system has become a main reference source for consumers' online shopping and it affects consumers' purchasing decisions. However, because of the cashback reward to favorable comments provided by the merchants, the trust of online comments is greatly reduced. Therefore, Taobao stipulates that: Within 180 days (inclusive) from the date of successful transaction, the buyer can add comments after making the Taobao credit evaluation, and the content of additional comments can not be modified. So when consumers browse comments and look for commodity information, they will pay special attention to the contents of additional comments. But in order to obtain more profits, the merchants come up with ways to give money to praise and even hire online water army to give praise. This has undoubtedly caused great trouble to consumers. Therefore, how to "get the fake and take the truth, get the rough and take the fine" and identify high-quality additional comments has become the subject of this research.

Related researches
For the question of the quality of comments, the researches of domestic and foreign scholars mainly focus on the initial comments and their main topics are the following two aspects.

Identify factors that affect the quality of online reviews
Kim S et al. [1] use SVM regression method to predict the quality of comments from five text features: structure, morphology, syntax, semantics and metadata. The experimental results show that comment length, unary grammar and product rating are the key factors to judge the usefulness of comment. Hyunmi Baek [2] believes that the inconsistency of comments, the ranking of reviewers, the number of comments, etc. have a certain impact on the credibility of comments. The research by Huang Tingting  [3] shows that whether the content of the review is closely related to the function of the product is an important indicator for judging the quality of the comment.

Study and analyze the model of online comment quality
Tang Xiaobo et al. [4] propose a topic-oriented high-quality comment mining model, which combines existing useful votes with other weighting indicators to automatically extract high-quality comments under each topic. Li Zhiyu [5] starts with the study of influencing factors of online comment effectiveness and establishes a commentary utility index system. Liu Jingjing et al. [6] note the problem of evaluation bias caused by the cumulative vote, so they use the data labeled manually and evaluate the quality of comments from three aspects: information content, subjectivity and readability.
Based on the carding of above theories and models, it provides an idea for constructing an additional comments quality evaluation index system, which is helpful to the construction of this model.

Theory and method
3.1. LDA model theory LDA (Latent Dirichlet Allocation) is a document theme generation model, which can be used to identify hidden subject information in large document sets or corpora. LDA gains the relationship between vocabularies, documents and topics in the corpus by generating a probability model. The LDA generation process of a document is as follows [7]: • Get the total number of all words in the document N~Poission (β); • Get the distribution of the document on the topic ~Dir(α); • Traverse each word from 1 to n. Fist, get the probability distribution of each topic ~Multinomial (θ m ). Then, get the probability distribution of topics and words ~Multinomial( Φ z ).
Among them, α and β are priori parameters, θ and Φ are estimated parameters. The experiment uses Gibbs sampling to estimate θ and Φ, and predicts the subject probability of the current word while keeping the subject distribution of other words unchanged, and indirectly obtains their values by marginalizing θ and Φ [7]: In the formula, C VK and C MK represent quantity matrixs of dimension V×K and M×K respectively, V is the number of words, K is the number of topics and M is the number of documents. C w −n j VK is the frequency that the word w assigned to the subject j; C d −n j MK is the number of words assigned to the subject j in the document d.
Assign random themes to the words in the document to form the initial Markov chain, and then assign them the subject again according to formula (1) in order to obtain the state of the next Markov chain, and iterate continuously until the Markov chain becomes stable. Finally, use the Gibbs sampling algorithm to estimate the θ value and Φ value for each word [7]: In this way, the probability distribution of each topic and the probability distribution of the topic and the word can be obtained, thereby we can obtain a "topic-word" probability distribution matrix and a "document-topic" probability distribution matrix.

Grey correlation analysis
In 1981, Professor Deng Julong, a Chinese cybernetic expert, first proposed the concept of the gray system. Later, he published many papers and monographs on the gray system and established the gray 2019 2nd International Symposium on Big Data and Applied Statistics Journal of Physics: Conference Series 1437 (2020) 012084 IOP Publishing doi:10.1088/1742-6596/1437/1/012084 3 system theory. The grey system theory puts forward the concept of gray correlation analysis for each subsystem, and intends to seek the numerical relationship between subsystems (or factors) in the system with certain methods. Therefore, the grey correlation analysis provides a quantitative measure for the development and change of a system, which is very suitable for dynamic process analysis. The grey correlation method is also an effective way to determine weights. It aims to determine the weight value by making a reasonable processing of the existing actual data in order to achieve a scientific description of the system development.

Design of the indicator system
For the online comments given by online consumers, this study attempts to establish a reasonable text quality evaluation system oriented to comment content. We can use it to evaluate the quality of users' reviews, identify high-quality additional comments quickly and improve the utility value of online comments.

Data collection
Taobao is currently the largest C2C e-commerce website in China, which has rich online comment resources. This study uses web crawler technology to get a large number of initial comments and additional comments from the website and store them in the corresponding documents.

Data preprocessing
The data obtained from the website is mixed. To ensure data quality and reduce the time wasted on invalid data, the original data needs to be preprocessed. Specific operations include the followings: • Delete the duplicate data. Delete the data whose initial comment is "This user did not fill out the comment" and whose comment content only contains special characters, numbers or pictures.
• Use the NLPIR-ICTCLAS Chinese word segmentation system to the text word segmentation and part of speech tagging.
• Delete the stop words in the comments.

•
Save the processed data

Relevance of comment topic
A study by Titov [8] shows that the higher the relevance of a comment to a topic, the greater its contribution to the topic and the higher the quality of the comment. Therefore, the first indicator in the model constructed in this study is the relevance of the text content to the topic. Referring to the research of Tang Xiaobo [9], we can confirm the idea of judging the relevance of commentary topic: Each comment in the comment text set corresponds to a document in the corpus. The topic distribution of each document can be represented as a vector, and the similarity between documents can be obtained by calculating the distance between documents [10]. Then in the "Comment-Subject" matrix of the corresponding LDA model, the topic distribution of each comment can be expressed as a vector =( 1 ( ) , 2 ( ) , …, ( ) ), where represents the subject distribution of the ith comment, ( ) represents the probability of the ith comment on topic k, and k is the number of topics . If the vector space of the two comments is closer, the similarity between the two comments is higher. Then, take each topic's comment set as a document, and combine the content in the next comment of the theme as a sentence. So we can calculate the similarity between each sentence in the document and other sentences by using the cosine angle, therefore to obtain the similarity of the comment set under a certain theme, which is recorded as Relativity.

Emotional consistence of comments
Zhou Hong et al. [11] point out that the emotional consistency of the initial comments and the additional comments has a great impact on consumers' information adoption. Therefore, the second indicator in the model constructed in this study is the emotional consistency of the initial and additional comments. This study uses polarity scores to measure the emotions of comments. It is stipulated that the score is positive for positive emotions and negative for negative emotions, and the emotional polarity scores are calculated with reference to Wang Qianqian [12]: • Extract the product attribute words and calculate their weights. In this study, the ICTCLAS word segmentation system is used for word segmentation and part-of-speech tagging, and then the tagged phrases are classified into parts of speech to extract product attribute words. Then the weight of an attribute word is determined according to the frequency of its occurrence in the whole comment corpus.
• Calculate the emotional score of the product attribute words. This study calculates the emotional scores of product attribute words with the method proposed by Zhang Jing and Jin Hao [13].
• Calculate the score of comment text. Suppose there are m comments, in which the i (0 < i < m) comments have y (0 < y <= n) emotional words. And the words' weights are: 1 , 2 , 3 ,…, , emotional scores are 1 , 2 , 3 ,…, , then the polarity score E of the ith comment is E= 1 * 1 + 2 * 2 + 3 * 3 +…+ * . It also stipulates that E is a good rating between 1 and 0.667, a middle rating between 0.666 and 0.334, and a negative rating between 0.333 and 0 • Calculate the consistency score. Wang Changzheng [14] believes that in the comments with additional comments, inconsistent comments can give readers a higher sense of usefulness than consistent comments. Therefore, this study stipulates that if the initial comment contradicts the additional comment sentiment tendency, the consistency score Consistence=1, otherwise Consistence=0.5.

Information amount of comments
It is generally believed that the more information a comment contains, the higher the quality of the comment. Therefore, the third indicator in the model constructed in this study is the amount of information contained in the additional comments. it is generally agreed that the amount of information contained in a comment is proportional to the number of comments. So, this study stipulates that the comments which have less than 10 Chinese characters, information amount Information=0.1, 20~30 Chinese characters, Information=0.2, and so on, 90~100 Chinese characters, Information=0.9, 100 Chinese characters or more, Information=1 4.6. The construction of model Based on the above indicators, this study constructed a multiple linear regression model: Q= 1 *Relativity+ 2 *Consistence+ 3 *Information (4) The weight of each indicator w i =(w 1 ，w 2， w 3 )( 0 < w i < 1 且 Σw i = 1)is determined by the questionnaire method and the gray correlation analysis method. Referring to the research of Zhou Linlin et al. [15], the specific steps are as follows: • Get the weight of each indicator by the questionnaire survey method X=(x ij ) nхm ,i=1, 2, …, n,j=1,2,…,m (5) • Determine the reference sequence 0 , then select the largest weight value from it and assign the value to each indicator as the reference weight value.

Data collection and processing
This study selects online comments of AL MI speaker sold by Xiaomi flagship store as the research object, and uses the reptile software to obtain 1880 data from Taobao. After preliminary screening, 900 valid data are left and divided into 600 training set and 300 test set. Use NLPIR-ICTCLAS Chinese Word Segmentation System to segment and part-of-speech tagging all commentary texts, the specific effect is shown in Table 1. Filter the stop words of texts, and the specific effect is shown in Table 2.

The calculation of review topic relevance
Use Java language to generate LDA probability model for comment text after word segmentation. Set the number of topics to 6, and the number of loop iterations to 100. The "topic-word" matrix and the "comment-theme" matrix are obtained through LDA clustering, as shown in Table 3 and Table 4 respectively.  It is stipulated that in the "comment-topic" matrix, the maximum number of similarities between each comment and the topic should be taken as the theme relevance of the comment.

Calculation of indicator weights
In this study, 500 electronic questionnaires are distributed to people of all ages, and they score the importance of additional comments' three indicators: relevance, consistency and information amount. Finally, 385 valid questionnaires are collected. The questionnaires are divided into four categories: under 20 years old, 20-30 years old, 30-40 years old, and over 40 years old. The data of the collected questionnaires are compiled and calculated, and the weights of the indicators are shown in Table 5. It is stipulated that in the "comment-topic" matrix, the maximum number of similarities between each comment and the topic should be taken as the theme relevance of the comment. According to the steps described in Section 4.6, introduce the gray correlation method, and then the weights of the three indicators are finally determined to be (0.38, 0.31, 0.31). So the model of this study is: Q=0.38*Relativity+0.31*Consistence+0.31*Information (9)

Analysis of Model experiment results
After the experiment, the results calculated by the model are obtained, and the Q value is defined as a low-quality comment in the interval [0, 0.5), and a high-quality comment in the interval [0.5, 1]. Applying the model and conclusion to the test set, we can get the accuracy rate is P=80.73%, the recall rate is R=64.90%, and the F value is F=71.95%. Therefore, it concludes that this model is a good evaluation model for additional comment texts.

Conclusion
With the rapid development of e-commerce, consumers and businesses urgently need a method to help them find useful information from a large number of disorderly online comments, so as to make correct judgments and choices in a short time. Based on the concept of text analysis, this paper proposes a quality evaluation model of online review, and chooses the additional comment of the MI Bluetooth speaker in Taobao as the test object. The usefulness of the model is verified by experiments. This model can help consumers judge the quality of online review and provide services for consumers to make purchasing decisions. At the same time, the model can also help merchants analyze and identify reliable review information to accurately grasp the user's consumer demand. In view of the shortcomings of the model, such as too few commodities as the experimental object, I will continue to improve in the future study.