Neural News Recommendation with Negative Feedback

News recommendation is important for online news services. Precise user interest modeling is critical for personalized news recommendation. Existing news recommendation methods usually rely on the implicit feedback of users like news clicks to model user interest. However, news click may not necessarily reflect user interests because users may click a news due to the attraction of its title but feel disappointed at its content. The dwell time of news reading is an important clue for user interest modeling, since short reading dwell time usually indicates low and even negative interest. Thus, incorporating the negative feedback inferred from the dwell time of news reading can improve the quality of user modeling. In this paper, we propose a neural news recommendation approach which can incorporate the implicit negative user feedback. We propose to distinguish positive and negative news clicks according to their reading dwell time, and respectively learn user representations from positive and negative news clicks via a combination of Transformer and additive attention network. In addition, we propose to compute a positive click score and a negative click score based on the relevance between candidate news representations and the user representations learned from the positive and negative news clicks. The final click score is a combination of positive and negative click scores. Besides, we propose an interactive news modeling method to consider the relatedness between title and body in news modeling. Extensive experiments on real-world dataset validate that our approach can achieve more accurate user interest modeling for news recommendation.


Introduction
Online news services such as Google News 1 and Microsoft News 2 can collect news from various sources and display them to users in a unified view [6,32]. However, a large number of news articles are generated every day and it is overwhelming for users to find their interested news [19]. Thus, personalized news recommendation is critical for these news services to target user interests and alleviate information overload [29].
Accurate user interest modeling is a core problem in news recommendation. Existing news recommendation methods usually learn representations of users from their historical news click behaviors [19,26,28,31]. For example, Okura et al. [19] proposed to learn user representations from the clicked news via a gated recurrent unit (GRU) network. Wang et al. [26] proposed to learn user representations from clicked news with a candidate-aware attention network that evaluates the relevance between clicked news and candidate news. In fact, users usually click news due to their interests in the news titles. However, news titles are usually short and the information they condensed is very limited and even misleading. Users may be disappointed after reading the content of their clicked news. Thus, news click behaviors do not necessarily indicate user interest, and modeling users solely based on news clicks may not be accurate enough.
Besides news clicks, users also provide other implicit feedback like the dwell time of news reading, which is an important indication of user interest. As shown in Fig. 1, the user clicks two news articles and respectively reads them for 3 and 220 seconds before closing the news webpages. We can infer that this user may not be interested in the first news because she reads the body of this news for only 3 seconds before her leaving. Thus, a short dwell time of news reading can be regarded as a kind of implicit negative user feedback, which can help model user interest more accurately. In addition, since users usually decide which news to click according to news titles and read news bodies for detailed information, incorporating both news title and body is useful for news and user interest modeling. Moreover, the title and body of the same news usually have some inherent relatedness in describing news content. For example, in Fig. 1 the title of the second news indicates that this news is about NFL games, and the body introduces the details of these events. Capturing the relations between news title and body may help better understand news content and facilitate subsequent user interest modeling.
In this paper, we propose a neural news recommendation approach with negative feedback (NRNF). We propose to distinguish positive and negative news click behaviors according to the reading dwell time, and learn separate user representations from the positive and negative news clicks with a Transformer to capture behavior relations and an additive attention network to select important news for user modeling. In addition, we propose to respectively compute a positive click score and a negative click score based on the relevance between candidate news representation and the user representations learned from positive and negative news clicks. Besides, we propose an interactive news modeling method to incorporate both news title and body by using an interactive attention network to model their relatedness. Massive experiments conducted on real-world dataset demonstrate that our approach can effectively improve the performance of user modeling by incorporating implicit negative user feedback.

Related Work
News recommendation is an important task in the data mining field, and has gained increasing attention in recent years [34]. Learning accurate representations of users is critical for news recommendation. Many existing news recommendation methods rely on manual feature engineering to build user representations [18,3,23,13,7,2,21,4,36]. For example, Li et al. [16] proposed to represent users using their demographics, geographic information and behavioral categories which summarize the consumption history on Yahoo!. Garcin et al. [8] proposed to model users by aggregating the LDA features of all clicked news into a user vector by averaging. Lian et al. [17] proposed to build user representations from various handcrafted features like user ID, demographics, locations and the feature collection of clicked news. However, these methods rely on manual feature engineering, which needs massive domain knowledge to craft. In addition, handcrafted features are usually not optimal for representing user interest.
In recent years, several news recommendation methods based on deep learning techniques are proposed [19,14,26,29,1,27,30,31,9,11]. For example, Okura et al. [19] proposed to learn representations of users from the representations of their browsed news using a GRU network. Wang et al. [26] proposed to learn user representations from clicked news based on their relevance to the candidate news. Zhu et al. [35] proposed to generate user representations using a combination of a CNN network and an attention-based LSTM network. Wu et al. [31] proposed to use multi-head self-attention network to learn user representations by capturing the interactions between clicked news. These methods only rely on news click behaviors to learn user representations, which may be insufficient because news clicks may not exactly reflect user preferences. Different from these methods, our approach can learn representations of users by incorporating the implicit negative feedback of users inferred from their reading dwell times, which can provide useful clues to calibrate the user interest inferred from news click behaviors.

Our Approach
In this section, we introduce the details of our neural news recommendation approach with negative feedback (NRNF). The architecture of NRNF is illustrated in Fig. 2. We will introduce each core technique in our method, including user modeling, click prediction, news modeling, and model training, in the following sections.

User Modeling with Negative Feedback
In this section, we introduce how to model user interests from their clicked news by considering negative feedback. Usually, the interests of users on the news they read carefully are very different from those they closed quickly after having a glance at the body. For example, as shown in Fig. 1, we can infer that the user may be indeed interested in the second news since she reads this news for about 220 seconds. However, she is probably not interested in or unsatisfied with the first news since she reads this news only for 3 seconds after click. Thus, incorporating short reading dwell time as the implicit negative feedback of users has the potential to enhance the modeling of user interest.
To characterize the differences of news in representing the real user interests, we use their reading dwell times t to distinguish positive news clicks from the negative ones. We regard the clicked news with t < T as negative news clicks and those with t ≥ T as positive news clicks, where T is a time threshold. Since simply aggregating all news together will ignore the differences of positive and negative news, in the user encoder module we propose to use a multiview learning framework to incorporate positive and negative news clicks as different views of users. Their details are introduced as follows.
The positive news click view is used to learn a positive user embedding from positive clicked news. We denote the P positive clicked news of a user as [D 1 , D 2 , ..., D P ]. We first use a news encoder to obtain their representations, which are denoted as R p = [r 1 , r 2 , ..., r P ]. Then, we use a positive feedback encoder to learn a unified user embedding from these news representations. It uses a similar architecture with the user encoder used by Wu et al. [31]. Since different clicked news usually have some interactions with each other, we use a news-level Transformer [25] to capture the interactions among them 3 , and the output sequence is computed as: where the three parameters in T ransf ormer(·) respectively denote the input query, key and value. In addition, since different clicked news have different importance in modeling user interests, we use an additive attention [33] network to select important news to learn more informative user representations. The final positive user embedding u p learned from this view is computed as: where q p is a query parameter vector, D p is regarded as the key and value. It summarizes user interests inferred from the positive user feedback. The negative news click view is used to learn representations of users from their negative clicked news. Similar to positive news, we use a combination of Transformer and additive attention network to learn the negative user embedding from the representations of the negative clicked news. We denote the negative user embedding learned from this view as u n , which condenses the information of negative user interest inferred from the negative clicked news.

Click Prediction
In this section, we introduce the details of click prediction in our approach. Usually, if a candidate news is similar to the news that a user reads carefully, it may have a high probability to be clicked by this user. However, if it is very similar to the news that are previously closed by a user very quickly, it will probably not be clicked by this user. Motivated by the observations above, we propose to calculate a positive click scoreŷ p and a negative click scorê y n respectively based on the relevance of candidate news to the positive and negative user interest. We denote the embedding of a candidate news D c as r c . Following [19],ŷ p andŷ n are respectively computed by the inner product of the candidate news embedding and the user embeddings learned from the positive and negative views, which are formulated as: The final click probability scoreŷ is a weighted summation of the positive scoreŷ p and the negative scoreŷ n , which is formulated as: where w p and w n are learnable parameters. A positive parameter means that the relevance between candidate news and the corresponding user embedding has a positive correlation with the final click score, and this correlation is negative if the parameter is negative.

News Modeling
In this section, we introduce the architectures of the news encoder in our approach for news modeling. It is used to learn representations of news from their titles and bodies. Since news titles and bodies usually have very different characteristics, simply concatenating title and body together may not be optimal. Thus, in this module we use a multi-view learning framework to incorporate the information of title and body as different views of news. In addition, since there is usually inherent relatedness between news title and body, we propose to use an interactive attention network to capture their relations and form the unified news embedding. The details of each module in the news encoder are introduced as follows.
The first module in the news encoder is a title transformer, which is used to capture the contexts of words in a news title. We first use a word and position embedding layer to convert the sequence of words in a news title into a sequence of low-dimensional word embeddings with their position information. Denote a news title with T words as [w t 1 , w t 2 , ..., w t T ]. It is converted into an embedding sequence E t = [e t 1 , e t 2 , ..., e t T ]. Then, we use a Transformer to learn the contextual representation of each word, which is computed as follows: The second module in the news encoder is a body transformer, which aims to capture the contexts of words in a news body. Similar to news title, we also use a word and position embedding layer to obtain the embeddings of the M words in a news body, which are denoted as It's not with salt Interactive Attention News Embedding Fig. 3 The architecture of the news encoder.
we use a Transformer to capture the contexts of words, and its output H b is computed as The third module is an interactive attention network, which is used to capture the relations between news title and body when forming the unified news embedding. We first use two additive attention network to respectively attend to the important words in news title and body to form the title and body representations(denoted as r t and r b ), which are formulated as where q t and q b are learnable query vectors. Then, we use the title representation as the query of the body attention network to select words in news body according to their relevance to the content of news title, and the output title-aware body representation c b is computed as follows: We learn a body-aware title representation c t in a similar way by using the body representation as the title attention query, which is formulated as: We finally aggregate all title and body representations into a unified news representation r, i.e., r = r t + r b + c t + c b .

Model Training
Following [12,28], we use negative sampling techniques to construct labeled data from the raw news impression logs for model training. For each news clicked by a user, we randomly sample Q news that are displayed in the same impression but not clicked by this user. We denote the click probability scores of the clicked news sample asŷ + and the Q unclicked news samples as . we re-formulate the news click prediction problem as a pseudo Q+1-way classification task to jointly predict these scores. We normalize these scores via the softmax function to compute the posterior click probability of the clicked news, which is formulated as follows: .
The final loss function we used is the negative log-likelihood of all clicked news samples, which is formulated as: where S is the set of clicked news samples for training.

Datasets and Experimental Settings
Our experiments were conducted on a real-world news recommendation dataset collected from MSN News 4 logs from Oct. 31, 2018 to Jan. 29, 2019. The detailed statistics are shown in Table 1. We sorted all sessions by time, and we used the first 80% of sessions for training, the next 10% for validation and the rest 10% for test. In our experiments, the word embeddings were 300-dimensional and initialized by Glove [20] The Transformers had 8 attention heads, whose output dimension was 32. The maximum lengths of title and body were respectively set to 30 and 200, respectively. The numbers of positive and negative clicked news were respectively limited to 50 and 20. Following [12], the negative sampling ratio Q was set to 4. The threshold T was set to 10 seconds. Adam [15] was used as the optimizer, and the batch size was 30. To mitigate overfitting, we applied dropout [24] techniques to the word embeddings and Transformer networks, and the dropout ratio was set to 0.2. These hyperparameters were tuned on the validation set. We independently repeated each experiment 10 times and reported the average results in terms of AUC, MRR, nDCG@5 and nDCG@10.

Performance Evaluation
We evaluate the performance of our approach by comparing it with many baseline methods, including: -LibFM [22], a factorization machine based method for recommendation.
We extracted the TF-IDF features from the titles and bodies of news clicked by users to build their representations. -DSSM [12], deep structured semantic model. We regarded the concatenation of the news clicked by a user as the query and candidate news as documents, and we used the same feature with LibFM for news and user representation.  -NRNF, our news recommendation approach with negative feedback.
For fair comparison, in GRU, DKN, DAN and NRMS we used the concatenation of news title and news body to learn news representations. The results of these methods are summarized in Table 2.
From Table 2, we have several observations. First, compared with baseline methods that use handcrafted features to represent news and users (e.g., LibFM, DSSM and DeepFM ), the models using neural networks for learning news and user representations (e.g., EBNR, NAML and NRNF ) achieve better recommendation performance. This is probably because neural networks can learn more informative news and user representations than hand-crafted features. Second, compared with other neural news recommender like DAN, NAML and NRMS, NRNF and its variant NRNF-basic achieve better performance. This is because the latter methods can capture the relatedness between news title and body to enhance news modeling, while other methods cannot. Third, NRNF outperforms other baseline methods, and further hypothesis tests show that the improvement is significant. This is probably because incorporating the implicit negative user feedback can help learn more accurate user interest representations.

Effectiveness of Different News Information
Next, we explore the influence of incorporating different news information for news modeling. We compare the performance of our approach with its variants with different combinations of news information, and the results are shown in Fig. 4. From Fig. 4, we have several observations. First, both news title and body are useful for news recommendation. This is because users usually decide which news to click based on its title, and read news body for detailed information. Thus, both news title and body can provide important information for news recommendation. Second, news body is more useful than news title in representing news. This is intuitive because news body usually contains the details of news, which can provide richer information than news title. Besides, combining both title and body of news can further improve the performance of our approach. These results validate the effectiveness of the multi-view learning framework in our approach.

Effectiveness of Attention Network
Then, we explore the influence of different attention networks on the performance of our approach. The results are shown in Fig. 5. From Fig. 5, we find that word-level attention networks are very useful for our approach. This is because different words in news titles and bodies usually have different importance in representing news. Thus, selecting important words can help learn more informative news representations. In addition, news-level attention network is also important. This is because different news usually have different informativeness in representing the preferences of users, and selecting informative news can benefit user representation learning. Besides, the interactive attention network can also boost the performance. This may be because it can help capture the relations between news title and body to enhance news modeling. Moreover, combining all these kinds of attention networks can further improve the performance of our approach. These results verify the effectiveness of each kind of attention mechanism in our approach.

Influence of Hyperparameters
In this section, we study the influence of an important hyperparameter on our approach, i.e., the dwell time threshold T . We compare the performance of our approach w.r.t different values of T . The results are illustrated in Fig. 6. We find the performance of our approach improves when the threshold T increases. This is probably because when T is too small, many negative news clicks cannot be distinguished from the positive ones, and the modeling of user interest is not accurate enough. In addition, the performance of our approach  declines when T goes too large. This is also intuitive because when T is too large, many positive news clicks will be mistakenly regarded as negative ones and the errors will be encoded into the negative user embedding, which leads to the sub-optimal performance. Therefore, a moderate selection of T , e.g., 10 seconds may be more appropriate for our approach, which is also consistent with our news browsing experiences.

Parameter Analysis
In this section, we conducted several studies on the results of other two key parameters learned by our model, i.e., w p and w n . The results in 10 independent experiments are illustrated in Fig. 7. According to these results, we find w p is consistently positive. It is intuitive because the news read by users carefully can usually represent the preferences of users. Thus, the similarities between the candidate news and the positive clicked news should have a positive impact on the click probability. In addition, we find w n is consistently negative. This may be because if a user reads a news article very quickly, we can infer that he/she is probably not interested in this news. Thus, the click probability should have a negative correlation with the similarities between candidate news and negative clicked news.

Qualitative Analysis of Negative News
In this section, we will conduct several qualitative analysis on the characteristics of the negative clicked news. First, we want to compare the topic distributions of positive and negative news, and the results are shown in Fig. 8. According to Fig. 8, we find the ratios of negative news in different topic categories have some differences. For example, it is interesting that the ratio of negative news is the highest in the "video" category. This may be because many users are attracted by the titles of the news in this category, but they are not interested in the content of these news. Thus, incorporating the informa- tion of negative feedback may also implicitly encode topical information into user representations, which is useful for more accurate news recommendation.
In addition, we aim to discover some common patterns of negative news, e.g., the contents and writing styles of their titles. We calculate the ratio of negative feedback (NF) for each news 5 , and the titles of the top 10 news with the highest NF ratios are listed in Table 3. From Table 3, we find many news with top NF ratios are clickbaits, and they have very similar writing styles, i.e., describing something in a sensationalized way but no details are provided. Thus, many users may be attracted by the news titles and click these news. However, after taking a glance at the news body, many users may be disappointed and close these news quickly. Thus, it is harmful for news platforms to improve user experiences if clickbaits are frequently recommended to users. Fortunately, our approach may have the potential to avoid recommending too many clickbaits to user by incorporating the negative feedback of users, which can improve the reading experiences of users.

Case Study
In this section, we conducted several case studies to show the effectiveness of our approach. First, we want to visually explore the effectiveness of incorporating the negative feedback of users. The clicked news and candidate news of a randomly selected user are listed in Table 4. We calculate the click probability score respectively using our approach and its variant without negative feedback. According to Table 4, the second candidate news is assigned the highest click score when negative feedback is not considered. This is probably because it is very similar to the second and third clicked news, since all of them are about Oscars. However, we can infer that this user is probably not very interested in this topic since she closes the second and third clicked news very quickly. Thus, it may be ineffective to recommend Oscars related news to this user. Fortunately, our approach can assign the second candidate news a low click score. This is because our approach can take the implicit negative user feedback into consideration, which is beneficial for modeling user preferences more accurately.

Conclusion
In this paper, we propose a neural news recommendation approach that can consider the implicit negative feedback of users. We propose to distinguish positive and negative news clicks according to users' reading dwell time, and we propose to respectively learn two different embeddings of a user from her positive and negative news clicks. In addition, we propose to compute a unified news click score based on a combination of the relevance scores between candidate news and these two user embeddings. Besides, we propose an interactive news modeling method that can learn unified news representations from title and body with full consideration of their relatedness. Extensive experiments on real-world dataset show that incorporating negative user feedback can effectively improve the performance of user modeling for news recommendation.