Computer Science
Computer Science-2021.gz:143301
Computer Science
9814f03fda4621a003993b7b36e01bf12c7270c4
Attention on Global-Local Representation Spaces in Recommender Systems

In this study, we present a novel clustering-based collaborative filtering (CF) method for recommender systems. Clustering-based CF methods can effectively deal with data sparsity and scalability problems. However, most of them are applied to a single representation space, which might not characterize complex user-item interactions well. We argue that the user-item interactions should be observed from multiple views and characterized in an adaptive way. To address this issue, we leveraged the global and local properties to construct multiple representation spaces by learning various training datasets and loss functions. An attention network was built to generate a blended representation according to the relative importance of the representation spaces for each user-item pair, providing a flexible way to characterize diverse user-item interactions. Substantial experiments were evaluated on four popular benchmark datasets. The results show that the proposed method is superior to several CF methods where only one representation space is considered.


INTRODUCTION
R ECOMMENDER systems are getting popular in many web applications. Because customers have various opinions and preferences, satisfying their requirements becomes an important issue for content and product providers. There is a great need for personalized recommender systems that suggest appropriate content and products to a user in a wide variety of online services, such as entertainment [1], travel [2], and social network [3].
The general approaches used in recommender systems include collaborative filtering (CF) and content-based filtering. The CF approach builds a model from a user's past behavior (items that are purchased, viewed, selected, or rated), as well as similar behaviors by other users. The model is then used to recommend unseen items that will likely interest the user. The content-based filtering approach uses a series of discrete, previously-tagged characteristics of an item to recommend other items with similar properties. A modern recommender system often combines these two approaches into a hybrid system. In this study, we focus on the CF approach.
Matrix factorization is one of the most popular CF techniques [4] [5] [6]. The relation between users and items are represented as an interaction matrix, and the matrix is then factorized into a user latent matrix and an item latent matrix. The similarity between the user and item is usually measured by the inner product or Euclidean distance between their latent vectors. Missing values in the interaction matrix can thus be inferred by the predicted user-item similarity and be used to make recommendations to a user for noninteracted items. Another widely used CF technique is the neighborhood model [7] [8]. It selects the nearest neighbors for each user/item based on their similarities. The missing values of the user/item are predicted by combining its neighbors' weighted similarities.
As the system scales with rapid growth in the numbers of users and items, the interaction proportion among the whole set of users and items becomes very small. Then these CF techniques must deal with the data sparsity and scalability problems. One remedy is clustering-based recommendation [9] [10] [11] [12] [13]. This approach groups similar users or items into subsets by means of clustering. From the view of clusters, interactions between the user and item subsets are no more sparse. In addition, the clustering technique can be integrated with the index structure to overcome the scalability issue. In other words, instead of exhaustive search, cluster pruning can be used to keep a few candidates for evaluation with respect to a given query. The complexity is reduced to the sub-linear time of the number of items, making recommendation more efficient. Therefore, the clustering-based approach has been extensively developed recently to address sparsity and scalability problems.
Another key point is that most existing methods are applied to a single representation space. However, useritem interaction behaviors are often diverse and complex; it is beneficial to observe them from multiple views and to characterize them in an adaptive way. For example, in the clustering-based approach, we can extract not only local features from clusters but also global features from the whole dataset. These features may reflect significant properties of various user-item interactions. Moreover, they can be combined to derive a more discriminating representation.
Following the above discussion, we propose a novel method for clustering-based CF recommender systems that involves leveraging global and local views. Multiple representation spaces are learned from the training data of global and local user-item interactions with different loss functions. To fuse the information from different representation spaces, arXiv:2104.12050v2 [cs.IR] 16 Nov 2021 an attention network is adopted to formulate an attentive representation for each user-item pair by blending the relative weights among the representation spaces. Compared with existing attention-based methods that use an attention model at the item level (considering which item is more important), the proposed attention model is used at the representation level (considering which representation is more important).
We highlight the contributions of the proposed method as follows: • Multiple representation spaces are constructed based on different global/local properties and loss functions. Rather than using only one representation as in most existing methods, multiple representations can provide an abundance of features to describe the relation between users and items.
• An attention network is presented to capture the relative importance of multiple representation spaces. It can dynamically fuse multiple representations for each user-item pair, forming a flexible measure to characterize diverse user-item interactions.
• Substantial experiments compared with various configurations and state-of-the-arts are evaluated under four benchmark datasets. The results can support the effectiveness and superiority of the proposed method.
The remainder of this paper is organized as follows. Section 2 presents related work on recommender systems. Section 3 elaborates the proposed method, including the global/local representation networks and attention network. Section 4 demonstrates and discusses experimental results. Finally, conclusions are summarized in Section 5.

RELATED WORK
Among the mass of literature studies in recommender systems, we focus on the up-to-date CF-based methods with brief introduction. We also review the clustering-based approach and attention mechanism highly related to our work.

CF-based recommender systems
The core idea of the CF approach is to model the preference of users toward items according to their historical interactions. Several techniques are proposed to address this issue. In addition to the above-mentioned matrix factorization and neighborhood model, here we introduce other popular techniques, namely, ranking formulation and deep learning.
Rendle et al. [14] presented a generic optimization criterion by Bayesian personalized ranking, called BPR-Opt, for personalized ranking in the item recommendation task. BPR-Opt is the maximum posterior estimator derived from a Bayesian analysis of the problem. The main idea is to rank observed items higher than unobserved items. They also provided a generic learning algorithm for optimizing models with respect to BPR-Opt. Following the similar ranking formulation, Hsien et al. [6] proposed collaborative metric learning (CML), which encoded user preferences and user-user/item-item similarity in a joint metric space. CML achieves significant speedup for top-N recommendation tasks using off-the-shelf approximate nearest-neighbor search with negligible accuracy reduction.
More recent approaches tend to use the deep learning technique. He et al. [5] presented a general framework called neural network-based collaborative filtering (NeuMF) for recommender systems. They used nonlinear neural networks as the user-item interaction function. NeuMF expresses and generalizes matrix factorization and leverages a multi-layer perceptron to learn the user-item interaction function. The authors transformed the identity of a user/item to a binarized sparse vector with one-hot encoding, which only explored the user/item feature in a limited manner. Due to the complexity of the scoring function, the NeuMF retrieval process is generally hard to accelerate. Rendle et al. [15] revisited the NeuMF experiments, and concluded that a dot product might be a better default choice for combining embeddings than learned similarities using MLP or NeuMF. Xue et al. [16] proposed a neural network framework to model higher-order item relations for item-based collaborative filtering by using multiple nonlinear layers above pairwise interaction modeling. Under this framework, the authors also used an attention model to differentiate the importance of pairwise interactions. Deng et al. [17] proposed a general framework called deep collaborative filtering (DeepCF) to combine the representation learning and matching function learning. They also proposed a model called a collaborative filtering network (CFNet) based on a plain-vanilla MLP model under the DeepCF framework. CFNet has great flexibility in learning complex matching functions, with good efficiency in learning lowrank relations between users and items.
To handle the high-dimensional and sparse matrix efficiently, Luo et al. presented a series studies for latent factorbased CF methods [18] [19] [20] [21]. For example, they investigated the stochastic gradient descent algorithm and develop several extensions to improve the accuracy [18]. In addition, some non-negative factorization algorithms are proposed to overcome the slow convergence problem under non-negativity constraints [19] [20]. Wu et al. designed a deep structure to construct the hierarchical latent factor models that can achieve high computation and storage efficiency [21].

Clustering-based recommender systems
The clustering-based approach is well known for its scalability to large and sparse systems. The basic steps include calculating the similarity based on rating data, and then applying a clustering algorithm to group similar users and items into clusters. Some advanced methods are introduced in the following.
Ji et al. [22] proposed a reconstructive method for improving matrix approximation by introducing clustering and transfer learning techniques. The proposed method compresses low-rank approximation into a cluster-level rating-pattern referred to as a codebook and then constructs an improved approximation by expanding the codebook. Wu et al. [23] proposed a scalable co-clustering method called CCCF. The idea is users may have different preferences over different subsets of items, where these subsets of items may overlap. CCCF can explore subgroups of users who share interests over similar subsets of items through user-item co-clustering. Chen et al. [12] proposed a recommender system called N2VSCDNNR based on node2vec [24] and clustering technology. N2VSCDNNR recommends item clusters with high weights to the target user cluster. The weights are obtained from the interaction frequency between items and users. Then the items in the highweight clusters are recommended to the target user. Kang and McAuley [13] proposed a clustering-based framework called CIGAR, which learns a preference-preserving binary embedding model and a candidate item re-ranking model. Although using binary codes can significantly reduce the inference cost, due to their constrained capability, the performance is limited compared to models that use real-valued representations.

Recommender systems with attention mechanisms
The attention mechanisms are components of prediction systems that enable a system to focus sequentially on different subsets of the input [25]. They have been widely adopted in recommender systems recently.
Chen et al. [26] presented an attentive collaborative filtering model to address implicit feedback in multimedia recommendation. They introduced item-level and componentlevel attention models to assign attentive weights for inferring the underlying user preferences encoded in implicit user feedback. Zhu et al. [27] presented a tree-based deep recommendation model that predicts user preferences using a max-heap-like tree probability formulation with an attention-based deep network. The underlying deep model and tree structure are learned in turn. Tay et al. [28] proposed a metric learning approach called latent relational metric learning for recommendations. They adopt the idea of translating users to items by translation vectors while relying on a memory attention network to learn the relation vectors. He et al. [29] proposed a user-generated list recommendation model that leverages the hierarchical structure of items, lists, and users to capture the containment relationship between lists and items. They applied a self-attention network to refine the item and list representations by considering the consistency of neighboring items and lists. Hu et al. [30] introduced a model named BCFNet, which is an extended version of DeepCF [17]. The BCFNet consists of three sub-models, including representation learning, matching function learning, and balance module. In particular, an attention mechanism is integrated to improve the ability of the representation learning and matching function learning.
For group recommendation, the attention mechanisms can be employed to aggregate the properties of a set of user/item group. Cao et al. [31] proposed using NeuMF with attention models to characterize user and user group interest at the same time. They further integrated the modeling of user-item interactions into their method, making it possible to reinforce the two tasks of recommending items for users and user groups. Chen et al. [1] considered the task of recommending a set of items (bundle) to a user. They designed a factorized attention network to fuse the set of item representations into a bundle representation.
The attention mechanisms are also applied in sequential recommendation. For example, Zheng et al. [2] pro-posed a memory-augmented hierarchical attention network (MAHAN) for next point-of-interest (POI) recommendation. MAHAN incorporates two networks to tackle short-term check-in sequences and long-term memories for preference modeling. A co-attention network is used to characterize the interaction between these long-term and shortterm preferences. Wu et al. [32] presented a similar idea, where the long-term preference is formulated by attention weight evaluation. Hao et al. [33] explored user's short-term preference based on the local and global features of the annular-graph of the user behavior sequence by the selfattention mechanism. Fan et al. [34] introduced the lowrank decomposed self-attention to generate content-aware representations, where n items are aggregated into k latent interests (n > k) to perform a lightweight self-attention computation.

Summary
To sum up, the CF techniques that employ neural networks to model user-item interactions become a popular choice. However, these techniques mainly focus on a single representation space. We argue that it is insufficient to model the complicated user-item interactions. The clustering-based methods are proposed in several studies to address the sparsity and scalability problems. Few of them take advantage of global and local representations from the cluster structure. We point out that multiple representations are beneficial to provide multi-views for observing user-item interactions. The attention mechanisms can provide an effective way to fuse these representations into a blended one. We develop the representation-level attention, which is different from the item-level attention used by existing methods.

PROPOSED METHOD
In the proposed method, we build multiple representation networks and an attention network for the recommendation purpose. Fig. 1 illustrates an overview of the training process for these networks. Given a set of user-item interactions, we prepare two categories of training data, namely, global and local, according to whether clustering is involved or not. Then the representation networks are trained by using different training data categories and loss functions. Next, multiple global and local representations, which are generated from the representation networks, are employed to train the attention network. The attention network combines these representations to obtain the attentive representations of the users and items, which are used to measure the compatibility between each user-item pair. Details are elaborated in the following subsections.

Global representation
The goal of the CF approach with implicit feedback is to explore the unobserved relations between user u and item i and to answer the question: will the user interact with the item in the future? A user-item interaction consists of, for example, the user assigning a score to an item or checking information about an item. The interaction history is usually described by an interaction matrix, denoted as A = [a ui ], which contains the user ID u, the item ID i, and the interaction a ui that represents the preference of user u for item i. This study focuses on evidence that the user interacted with an item, regardless of the extent of the interaction. Therefore, the matrix element a ui is defined by: A triplet training set is sampled from A and expressed as: where i + and i − are the positive and negative items, respectively. In practice, it is impossible to enumerate all triplet combinations and many of them are trivial for training. We adopt a simple strategy in this study [6] [35]. Given the row vector of user u in interaction matrix A, for each positive item i + (where a ui + = 1), we randomly choose an negative The iteration process will repeat until the number of triplets is sufficient.
To derive the global representation, local information such as cluster assignment of the training data is ignored during the training phase. Let G ∈ R d be a global representation space of d-dimensions. To project user u and item i into G, the representation network is designed based on the Siamese network [36], as shown in Fig. 2. The embedding layer transforms a user or item ID into a dense vector of a fixed size. The embedding vector is then forwarded to hidden layers for nonlinear transformation. Note that we modify the typical Siamese network so that the variant can accept a triplet (u, i + , i − ) ∈ T as the input in the training process. In addition, we separate the user from the item without sharing the hidden layers. In other words, the network parameters can be very different between the user part and the item part. This is because users and items are different in nature, they should be processed through different paths in order to be projected to the common representation space. Through the respective embedding and hidden layers, inputs u, i + , and i − are transformed with L2 normalization into G as representation outputsû, i + , andî − , respectively. The training of the representation network is guided by a triplet loss function. Using different loss functions to generate variant representations can provide multiple views to characterize each user and item. We introduce two loss functions for this purpose, the dot product loss and the Euclidean distance loss; more loss functions can be conducted if necessary. The dot product loss originates from BPR-Opt [14]. Its variant adopts the conventional matrix factorization, referred to as BPR-MF, which is generally used as an underlying preference estimator. BPR-MF can be optimized by a contrastive pairwise ranking objective function of a triplet, referred to as product loss: where σ is the sigmoid function and · denotes the dot product computation. Because enumerating all triplets in T is typically intractable, BPR-MF uses stochastic gradient descent (SGD) to optimize the model. Specifically, in each step of SGD, a batch of triplets is dynamically sampled from T . In addition, L2 regularization was adopted on the user and item representations, which is crucial to alleviate overfitting.
The second global representation was generated based on the Euclidean distance metric, where the optimization function is distance loss expressed by [37]: where α is the margin.
To make the optimization effective, hard examples of triplets that result in large loss can be selected frequently during training. The network parameters are updated through SGD to minimize the loss function iteratively. In the following, the global representations of user u and item i are denoted asû andî, respectively.

Local representation
Roughly speaking, the global representations provide a macro view of users and items. However, they are insufficient to model complex user-item interactions. The local representations, on the other hand, are derived by learning local data, which are extracted from the clusters that are close to the user. They can model the user-item interactions from a micro view.
For simplicity, we assume the global representation space G is a Euclidean space. K-means clustering is used to construct a cluster set of size M . Given a query (user) u and its global representationû, J clusters are chosen from the M clusters that are the closest toû and are denoted as candidate clusters {C j } J j=1 . The corresponding codebook of centroids is {µ j |µ j ∈ G} J j=1 . To construct the local representation network, we employ the same network architecture defined in Fig. 2 but train with the local properties of the candidate clusters.
Suppose that a positive itemî + and a negative itemî − belong to two candidate clusters, i.e.,î + ∈ C j andî − ∈ C k , where j, k ∈ {1, · · · , J}. There are two considerations for generating the triplet training set. The first case is j = k, i.e., i + andî − are in the same cluster. They can be included in the local intra-triplet set T † ∈ T when satisfying the following criterion: The intuition is that, if the distance between u and i + is larger than that between u and i − , it will incur positive triplet loss. We can use this hard example to tune the representation network so that the user is close to its positive item and far away from its negative item. The other case is j = k, i.e.,î + andî − are in different clusters. A similar definition for the local inter-triplet set T ‡ ∈ T is given: Unlike Eq. (5) that considers the relation between the user and items, Eq. (6) addresses the relation between the user and clusters. That is, if the distance between u and C j (where i + belongs to) is larger than that between u and C k (where i − belongs to), we take this hard example to tune the user representation network so that the user is close to its positive cluster C j and far away from its negative cluster C k . Consequently, both T † and T ‡ serve as the local training dataset for learning local representations. The outputs of the local representation network are expressed as the userũ, the positive itemĩ + , and the negative itemĩ − . In a similar way to Eq. 3 and Eq. 4, the loss functions for training local representations are expressed as l prod (ũ,ĩ + ,ĩ − ) for product loss and l dist (ũ,ĩ + ,ĩ − ) for distance loss.

Attentive representation
By leveraging the global and local properties with different loss functions, multiple representation networks can be trained to generate the respective representations for each user/item. Moreover, an attention network is proposed to find the best combination based on these representations. That is, the proposed attention network can map a set of user-item representations to weighted sum representations of the user and item. The weight is assigned to each representation space by computing a compatibility function of the user and item. Details are described in the following.
Suppose that we construct R representation networks to derive corresponding representation spaces; each user/item has R representations. Fig. 3 gives an overview of the proposed network consisting of R representation networks followed by an attention network, where four representation spaces (R = 4) are taken as an example. In the training process, the input layer receives a triplet (u, i + , i − ). Let g (r) be the rth representation network, r ∈ 1, 2, ..., R, which transforms the user and items to their rth representations. We express the rth representations of the triplet as g (r) (u), g (r) (i + ), g (r) (i − ) , and forward them to the rth attention channel.
Each attention channel can be divided into three parts. The first part is a transformation function, denoted as f (r) , which maps the user and positive item to the respective vectors u (r) and i (r) : Note that the negative item is excluded in the attention network. Consequently, the second part is to calculate the compatibility between the user and positive item. The idea is paying more attention to the representation space where the user and the positive item are more compatible, as measured by the dot product. The compatibility is then convert to a normalized weight, denoted as α (r) , by applying the softmax function over all representation spaces.
In the last part, a weighted sum is calculated as the linear combinations of the R representation spaces for each user/item:ū The outputsū,ī + , andī − are denoted as attentive representations. The proposed model is trained by the triplet loss function based on either product loss l prod (ū,ī + ,ī − ) or distance loss l dist (ū,ī + ,ī − ), and the training data are randomly sampled from the global and local triplet sets T , T † , and T ‡ .

Recommendation as retrieval
We treat the recommendation task as a retrieval process that searches for similar items from multi-view representation

Attention network
Output layer (attentive representation) � Triplet loss (product/distance) Linear combination

Representation networks
Input layer (user/item ID) spaces for a given user. In clustering-based recommendation, a coarse-to-fine search strategy can be applied without scanning the dataset exhaustively, making recommendation more efficient and scalable. For each item i in the dataset, R representations are extracted by the set of representation networks {g (r) (i)|r = 1, 2, ..., R}. We choose one of the global representations and perform k-means clustering to create M clusters and to build the index structure with M clusters, each of which has attached an inverted list of associated items. When a user query u is submitted, the user representations {g (r) (u)} are extracted and used to search for the K nearest item clusters. We denote the items stored in the nearest clusters as the candidate items. For each pair of user u and candidate item i , the attention network is used to obtain the attentive representationsū andī defined in Eq. 9 and to compute their Euclidean distance or dot product to estimate their similarity. The candidates are sorted to output the top-N items as the recommendation result for user u. The retrieval process is summarized in Algorithm 1.
By taking advantage of the coarse-to-fine search strategy, only some dataset items are selected as candidates for the attention and re-ranking computation. Although applying the attention network for each user-item pair requires additional computations, these can be accelerated by GPUs, and thus the runtime is usually minor. Therefore, the time complexity of the retrieval process can be considered sublinear with respect to the size of the item dataset. {C k |C k ∈ Φ} K k=1 , K < M for u by calculating the maximum dot products or minimal Euclidean distances between u and the item cluster centroid; 3 Let I = {i|i ∈ ∪ k∈{1,2,...,K} L k , L k ∈ Θ } be the set of candidate items from the top-K item clusters; 4 Return the top-N items {i j |i j ∈ I} N j=1 with the maximum dot products or minimum Euclidean distances betweenū andī j from Eq. 9;

EXPERIMENTAL RESULTS
This section describes the evaluation of the proposed method and a comparison with several CF-based recommender systems. The experiments were carried out under Windows 10 with an Intel Core i7-6700 CPU, an NVIDIA GeForce GTX 1080Ti GPU, and 64GB RAM. The programming environment was the Keras package of Tensorflow under Python 3.7. The source code is available

Benchmark datasets
Four popular benchmark datasets, including MovieLens, Yelp, and Amazon, were used in the experimental evaluation. We briefly introduce these datasets, as well as the corresponding training/test data partitions: • MovieLens-100k. This dataset consists of 100,000 records of user-movie ratings from 943 users and 1,682 movies. Each user has rated at least 20 movies. We randomly sampled half of the user-movie views as test data, and used the other half for training.
• Yelp Madison. The Yelp dataset is a set of user reviews of businesses in local cities. Here, we selected the city of Madison. Users who had more than five business interactions were kept; the others were removed from the dataset. For each of the remaining users, we randomly sampled three userbusiness interactions as the test data, and used the rest for training.
• Yelp Pittsburgh. Yelp Pittsburgh is another dataset containing user reviews of businesses in Pittsburgh. The user filtering and training/test data generation were the same as for Yelp Madison.
• Amazon. This dataset contains product reviews from customers of Amazon.com. For this study, the category Movie and TV was chosen. Users with fewer than six reviews were removed from the dataset. The training/test data were generated in the same way as for Yelp Madison and Yelp Pittsburgh. Table 1 summarizes some statistic properties of these datasets. Their scales and densities are very different from each other, providing a diverse way for observation.

Evaluation metrics
Several widely used accuracy metrics in top-N recommendation, including precision, recall, hit rate (HR), averagereciprocal hit rank (ARHR), and normalized discounted cumulative gain (NDCG), were used for experiment evaluation. For a user query u, the top-N recommended items are denoted as Q u = {i 1 , i 2 , ..., i N } and the ground truth (positive) items as G u = {i 1 , i 2 , ..., i |Gu| }, where | · | is the set cardinality. These metrics are defined as follows: where rank i is the ranking position of item i. Although a higher value reflects a better accuracy in general, it is recommended to assess the model performance from multiple metrics rather than any single metric.

Hyperparameter and ablation study
To study the impact of model parameters, we take the following four global and local representations, as well as two attentive representations, for consideration: • Global representation / distance loss (GD) was generated based on the global training set (Eq. (2)) and the distance loss function (Eq. (4)).
• Global representation / product loss (GP) was generated based on the global training set and product loss function (Eq. (3)).
• Local representation / distance loss (LD) was generated based on the local training set (Eqs. (5) and (6)) and the distance loss function.
• Local representation / product loss (LP) was generated based on the local training set and the product loss function.
• Attentive representation / distance loss (AD) was blended from the above four representations by the proposed attention network and the distance loss function.
• Attentive representation / product loss (AP) was blended by the proposed attention network and the product loss function.
Except for using different training datasets and loss functions, these representations adopted the same representation network shown in Fig. 2, which contains one embedding layer and two hidden layers, each of which has 64 neurons. The dimensionality of the output layer was also set to 64.  Fig. 4, where each column shows a particular dataset with different M . Each plot shows six recall curves of the proposed representations under the top-K candidate clusters, K ∈ {1, 2, 3, 4, 5}. We ranked the items in the candidate clusters to return the top-20 items for recommendation.
As the number of candidate clusters K is increased, more positive items are found from the candidate clusters, and therefore the recall curves is generally improved. In addition, most of them show a sharp rise at K = 2. However, some recall curves have a degraded performance as K increases. This means that the corresponding model does not generate good representation spaces, so that the relevance between the user and positive item is less than that between the user and negative item. The proposed attentive representations (AD and AP) are relatively stable under these configurations, showing that they became more robust by adaptively fusing multiple representations.
In general, local representations (LD and LP) are better than global representations (GD and GP). However, no single representation always performs better than others because of the variety of user-item interaction behaviors. Again, the proposed attention network combining different views of representation spaces yield the best recall on all the benchmark datasets. Next, we observe the recommendation result from more evaluation metrics, as shown in Figs. 5, 6, 7 and 8 for MovieLens, Yelp Madison, Yelp Pittsburgh, and Amazon, respectively. Each figure contains four subplots, including recall, precision, HR, and ARHR (from left to right). The X-axis represents the number of top-N recommendation items, N ∈ {5, 10, 15, 20, 25, 30}, and the Y-axis represents the accuracy. The number of candidate clusters is fixed at K = 2. Again we see the proposed attention networks outperform the other representation networks under these accuracy metrics. Particularly, the proposed method AP is superior to AD under the ARHR metric, even they are comparable under the other three metrics. Note that ARHR measures the ranking quality that assigns a high score to hits at top position ranks. That is, AP could provide a better ranking quality. Another interesting phenomenon is, for the HR metric, most of the curves show a sharp rise at N = 10. Since HR is an indication whether at least one positive item is in the top-N set, we suggest N ≥ 10 would be a good number of recommendation items.
An ablation study is introduced for the attention network of the global and local representations. Three representation types are considered: global, local, and all. The global type consists of GD, GP, GAD (global attentive representations with distance loss), and GAP (global attentive representations with product loss). The local type follows the similar definition. We fixed the number of candidate clusters at K = 2 and the number of recommended items at N = 15. Tables 2 and 3 list the performance in the four benchmark datasets. The number in boldface indicates the best accuracy in each metric column. It is clear to see that with the attention network, the global attentive representations (i.e., GAD and GAP) gain further improvement by combining the two global representations. The local attentive representations (i.e., LAD and LAP) also gain the similar improvement. By combining all global and local representations together, the attention network can yield the best accuracy, as demonstrated by AD and AP.

Comparison with other CF methods
We compared several CF recommendation systems, including network-based, factorization-based, and clusteringbased approaches:

CIGAR [13]
: This is also a clustering-based method that uses a hashing technique to retrieve candidate items and learns a ranking model to re-rank the candidate items.
Tables 4 and 5 present their performances in terms of recall, precision, HR, and ARHR. The clustering-based approach (i.e., N2VSCDNNR, CIGAR, AP and AD) yields a better performance than the other approaches in the diverse benchmark datasets, demonstrating its superiority to address the sparsity and scalability problems to a certain extent. Furthermore, CIGAR performs better than N2VSCDNNR in the sparser and larger datasets, i.e., Yelp and Amazon. However, these compared methods employ a single representation, which is insufficient to model complex user-item interactions. The proposed attention model combines multiple representations to regenerate joint representations for each user-item pair dynamically, thus gain the best performance in various benchmark datasets.
Finally, we performed leave-one-out evaluation to compare our methods with two other CF methods. The compared methods are end-to-end learning-based recommender systems: • NeuMF [5]: This system generalizes matrix factorization by nonlinear neural networks to model useritem interactions.
• BCFNet [30]: This system, which is an extension of DeepCF [17], uses an attention mechanism to improve the representation learning and matching function learning.
In leave-one-out evaluation, we held-out the latest interaction as a test item for each user and utilized the remaining data for training. For testing we followed the same way of the above two methods: we randomly selected 99 unobserved interactions as the negative examples for each user,  together with the latest interaction as the positive example.
In total 100 test are ranked, and the top-10 items were evaluated by HR and NDCG. Table 6 lists the leave-one-out evaluation for the four benchmark datasets. Note the BCFNet's result for Amazon is not available because the memory consumption is too large to be executed in our computer. For each dataset, we generated three representation spaces of 32, 64, and 128 dimensions. Our methods yield better HR and NDCG scores in the high-dimensional representation space. In particular, our NDCG scores outperform NeuMF and BCFNet with a large gap, indicating our methods can rank the positive example at a superior position. The proposed method is able to find better representations for users and items through separate representation networks. Separating items into positive and negative ones also has an important impact on the recommendation task. In addition, similar to BCFNet, the attention mechanism can be used to highlight important features. However, because the local representations are derived by learning local data, which are extracted from the clusters close to the user, when the density vector is low-dimensional, it may be easier to mix some dissimilar data points in a cluster than that is high-dimensional. The local representations that come out may be poor, and in turn affect the final recommendation results.

CONCLUSION
In this study, we proposed a novel clustering-based CF method for recommender systems. The user and item representations are learned from multiple views in terms of global/local representation spaces and dot product/Euclidean distance loss functions. An attention network is designed to generate a joint representation of these views for each user-item pair dynamically. Experimental results show the proposed method is effective and competitive compared to several CF methods where only one representation space is considered.