We adopted timestamp-based leave-k-out evaluation method [1]}, [2]}, [3]}. For each user, we held-out the last \(k\)  feedback for evaluation and used the rest for training. In the existing work, \(k\)  was usually set to 1. However, we observed that when \(k\)  was too small, the variance in the results was large, i.e., the local models were subject to overfitting (potentially more than a global model as they were trained on smaller subsets). Thus, local models tend to achieve much higher accuracy when \(k = 1\) . For a fair comparison, we thus set \(k = 5\) , indicating more stable accuracy.
