<jupyter_start><jupyter_text>Copyright (c) Microsoft Corporation. All rights reserved.Licensed under the MIT License.  NAML: Neural News Recommendation with Attentive Multi-View LearningNAML \[1\] is a multi-view news recommendation approach. The core of NAML is a news encoder and a user encoder. The newsencoder is composed of a title encoder, a body encoder, a vert encoder and a subvert encoder. The CNN-based title encoder and body encoder learn title and body representations by capturing words semantic information. After getting news title, body, vert and subvert representations, an attention network is used to aggregate those vectors. In the user encoder, we learn representations of users from their browsed news. Besides, we apply additive attention to learn more informative news and user representations by selecting important words and news. Properties of NAML:- NAML is a multi-view neural news recommendation approach.- It uses news title, news body, news vert and news subvert to get news repersentations. And it uses user historical behaviors to learn user representations.- NAML uses additive attention to learn informative news and user representations by selecting important words and news. Data format: train dataOne simple example: `1 0 0 0 0 Impression:0 User:502 CandidateTitle0:17917,36557,47926,32224,24113,48923,19086,5636,3703,0... CandidateBody0:17024,53305,8832,29800,9787,4068,48731,48923,19086,38699,5766,22487,38336,29800,8548,39128,33457,7789,30543,7482,8548,49004,53305,22999,32747,21103,11799,5766,4868,17115,7482,15118,48731,2025,7789,23336,7789,48731,19086,10630,11128,36557,3703,47354,611,7789,19086,5636,51521,30706... CandidateVert0:14... CandidateSubvert0:219... ClickedTitle0:48,33405,35198,5969,5636,35845,850,48731,46799,24113... ClickedBody0:36557,67,34519,24113,8548,48,33405,35198,5969,14340,7053,850,8823,9498,46799,24113,12506,32747,31130,3074,48731,20869,14264,38289,37310,7789,36557,34967,48731,36916,23321,3595,48731,47354,4868,15719,7482,12771,50693,47354,17523,48,20918,17900,35198,48731,20869,1220,14264,7789... ClickedVert0:14... ClickedSubvert0:99... `In general, each line in data file represents one positive instance and n negative instances in a same impression. The format is like: `[label0] ... [labeln] [Impression:i] [User:u] [CandidateTitle0:w1,w2,w3,...] ... [CandidateBody0:w1,w2 ..] ... [CandidateVert0:v] ... [CandidateSubvert0:s] ... [ClickedTitle0:w1,w2,w3,...] ... [ClickedBody0:w1,w2,w3,...] ... [ClickedVert0:v] ... [ClickedSubvert0:s] ...`It contains several parts seperated by space, i.e. label part, Impression part ``, User part ``, CandidateNews part, ClickedHistory part. CandidateNews part describes the target news article we are going to score in this instance. It is represented by (aligned) title words, body words, news vertical index and subvertical index. To take a quick example, a news title may be : `Trump to deliver State of the Union address next week` , then the title words value may be `CandidateTitlei:34,45,334,23,12,987,3456,111,456,432`. ClickedHistory describe the k-th news article the user ever clicked and the format is the same as candidate news. Every clicked news has title words, body words, vertical and subvertical. We use a fixed length to describe an article, if the title or body is less than the fixed length, just pad it with zeros. test dataOne simple example: `1 Impression:0 User:1529 CandidateTitle0:5327,18658,13846,6439,611,50611,0,0,0,0 CandidateBody0:13846,3197,27902,225,5327,45008,29145,7789,509,7395,11502,36557,13846,23680,26492,38072,20507,5636,4247,32747,50132,7482,41049,32747,43022,50611,35979,7789,1191,36557,52870,21622,48148,42737,48731,36557,13846,23680,13173,7482,13848,38072,20507,7789,41675,36875,51461,12348,21045,42160 CandidateVert0:14 CandidateSubvert0:19 ClickedTitle0:9079,3703,32747,8546,19377,50184,32747,24026,40010,49754 ... ClickedBody0:26061,48731,8576,7789,8683,9079,5636,45084,46805,3703,509,43036,11502,28883,9498,18450,32747,8546,33405,35647,50184,7482,41143,8220,43618,38072,35198,43390,28057,32552,45245,10764,16247,4221,41038,36557,43683,46805,7789,29727,2179,51003,34797,897,21045,12974,23382,46287,48731,15206 ... ClickedVert0:14 ... ClickedSubvert0:219 ...`In general, each line in data file represents one instance. The format is like: `[label] [Impression:i] [User:u] [CandidateTitle0:w1,w2,w3,...] [CandidateBody0:w1,w2,w3,...] [CandidateVert0:v] [CandidateSubvert0:s] [ClickedTitle0:w1,w2,w3,...] ... [ClickedBody0:w1,w2,w3,...] ... [ClickedVert0:v] ... [ClickedSubvert0:s] ...`  Global settings and imports<jupyter_code>import sys
sys.path.append("../../")
from reco_utils.recommender.deeprec.deeprec_utils import download_deeprec_resources 
from reco_utils.recommender.newsrec.newsrec_utils import prepare_hparams
from reco_utils.recommender.newsrec.models.naml import NAMLModel
from reco_utils.recommender.newsrec.io.naml_iterator import NAMLIterator
import papermill as pm
from tempfile import TemporaryDirectory
import tensorflow as tf
import os

print("System version: {}".format(sys.version))
print("Tensorflow version: {}".format(tf.__version__))

tmpdir = TemporaryDirectory()<jupyter_output>/data/anaconda/envs/reco_gpu/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/data/anaconda/envs/reco_gpu/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/data/anaconda/envs/reco_gpu/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/data/anaconda/envs/reco_gpu/lib/python3.6/site-packages/tensorf[...]<jupyter_text>Download and load data<jupyter_code>data_path = tmpdir.name
yaml_file = os.path.join(data_path, r'naml.yaml')
train_file = os.path.join(data_path, r'train.txt')
valid_file = os.path.join(data_path, r'test.txt')
wordEmb_file = os.path.join(data_path, r'embedding.npy')
if not os.path.exists(yaml_file):
    download_deeprec_resources(r'https://recodatasets.blob.core.windows.net/newsrec/', data_path, 'naml.zip')<jupyter_output>100%|██████████| 72.6k/72.6k [00:04<00:00, 18.0kKB/s]<jupyter_text>Create hyper-parameters<jupyter_code>epochs=5
seed=42
hparams = prepare_hparams(yaml_file, wordEmb_file=wordEmb_file, epochs=epochs)
print(hparams)
iterator = NAMLIterator<jupyter_output><empty_output><jupyter_text>Train the NAML model<jupyter_code>model = NAMLModel(hparams, iterator, seed=seed)
print(model.run_eval(valid_file))
model.fit(train_file, valid_file)
res_syn = model.run_eval(valid_file)
print(res_syn)
pm.record("res_syn", res_syn)<jupyter_output>{'group_auc': 0.5599, 'mean_mrr': 0.2027, 'ndcg@5': 0.2065, 'ndcg@10': 0.268}
{'fraction_non_alphanumeric': 0.12861691817804236, 'fraction_numerical': 0.1794268309566662, 'mean_word_length': 6.912376779846659, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 14, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '350299', 'license': 'MIT', 'n_tokens_mistral': 3330, 'n_tokens_neox': 2587, 'n_words': 598, 'path': 'notebooks/00_quick_start/naml_synthetic.ipynb', 'repo_name': 'danyang-liu/recommenders', 'size': '7223'}