<jupyter_start><jupyter_text>This notebook was created to practise collaborative filtering [assignment 09](https://github.com/simecek/dspracticum2020/tree/master/lecture_09) in subject [Data Science Practicum 2020](https://github.com/simecek/dspracticum2020) and was written following keras example [Collaborative Filtering for Movie Recommendations](https://keras.io/examples/structured_data/collaborative_filtering_movielens/). Setup<jupyter_code>import pandas as pd
import numpy as np
from tensorflow import keras
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import math
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Prepare the data<jupyter_code>path = keras.utils.get_file(
    "tweet_scores.csv", origin="https://raw.githubusercontent.com/simecek/dspracticum2020/master/lecture_09/assignment/fake_v1_100x252.csv"
)
data = pd.read_csv(path, index_col=0)
data<jupyter_output><empty_output><jupyter_text>To follow Collaborative Filtering tutorial we need to reshape our data to form `[userId, tweetId, rating]` and keep only combinations with non-nan ratings.<jupyter_code>df = pd.DataFrame([user, tweet, data[tweet][user]] for user in data.index for tweet in data.columns if not math.isnan(data[tweet][user]))
df.columns=['userId', 'tweetId', 'rating']
df<jupyter_output><empty_output><jupyter_text>Now we need to perform some preprocessing to encode users and tweets as integer indices.<jupyter_code>user_ids = df["userId"].unique().tolist()
user2user_encoded = {x: i for i, x in enumerate(user_ids)}
userencoded2user = {i: x for i, x in enumerate(user_ids)}
tweet_ids = df["tweetId"].unique().tolist()
tweet2tweet_encoded = {x: i for i, x in enumerate(tweet_ids)}
tweet_encoded2tweet = {i: x for i, x in enumerate(tweet_ids)}
df["user"] = df["userId"].map(user2user_encoded)
df["tweet"] = df["tweetId"].map(tweet2tweet_encoded)

num_users = len(user2user_encoded)
num_tweets = len(tweet_encoded2tweet)
df["rating"] = df["rating"].values.astype(np.float32)
# min and max ratings will be used to normalize the ratings later
min_rating = min(df["rating"])
max_rating = max(df["rating"])

print(
    "Number of users: {}, Number of Tweets: {}, Min rating: {}, Max rating: {}".format(
        num_users, num_tweets, min_rating, max_rating
    )
)<jupyter_output>Number of users: 100, Number of Tweets: 252, Min rating: 1.0, Max rating: 6.0<jupyter_text>Prepare training and validation data<jupyter_code>df = df.sample(frac=1, random_state=42)
x = df[["user", "tweet"]].values
# Normalize the targets between 0 and 1. Makes it easy to train.
y = df["rating"].apply(lambda x: (x - min_rating) / (max_rating - min_rating)).values
# Assuming training on 90% of the data and validating on 10%.
train_indices = int(0.9 * df.shape[0])
x_train, x_val, y_train, y_val = (
    x[:train_indices],
    x[train_indices:],
    y[:train_indices],
    y[train_indices:],
)<jupyter_output><empty_output><jupyter_text>Create the modelWe embed both users and tweet in to 50-dimensional vectors.The model computes a match score between user and tweet embeddings via a dot product, and adds a per-tweet and per-user bias. The match score is scaled to the [0, 1] interval via a sigmoid (since our ratings are normalized to this range).<jupyter_code>EMBEDDING_SIZE = 10

class RecommenderNet(keras.Model):
    def __init__(self, num_users, num_tweets, embedding_size, **kwargs):
        super(RecommenderNet, self).__init__(**kwargs)
        self.num_users = num_users
        self.num_tweets = num_tweets
        self.embedding_size = embedding_size
        self.user_embedding = layers.Embedding(
            num_users,
            embedding_size,
            embeddings_initializer="he_normal",
            embeddings_regularizer=keras.regularizers.l2(1e-6),
        )
        self.user_bias = layers.Embedding(num_users, 1)
        self.tweet_embedding = layers.Embedding(
            num_tweets,
            embedding_size,
            embeddings_initializer="he_normal",
            embeddings_regularizer=keras.regularizers.l2(1e-6),
        )
        self.tweet_bias = layers.Embedding(num_tweets, 1)

    def call(self, inputs):
        user_vector = self.user_embedding(inputs[:, 0])
        user_bias = self.user_bias(inputs[:, 0])
        tweet_vector = self.tweet_embedding(inputs[:, 1])
        tweet_bias = self.tweet_bias(inputs[:, 1])
        dot_user_tweet = tf.tensordot(user_vector, tweet_vector, 2)
        # Add all the components (including bias)
        x = dot_user_tweet + user_bias + tweet_bias
        # The sigmoid activation forces the rating to between 0 and 1
        return tf.nn.sigmoid(x)


model = RecommenderNet(num_users, num_tweets, EMBEDDING_SIZE)
model.compile(
    loss=tf.keras.losses.BinaryCrossentropy(), 
    optimizer=keras.optimizers.Adam(lr=0.001),
    metrics=['accuracy']
)<jupyter_output><empty_output><jupyter_text>Train the model based on the data split<jupyter_code>history = model.fit(
    x=x_train,
    y=y_train,
    batch_size=64,
    epochs=50,
    verbose=1,
    validation_data=(x_val, y_val),
)<jupyter_output>Epoch 1/50
71/71 [==============================] - 0s 6ms/step - loss: 0.7025 - accuracy: 0.0071 - val_loss: 0.7039 - val_accuracy: 0.0080
Epoch 2/50
71/71 [==============================] - 0s 4ms/step - loss: 0.6764 - accuracy: 0.0124 - val_loss: 0.6985 - val_accuracy: 0.0100
Epoch 3/50
71/71 [==============================] - 0s 4ms/step - loss: 0.6696 - accuracy: 0.0142 - val_loss: 0.6950 - val_accuracy: 0.0120
Epoch 4/50
71/71 [==============================] - 0s 4ms/step - loss: 0.6678 - accuracy: 0.0147 - val_loss: 0.6942 - val_accuracy: 0.0120
Epoch 5/50
71/71 [==============================] - 0s 4ms/step - loss: 0.6711 - accuracy: 0.0140 - val_loss: 0.6927 - val_accuracy: 0.0120
Epoch 6/50
71/71 [==============================] - 0s 4ms/step - loss: 0.6701 - accuracy: 0.0138 - val_loss: 0.6944 - val_accuracy: 0.0120
Epoch 7/50
71/71 [==============================] - 0s 4ms/step - loss: 0.6687 - accuracy: 0.0142 - val_loss: 0.6951 - val_accuracy: 0.0120
Epoch 8/50
71/71 [==[...]<jupyter_text>Plot training and validation accuracy<jupyter_code>plt.plot(history.history["accuracy"])
plt.plot(history.history["val_accuracy"])
plt.title("model accuracy")
plt.ylabel("accuracy")
plt.xlabel("epoch")
plt.legend(["train", "val"], loc="upper left")
plt.show()<jupyter_output><empty_output><jupyter_text>Predict score for all tweets Just a reminder, this is our input table with missing scores.<jupyter_code>data<jupyter_output><empty_output><jupyter_text>There is definitely a better way than doing it one by one in nested for loop, but I wanted to make sure I will not missmatch indices. So run this only if you have few spare minutes.<jupyter_code>def scale_score(score):
  '''
  Scales score from interval <0, 1> to interval <min_rating, max_rating>.
  '''
  return score * (max_rating - min_rating) + min_rating
for user in data.index: 
    for tweet in data.columns:
        if math.isnan(data[tweet][user]):
            user_id = user2user_encoded[user]
            tweet_id = tweet2tweet_encoded[tweet]
            predicted = model.predict([[user_id, tweet_id]])
            data[tweet][user] = scale_score(predicted[0][0])
data<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14350205198358412, 'fraction_numerical': 0.04213406292749658, 'mean_word_length': 4.455970149253731, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '339198', 'license': 'MIT', 'n_tokens_mistral': 2725, 'n_tokens_neox': 2474, 'n_words': 612, 'path': 'assignment_09/TweetRecommendation.ipynb', 'repo_name': 'katarinagresova/M7DataSP_2020', 'size': '7310'}