<jupyter_start><jupyter_text>Copyright (c) Microsoft Corporation. All rights reserved.Licensed under the MIT License.  Data split Data splitting is one of the most vital tasks in assessing recommendation systems. Splitting strategy greatly affects the evaluation protocol so that it should always be taken into careful consideration by practitioners.The code hereafter explains how one applies different splitting strategies for specific scenarios.  0 Global settings<jupyter_code># set the environment path to find Recommenders
import sys
sys.path.append("../../")

import pyspark
import pandas as pd
import numpy as np
from datetime import datetime, timedelta

from reco_utils.common.spark_utils import start_or_get_spark
from reco_utils.dataset.download_utils import maybe_download
from reco_utils.dataset.python_splitters import (
    python_random_split, 
    python_chrono_split, 
    python_stratified_split
)
from reco_utils.dataset.spark_splitters import (
    spark_random_split, 
    spark_chrono_split, 
    spark_stratified_split,
    spark_timestamp_split
)

print("System version: {}".format(sys.version))
print("Pyspark version: {}".format(pyspark.__version__))
DATA_URL = "http://files.grouplens.org/datasets/movielens/ml-100k/u.data"
DATA_PATH = "ml-100k.data"

COL_USER = "UserId"
COL_ITEM = "MovieId"
COL_RATING = "Rating"
COL_PREDICTION = "Rating"
COL_TIMESTAMP = "Timestamp"<jupyter_output><empty_output><jupyter_text>1 Data preparation  1.1 Data understanding For illustration purpose, the data used in the examples below is the MovieLens-100K dataset.<jupyter_code>filepath = maybe_download(DATA_URL, DATA_PATH)
data = pd.read_csv(filepath, sep="\t", names=[COL_USER, COL_ITEM, COL_RATING, COL_TIMESTAMP])<jupyter_output><empty_output><jupyter_text>A glimpse at the data<jupyter_code>data.head()<jupyter_output><empty_output><jupyter_text>A little more...<jupyter_code>data.describe()<jupyter_output><empty_output><jupyter_text>And, more...<jupyter_code>print(
    "Total number of ratings are\t{}".format(data.shape[0]),
    "Total number of users are\t{}".format(data[COL_USER].nunique()),
    "Total number of items are\t{}".format(data[COL_ITEM].nunique()),
    sep="\n"
)<jupyter_output>Total number of ratings are	100000
Total number of users are	943
Total number of items are	1682<jupyter_text>1.2 Data transformation Original timestamps are converted to ISO format.<jupyter_code>data[COL_TIMESTAMP]= data.apply(
    lambda x: datetime.strftime(datetime(1970, 1, 1, 0, 0, 0) + timedelta(seconds=x[COL_TIMESTAMP].item()), "%Y-%m-%d %H:%M:%S"), 
    axis=1
)
data.head()<jupyter_output><empty_output><jupyter_text>2 Experimentation protocol Experimentation protocol is usually set up to favor a reasonable evaluation for a specific recommendation scenario. For example,* *Recommender-A* is to recommend movies to people by taking people's collaborative rating similarities. To make sure the evaluation is statisically sound, the same set of users for both model building and testing should be used (to avoid any cold-ness of users), and a stratified splitting strategy should be taken.* *Recommender-B* is to recommend fashion products to customers. It makes sense that evaluation of the recommender considers time-dependency of customer purchases, as apparently, tastes of the customers in fashion items may be drifting over time. In this case, a chronologically splitting should be used.  3 Data split  3.1 Random split Random split simply takes in a data set and outputs the splits of the data, given the split ratios.<jupyter_code>data_train, data_test = python_random_split(data, ratio=0.7)
data_train.shape[0], data_test.shape[0]<jupyter_output><empty_output><jupyter_text>Sometimes a multi-split is needed.<jupyter_code>data_train, data_validate, data_test = python_random_split(data, ratio=[0.6, 0.2, 0.2])
data_train.shape[0], data_validate.shape[0], data_test.shape[0]<jupyter_output><empty_output><jupyter_text>Ratios can be integers as well.<jupyter_code>data_train, data_validate, data_test = python_random_split(data, ratio=[3, 1, 1])<jupyter_output><empty_output><jupyter_text>For producing the same results.<jupyter_code>data_train.shape[0], data_validate.shape[0], data_test.shape[0]<jupyter_output><empty_output><jupyter_text>3.2 Chronological split Chronogically splitting method takes in a dataset and splits it on timestamp.   3.2.1 "Filter by" Chrono splitting can be either by "user" or "item". For example, if it is by "user" and the splitting ratio is 0.7, it means that first 70% ratings for each user in the data will be put into one split while the other 30% is in another. It is worth noting that a chronological split is not "random" because splitting is timestamp-dependent.<jupyter_code>data_train, data_test = python_chrono_split(
    data, ratio=0.7, filter_by="user",
    col_user=COL_USER, col_item=COL_ITEM, col_timestamp=COL_TIMESTAMP
)<jupyter_output><empty_output><jupyter_text>Take a look at the results for one particular user: * The last 10 rows of the train data:<jupyter_code>data_train[data_train[COL_USER] == 1].tail(10)<jupyter_output><empty_output><jupyter_text>* The first 10 rows of the test data:<jupyter_code>data_test[data_test[COL_USER] == 1].head(10)<jupyter_output><empty_output><jupyter_text>Timestamps of train data are all precedent to those in test data.  3.3.2 Min-rating filter A min-rating filter is applied to data before it is split by using chronological splitter. The reason of doing this is that, for multi-split, there should be sufficient number of ratings for user/item in the data.For example, the following means splitting only applies to users that have at least 10 ratings.<jupyter_code>data_train, data_test = python_chrono_split(
    data, filter_by="user", min_rating=10, ratio=0.7,
    col_user=COL_USER, col_item=COL_ITEM, col_timestamp=COL_TIMESTAMP
)<jupyter_output><empty_output><jupyter_text>Number of rows in the yielded splits of data may not sum to the original ones as users with fewer than 10 ratings are filtered out in the splitting.<jupyter_code>data_train.shape[0] + data_test.shape[0], data.shape[0]<jupyter_output><empty_output><jupyter_text>3.3 Stratified split Chronogically splitting method takes in a dataset and splits it by either user or item. The split is stratified so that the same set of users or items will appear in both training and testing data sets. Similar to chronological splitter, `filter_by` and `min_rating_filter` also apply to the stratified splitter.The following example shows the split of the sample data with a ratio of 0.7, and for each user there should be at least 10 ratings.<jupyter_code>data_train, data_test = python_stratified_split(
    data, filter_by="user", min_rating=10, ratio=0.7,
    col_user=COL_USER, col_item=COL_ITEM
)
data_train.shape[0] + data_test.shape[0], data.shape[0]<jupyter_output><empty_output><jupyter_text>3.4 Data split in scale Spark DataFrame is used for scalable splitting. This allows splitting operation performed on large dataset that is distributed across Spark cluster. For example, the below illustrates how to do a random split on the given Spark DataFrame. For simplicity reason, the same MovieLens data, which is in Pandas DataFrame, is transformed into Spark DataFrame and used for splitting.<jupyter_code>spark = start_or_get_spark()
data_spark = spark.read.csv(filepath)
data_spark_train, data_spark_test = spark_random_split(data_spark, ratio=0.7)<jupyter_output><empty_output><jupyter_text>Interestingly, it was noticed that Spark random split does not guarantee a deterministic result. This sometimes leads to issues when data is relatively small while users seek for a precision split.<jupyter_code>data_spark_train.count(), data_spark_test.count()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08248747269690351, 'fraction_numerical': 0.014904278555826801, 'mean_word_length': 6.288389513108614, 'pattern_counts': {'":': 0, '<': 72, '<?xml version=': 0, '>': 72, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '27124', 'license': 'MIT', 'n_tokens_mistral': 2436, 'n_tokens_neox': 2318, 'n_words': 799, 'path': 'notebooks/01_prepare_data/data_split.ipynb', 'repo_name': 'bethz/Recommenders', 'size': '7783'}