<jupyter_start><jupyter_text>Data Preparation Load the recommendation train and test data files, and convert them into "prepared" data that can be directly consumed by session_rec repo to run baselines for the recommendation task.<jupyter_code>def label_encode_series(series: pd.Series):
    """
    Applies label encoding to a Pandas series and returns the encoded series,
    together with the label to index and index to label mappings.
    :param series: input Pandas series
    :return: Pandas series with label encoding, label-integer mapping and integer-label mapping.
    """
    labels = set(series.unique())
    label_to_index = {l: idx for idx, l in enumerate(labels)}
    index_to_label = {v: k for k, v in label_to_index.items()}
    return series.map(label_to_index), label_to_index, index_to_label<jupyter_output><empty_output><jupyter_text>Generate 'prepared' train<jupyter_code>train_data_df = pd.read_csv(TRAIN_PATH)
train_data_df
session_ids = set(train_data_df['session_id_hash'].unique())
train_cutoff = int(len(session_ids) * TRAIN_RATIO)
train_session_ids = list(session_ids)[:train_cutoff]
train_data_df = train_data_df[train_data_df['session_id_hash'].isin(train_session_ids)]<jupyter_output><empty_output><jupyter_text>Filter out* `remove from cart` events to avoid feeding them to session_rec as positive signals* rows with null product_sku_hash* sessions with only one action<jupyter_code>train_data_df = train_data_df[train_data_df['product_action'] != 'remove']
train_data_df = train_data_df.dropna(subset=['product_sku_hash'])
train_data_df['session_len_count'] = train_data_df.groupby('session_id_hash')['session_id_hash'].transform('count')
train_data_df = train_data_df[train_data_df['session_len_count'] >= 2]
# sort by session, then timestamp
train_data_df = train_data_df.sort_values(['session_id_hash', 'server_timestamp_epoch_ms'], ascending=True)

# Encode labels with integers
item_id_int_series, item_label_to_index, item_index_to_label = label_encode_series(train_data_df.product_sku_hash)
item_string_set = set(item_label_to_index.keys())

# Add tokenized session ID, tokenized item ID, and seconds since epoch time.
train_data_df[SessionId] = train_data_df.groupby([train_data_df.session_id_hash]).grouper.group_info[0]
train_data_df[Time] = train_data_df.server_timestamp_epoch_ms / 1000
train_data_df[ItemId] = item_id_int_series

# Get final dataframe
final_train_df = train_data_df[[SessionId, ItemId, Time]]

final_train_df
# Generate CSV and label encoder
final_train_df.to_csv(PREPARED_TRAIN_PATH, sep='\t', index=False)
pickle.dump(item_index_to_label, ITEM_LABEL_ENCODING_MAP_PATH.open(mode='wb'))
print("Done generating 'prepared' for training")<jupyter_output>Done generating 'prepared' for training<jupyter_text>Generate 'prepared' test<jupyter_code>with TEST_PATH.open() as f:
    test_data = json.load(f)
test_output = []
dummy_label_item = next(iter(item_string_set))
for idx, query_label in enumerate(test_data):
    query = query_label['query']

    # Here, since we must give a label in order to use session_rec repo, we use a dummy item
    # from train data to fill the label for testing data generation;
    # it is fine since we do not need to obtain the real metrics from the session_rec run,
    # the final metrics will be generated using our own scripts)
    first_label_item = dummy_label_item

    cleaned_query_events = []
    for q in query:
        # Skip if it is an action with no item id or it is remove-from-cart or it is unseen from train data
        if q['product_sku_hash'] is None \
                or q['product_action'] == 'remove' \
                or q['product_sku_hash'] not in item_string_set:
            continue
        q_session_id = idx
        q_session_item = q['product_sku_hash']
        q_session_time = q['server_timestamp_epoch_ms'] / 1000
        q_event = {SessionId: q_session_id,
                   ItemId: q_session_item,
                   Time: q_session_time}
        cleaned_query_events.append(q_event)
    if len(cleaned_query_events) > 0:
        # sessionId is mapped to idx integer
        l_event = {SessionId: idx,
                   ItemId: first_label_item,
                   Time: cleaned_query_events[-1][Time] + 1}
        test_output = test_output + cleaned_query_events
        test_output = test_output + [l_event]
# Create the final dataframe and apply label encoding.
test_output_df = pd.DataFrame(test_output)
test_output_df['ItemId'] = test_output_df.ItemId.map(item_label_to_index)
test_output_df
test_output_df.to_csv(PREPARED_TEST_PATH, sep='\t', index=False)
print("Done generating 'prepared' for testing")
%%writefile config.yml
type: single # single|window, maybe add opt
key: models #added to the csv names
evaluation: evaluation_last #evaluation|evaluation_last|evaluation_multiple
data:
  name: sigir #added in the end of the csv names
  folder: data/prepared/
  prefix: sigir

results:
  folder: results/last/sigir/

metrics:
- class: accuracy.HitRate
  length: [3,5,10,15,20]
- class: accuracy.MRR
  length: [3,5,10,15,20]
- class: accuracy_multiple.NDCG
  length: [3,5,10,15,20]
- class: coverage.Coverage
  length: [20]
- class: popularity.Popularity
  length: [20]
- class: saver.Saver
  length: [20]

algorithms:
- class: gru4rec.gru4rec.GRU4Rec
  params: { loss: 'top1-max', final_act: 'linear', dropout_p_hidden: 0.1, learning_rate: 0.08, momentum: 0.1, constrained_embedding: False }
  key: gru4rec
- class: STAMP.model.STAMP.Seq2SeqAttNN
  params: { init_lr: 0.003, n_epochs: 10, decay_rate: 0.4}
  key: stamp
- class: narm.narm.NARM
  params: { epochs: 20, lr: 0.007, hidden_units: 100, factors: 100 }
  key: narm
!git clone https://github.com/rn5l/session-rec.git
!cp -r $PREPARED_FOLDER /content/coveo/session-rec/data
!cp /content/coveo/config.yml /content/coveo/session-rec/conf
!pip install scikit-optimize
!pip install python-telegram-bot
%cd session-rec
!python run_config.py conf/example_sigir.yml
!mkdir -p /content/coveo/session-rec/results/last/sigir<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0971211118464593, 'fraction_numerical': 0.012739907346128392, 'mean_word_length': 4.891812865497076, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 18, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '175299', 'license': 'Apache-2.0', 'n_tokens_mistral': 2102, 'n_tokens_neox': 2020, 'n_words': 504, 'path': '_docs/nbs/T582311-Coveo-Baseline-Model.ipynb', 'repo_name': 'sparsh-ai/recohut', 'size': '6044'}