<jupyter_start><jupyter_text>Hypothesis: Users interact most with the items which they afterwards click outThus, **interaction** with items correlates with them being clicked out. We can measure **interaction** in two way:1. The number of actions which reference an item.2. The time duration spent interacting with an item.  Action types with item interaction - the reference value is the item ID- `clickout item`: user makes a click-out on the item and gets forwarded to a partner website. Other items that were displayed to the user and their associated prices are listed under the ‘impressions’ and ‘prices’ column for this action.- `interaction item rating`: user interacts with a rating or review of an item.- `interaction item info`: user interacts with item information.- `interaction item image`: user interacts with an image of an item.- `interaction item deals`: user clicks on the view more deals button.- `search for item`: user searches for an accommodation.  Number of actions which reference the clicked out item<jupyter_code>action_types_hist = Counter(df['action_type'])
action_types_hist
interaction_action_types = {
    'clickout item',
    'interaction item rating',
    'interaction item info',
    'interaction item image',
    'interaction item deals',
    'search for item'
}
# all interaction actions have a reference to an item in the train data set

df_interactions = df[df['action_type'].isin(interaction_action_types)].set_index(keys=['session_id', 'step'])

print('Percentage of interaction actions: {}'.format(
    len(df_interactions) / len(df)
))
df_interactions.head(100)<jupyter_output><empty_output><jupyter_text>Research Questions for Item Interactions1. What percentage of the actions in a single session are interactions with the click out item? Compute this for each session and visualize it.2. Is the clickout item the most interacted with? I.e., what is the percentage of sessions whose clickout item is the most interacted with?3. In the sessions where the clickout item is not the most interacted with, what is the interaction rank of the clickout item? I.e. compute the interaction rank of the clickout item.    - where **interaction rank** is the rank of the item in the ranking of the items with which the user has interacted the most.<jupyter_code>def count_last_clickout_item_interactions(steps):
    clickout_actions = steps[steps['action_type'] == 'clickout item']
    last_clickout_action = clickout_actions[:-1]

    # last_clickout_action['reference'] is a potentially empty pandas Series
    last_clickout_action_refs = last_clickout_action['reference'].values
    last_clickout_action_ref = last_clickout_action_refs[0] if len(last_clickout_action_refs) > 0 else None
    
    if last_clickout_action_ref is None:
        return None;

    steps_with_clickout_ref = steps[steps['reference'] == last_clickout_action_ref]
    interaction_actions = steps_with_clickout_ref[steps_with_clickout_ref['action_type'].isin(interaction_action_types)]
    return len(interaction_actions)

# for each session
    # ref = get the reference of the last clickout item
    # find the count of actions where the reference is equal to ref
def compute_interaction_ranks(df_interactions):
    df_interaction_rank = df_interactions.groupby('session_id').apply(count_last_clickout_item_interactions)
    df_interaction_rank = df_interaction_rank.to_frame().rename(columns={0: 'interaction count with clickout item'})
    df_interaction_rank['total steps'] = df_interactions.groupby('session_id').apply(len)
    return df_interaction_rank
df_interaction_rank = compute_interaction_ranks(df_interactions)
df_interaction_rank = compute_interaction_ranks(df_interactions.head(1000000))
df_interaction_rank
%time<jupyter_output>CPU times: user 4 µs, sys: 0 ns, total: 4 µs
Wall time: 6.91 µs
CPU times: user 2 µs, sys: 0 ns, total: 2 µs
Wall time: 4.05 µs<jupyter_text>It seems that we should split the session into subsessions where each subsession has a single clickout item eventTODO: To test this, add the number of clickout item actions to the `df_interaction_rank` and see if the `percentage` you compute below is inversely correlated with a high number of clickout items per session<jupyter_code>df_interaction_rank = df_interaction_rank.dropna()
df_interaction_rank['percentage'] = df_interaction_rank['interaction count with clickout item'] / df_interaction_rank['total steps']
df_interaction_rank
stats = (df_interaction_rank['percentage'] * 100).value_counts().sort_index().to_frame().reset_index().rename(columns={'index': 'percentage', 'percentage': 'count'})
stats
bins = np.linspace(0, 100, 21)
groups = stats.groupby(pd.cut(stats['percentage'], bins)).sum()
groups.drop(columns='percentage')
df_clickout_item_interactions_proportions = groups.drop(columns='percentage')
df_clickout_item_interactions_proportions
plt.figure(2, figsize=(20,15))
sns.barplot(
    x='percentage',
    y='count',
    data=df_clickout_item_interactions_proportions.reset_index(),
    palette='Spectral'
).set(
    xlabel='percentage of interactions with the clickout item from all actions',
    ylabel='number of sessions'
)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08153013910355487, 'fraction_numerical': 0.008693972179289026, 'mean_word_length': 5.86604774535809, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '421626', 'license': 'MIT', 'n_tokens_mistral': 1542, 'n_tokens_neox': 1474, 'n_words': 517, 'path': 'notebooks/03-interaction-rank.ipynb', 'repo_name': 'antonpetkoff/recsys-2019', 'size': '5176'}