<jupyter_start><jupyter_text>* Download the jester dataset with joke preferences from S3.<jupyter_code>URL = 'https://raw.githubusercontent.com/PacktPublishing/Machine-Learning-for-Time-Series-with-Python/main/chapter11/jesterfinal151cols.csv'<jupyter_output><empty_output><jupyter_text>* Download them using pandas.<jupyter_code>import pandas as pd
jester_data = pd.read_csv(URL, header=None)<jupyter_output><empty_output><jupyter_text>* Make some cosmetic adjustments. * The rows refer to users, the columns to jokes.<jupyter_code># jester_data.columns = [f"joke_{col}" for col in jester_data.columns]
jester_data.index.name = "users"
jester_data.head()<jupyter_output><empty_output><jupyter_text>* The encoding of choices is a bit weird, so fix this as well.<jupyter_code>for col in jester_data.columns:
    jester_data[col] = jester_data[col].apply(lambda x: 0.0 if x>=99 or x<7.0 else 1.0)
jester_data<jupyter_output><empty_output><jupyter_text>* Get rid of people who didn't choose any joke at all.<jupyter_code># keep users with at least one rating for a joke
jester_data = jester_data[jester_data.sum(axis=1) > 0]
jester_data
pip install git+https://github.com/benman1/mab-ranking<jupyter_output>Collecting git+https://github.com/benman1/mab-ranking
  Cloning https://github.com/benman1/mab-ranking to /tmp/pip-req-build-ygvaa_k1
  Running command git clone -q https://github.com/benman1/mab-ranking /tmp/pip-req-build-ygvaa_k1
Collecting flake8
  Downloading flake8-4.0.1-py2.py3-none-any.whl (64 kB)
[K     |████████████████████████████████| 64 kB 2.2 MB/s 
[?25hRequirement already satisfied: jupyter in /usr/local/lib/python3.7/dist-packages (from mab-ranking==0.0.1) (1.0.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from mab-ranking==0.0.1) (3.2.2)
Collecting mock
  Downloading mock-4.0.3-py3-none-any.whl (28 kB)
Requirement already satisfied: pluggy in /usr/local/lib/python3.7/dist-packages (from mab-ranking==0.0.1) (0.7.1)
Requirement already satisfied: pytest in /usr/local/lib/python3.7/dist-packages (from mab-ranking==0.0.1) (3.6.4)
Collecting pytest-cov
  Downloading pytest_cov-3.0.0-py3-none-any.whl (20 kB)
Collecting tox
  Downloa[...]<jupyter_text>* Set up bandits as follows.* Choose independent bandits with Thompson sampling from the Betadistribution. Recommend the best 10 jokes.<jupyter_code># setting up the bandits:
from mab_ranking.bandits.rank_bandits import IndependentBandits
from mab_ranking.bandits.bandits import BetaThompsonSampling, DirichletThompsonSampling

independent_bandits = IndependentBandits(
    num_arms=jester_data.shape[1],
    num_ranks=10,
    bandit_class=DirichletThompsonSampling
)<jupyter_output><empty_output><jupyter_text>* Start the simulation.* Get feedback on the 10 jokes that display as chosen by our independentbandits.* Simulate 7,000 iterations (visits). At each visit, change our choicesaccording to the updated reward expectations.<jupyter_code>from tqdm import trange

num_steps = 7000
hit_rates = []
for _ in trange(1, num_steps + 1):
    selected_items = set(independent_bandits.choose())
    # Pick a users choices at random
    random_user = jester_data.sample().iloc[0, :]
    ground_truth = set(random_user[random_user == 1].index)
    hit_rate = len(ground_truth.intersection(selected_items)) / len(ground_truth)
    feedback_list = [1.0 if item in ground_truth else 0.0 for item in selected_items]
    independent_bandits.update(selected_items, feedback_list)
    hit_rates.append(hit_rate)<jupyter_output>100%|██████████| 7000/7000 [01:23<00:00, 83.66it/s]<jupyter_text>* Plot the hit rate, the jokes that users are selecting, as follows.* Introduce a rolling average (over 200 iterations) to get a smoother graph.<jupyter_code>import matplotlib.pyplot as plt

stats = pd.Series(hit_rates)
plt.figure(figsize=(12, 6))
plt.plot(stats.index, stats.rolling(200).mean(), "--")
plt.xlabel('Iteration')
plt.ylabel('Hit rate')<jupyter_output><empty_output><jupyter_text>* Supply the categorical user group information, the context.* Cluster users by their preferences, and use the clusters as context.* This creates 5 user groups.<jupyter_code>from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler().fit(jester_data)
kmeans = KMeans(n_clusters=5, random_state=0).fit(scaler.transform(jester_data))
contexts = pd.Series(kmeans.labels_, index=jester_data.index)
contexts.value_counts()<jupyter_output><empty_output><jupyter_text>* Reset our bandits.* Then, redo the simulation. Only now, supply the user context.<jupyter_code>independent_bandits = IndependentBandits(
    num_arms=jester_data.shape[1],
    num_ranks=10,
    bandit_class=DirichletThompsonSampling
)

num_steps = 7000
hit_rates = []
for _ in trange(1, num_steps + 1):
    # Pick a users choices at random
    random_user = jester_data.sample().iloc[0, :]
    context = {"previous_action": contexts.loc[random_user.name]}
    selected_items = set(independent_bandits.choose(
        context=context
    ))
    ground_truth = set(random_user[random_user == 1].index)
    hit_rate = len(ground_truth.intersection(selected_items)) / len(ground_truth)
    feedback_list = [1.0 if item in ground_truth else 0.0 for item in selected_items]
    independent_bandits.update(selected_items, feedback_list, context=context)
    hit_rates.append(hit_rate)<jupyter_output>100%|██████████| 7000/7000 [01:24<00:00, 82.89it/s]<jupyter_text>* Visualize the hit rate over time again.<jupyter_code>import matplotlib.pyplot as plt

stats = pd.Series(hit_rates)
plt.figure(figsize=(12, 6))
plt.plot(stats.index, stats.rolling(200).mean(), "--")
plt.xlabel('Iteration')
plt.ylabel('Hit rate')<jupyter_output><empty_output><jupyter_text>* See that the hit rate is a bit higher than before.* This model ignores the order of the recommended jokes on hypothetical website.<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1205123883364234, 'fraction_numerical': 0.030844429462329345, 'mean_word_length': 5.773972602739726, 'pattern_counts': {'":': 1, '<': 49, '<?xml version=': 0, '>': 48, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '530694', 'license': 'MIT', 'n_tokens_mistral': 2231, 'n_tokens_neox': 2051, 'n_words': 468, 'path': 'Reinforcement Learning for TS/Recommendations.ipynb', 'repo_name': 'thirasit/Time-series-analysis-with-ML', 'size': '5933'}