<jupyter_start><jupyter_text>Assignment 1  Problem 1(a)  <NAME>  McGill ID: 260800927  COMP 767- Reinforcement Learning Winter 2019  (a) summarize the main results in the paper [1][1] link: https://people.eecs.berkeley.edu/˜kjamieson/resources/bestArmSurvey.pdf  Comparison of sampling procedure : * Similarity : Sample complexity of Action Elimination, UCB, LUCB algorithms are very similar * Differences : For Action Elimination, it's hard to get the best arm until we do many pulls around figure 1 requires around 30 pulls in paper[1] . But for UBC and LUCB we can identify best arm very quickly as it gives large separation in first and second arm For different parameter result will be different but will hold the same qualitive shape of the curve. The sample complexity derived for LUCB is the best result for sampling strategy  (b) Reproducing the results in Figure 1 of [1]<jupyter_code>from IPython.display import SVG
"""
arguments

run = 'Reproduce_Paper_Results' or 'Book_Example_10arm_Bandit'
Sampling_Strategy = 'Action_Elimination' or 'UCB'

"""

!python Comparison_Sampling_Strategies.py --run='Reproduce_Paper_Results' --Sampling_Strategy='Action_Elimination' --n_trials=100 --n_time_steps=2000
def show_svg():
    return SVG('Action_Elimination_Reproduce_Paper_Results.svg')
show_svg()
!python Comparison_Sampling_Strategies.py --run='Reproduce_Paper_Results' --Sampling_Strategy='UCB' --n_trials=40 --n_time_steps=2000
def show_svg():
    return SVG('UCB_Reproduce_Paper_Results.svg')
show_svg()<jupyter_output><empty_output><jupyter_text>(c) Empirical comparison on the bandit problem provided in the Sutton & Barto book<jupyter_code>!python Comparison_Sampling_Strategies.py --run='Book_Example_10arm_Bandit' --Sampling_Strategy='Action_Elimination' --n_trials=100 --n_time_steps=4000

def show_svg():
    return SVG('Action_Elimination_Book_Example_10arm_Bandit.svg')
show_svg()
!python Comparison_Sampling_Strategies.py --run='Book_Example_10arm_Bandit' --Sampling_Strategy='UCB' --n_trials=40 --n_time_steps=4000
def show_svg():
    return SVG('UCB_Book_Example_10arm_Bandit.svg')
show_svg()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08908582089552239, 'fraction_numerical': 0.028917910447761194, 'mean_word_length': 6.526315789473684, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '635322', 'license': 'MIT', 'n_tokens_mistral': 744, 'n_tokens_neox': 688, 'n_words': 194, 'path': 'Assignment01/Problem01/Assignment1_Problem1a.ipynb', 'repo_name': 'SaminYeasar/COMP-767-Reinforcement-Learniing', 'size': '2156'}