<jupyter_start><jupyter_text>Homework 3 Dates and directions* **Due 06 March 2020*** **Description** This notebook introduces use of `ray` and `rllib` and explores `Pong-v0` (an atari game). The *goal* is to train policies for pong using PPO and Q-learning and explore how choices effect performance (both of agent and training). Other notes and snippets will likely be added as time goes on.  References1. https://ray.readthedocs.io/en/latest/auto_examples/plot_pong_example.html1. https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html1. https://gist.github.com/karpathy/a4166c7fe253700972fcbc77e4ea32c51. https://kyso.io/eoin/openai-gym-jupyter?utm_campaign=News&utm_medium=Community&utm_source=DataCamp.com Additional configuration1. Install ffmpeg `apt install ffmpeg` *(hoping to avoid this with using cluster resource)*  Potentially helpful snippets<jupyter_code>import gym
from gym import wrappers

env = gym.make('Pong-v0')
env = wrappers.Monitor(env, "./gym-results", force=True)
env.reset()

for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
    if done: break
env.close()
import io
import base64
from IPython.display import HTML

video = io.open('./gym-results/openaigym.video.%s.video000000.mp4' % env.file_infix, 'r+b').read()
encoded = base64.b64encode(video)
HTML(data='''
    <video width="360" height="auto" alt="test" controls><source src="data:video/mp4;base64,{0}" type="video/mp4" /></video>'''
.format(encoded.decode('ascii')))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13107416879795397, 'fraction_numerical': 0.037723785166240406, 'mean_word_length': 5.774891774891775, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '179011', 'license': 'CC0-1.0', 'n_tokens_mistral': 569, 'n_tokens_neox': 522, 'n_words': 132, 'path': 'homework/3/RL for Atari.ipynb', 'repo_name': 'cianhwang/ece590hineman', 'size': '1564'}