<jupyter_start><jupyter_text>Pong - Reinforcement LearningIf you run this notebook on a remote AWS EC2 GPU server you first need to install the xvfb, mesa, and egl libraries. Assuming the server is running Ubuntu 18.04 you can do this with:    sudo apt-get update -y      sudo apt-get install -y mesa-utils libegl1-mesa xvfb freeglut3-devThen start a screen session with the command:    screenThis will prevent the notebook from exiting if your ssh session disconnects. Once in the screen session start your notebook with:    jupyter notebook --no-browser --ip=On an EC2 server running Ubuntu 18.04 with updated nvidia drivers the notebook should run without problems. However on older OS or with outdated drivers you could run into plenty of trouble related to "headless" rendering of opengl.<jupyter_code>import sys
import os
p0 = os.path.abspath('.')
p1 = os.path.abspath(os.path.join(p0, '..'))

sys.path.insert(0, p1)
import jupylet.rl
import PIL.Image

import numpy as np

import matplotlib.pyplot as plt
%matplotlib inline
def show_images(iml, flip=True, columns=4, figsize=(17, 8)):
    
    plt.figure(figsize=figsize)
    for i, image in enumerate(iml):
        
        if flip and isinstance(image, np.ndarray):
            image = image[::-1]
        else:
            image = image.transform(PIL.Image.FLIP_TOP_BOTTOM)
        
        plt.subplot(len(iml) // columns + 1, columns, i + 1)
        plt.axis('off')
        plt.imshow(image)
        

def show_image(image, flip=True):
    
    if isinstance(image, np.ndarray):
        image = PIL.Image.fromarray(image)
        
    if flip:
        image = image.transpose(PIL.Image.FLIP_TOP_BOTTOM)
        
    return image
os.getpid()
pong = jupylet.rl.GameProcess('pong')
%time pong.start()
#pong.call('save', 'pong-start.state')
%timeit -n100 _ = pong.step(n=1)
len(pong.step()['screen0'])
show_image(pong.observe())
games = jupylet.rl.Games(['pong'] * 8)
games.start()
show_images(games.observe())
player0 = np.random.randint(0, 2, (len(games.games), 5))
player0
player1 = np.random.randint(0, 2, (len(games.games), 5))
player1
%timeit -n200 games.step(player0, player1)
sl = games.step(player0, player1)
al = [s.pop('screen0') for s in sl]
sl
show_images(al)
show_images(games.reset())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11504811898512686, 'fraction_numerical': 0.02099737532808399, 'mean_word_length': 3.5831663326653307, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '370556', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 820, 'n_tokens_neox': 775, 'n_words': 230, 'path': 'examples/22-pong-RL.ipynb', 'repo_name': 'ofer1992/jupylet', 'size': '2286'}