<jupyter_start><jupyter_text>______Copyright by <NAME> Inc.For more information, visit us at www.pieriandata.com  Open AI Gym BasicsLet's explore the basics of a **gym** environment, later on we wil interact with it and create agents to learn from it. In this notebook we will get to know the open ai gym library and its environments:https://gym.openai.com/It contains various implementations of tasks on which you can try Reinforcement Learning agents. At first we will import the necessary environments<jupyter_code>import time  # To slown down the game rendering to human speed
import gym
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Now we can go and chose an environment we want to try and where we want to train an agent on from here:https://gym.openai.com/envs/Let's say we want to play the **breakout game** https://gym.openai.com/envs/Breakout-v0/, then we need to perform the following two steps:1. Define the name of the environment2. Create the environment<jupyter_code>env_name = "Breakout-v0"  # Use the exact same name as stated on gym.openai
env = gym.make(env_name)  # use gym.make to create your environment<jupyter_output><empty_output><jupyter_text>Thats all! You successfully created the environment!Those steps are identical for all environments.If you want to take a look at your game, gym comes with a handy utils function called **play** for Atari games.Note that you start the game with *space* and move with *a* and *d*How many points can you score?<jupyter_code>from gym.utils import play  # import the play module
play.play(env, zoom=2)  # call the play function<jupyter_output><empty_output><jupyter_text>Now after getting to know how we can select an environment and interact with it, it is time to dive deeper into the world of gym.Gym offers us many functions to interact with the created environment. The most important are:* reset() - which resets the state of the environment to the initial state. I.e it restarts the game * step(action) - performs an action on your environment. In our case it either starts the game or moves to the left, right, or just remains at the current position. It returns the current state of the environment, the reward, if the game is done and some debugging information* render(mode) - Renders your environment and displays it on your monitorWe will get to know those functions in the next few cells First we will take a look at the **render** function:render(mode) takes one argument, namely the render *mode*. *mode* can take two possible values:* "human"* "rgb_array""human" directly displays the game on your screen, while "rgb_array" returns the array containing the state of the current environment aka the image render shows us.<jupyter_code>for _ in range(200):  # Run the environment for 200 steps
    env.render(mode="human")  # directly displays you the current state of the game
env.close()  # as render always opens a new window when called, it is important to close it again
array = env.render(mode="rgb_array")  # returns the image as a 2d numpy array
print(array[60][50])  # display the colour of some pixel (row 60 and column 50 - so the red blocks)

plt.imshow(array)  # you can use matplotlib to take a look at your environment - it is the same image as above
array<jupyter_output><empty_output><jupyter_text>Note that even we ran 1000 steps over the environment, nothing happened! This is because no action was taken on this environment.This is the point where the **step** function joins our game. It performs a selected action on our environment, e.g moving to the left.You can take a look at the possible actions via the **action_space.n** property.It returns 4 for our game which is in line with our above findings that we can either start the game, move left or right, or stand still.<jupyter_code>print(f"There are {env.action_space.n} possible actions")<jupyter_output>There are 4 possible actions<jupyter_text>Now it is time to watch some action happening. For this we can select a random possible action.We could do this by using **numpy.random.randint(0, 4)** or by using the provided sample function **env.action_space.sample()** which directly returns you a possible action.Note how the "ale.lives" info reduces for each live you lose<jupyter_code>for _ in range(200):
    env.render(mode="human")  # display the current state
    random_action = env.action_space.sample()  # get the random action
    observation, reward, done, info = env.step(random_action) # perform the action the current state of the environment
    print(f"Reward: {reward}, Done: {done}, Info: {info}")
    time.sleep(0.01)  # slow down the game a bit
env.close()  # dont forget to close the environment<jupyter_output>Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: F[...]<jupyter_text>If we would now again render the environment, we continue at the same stage we left - thus to start again we need to call the **reset** function.The reset function returns the initial state of the environment.<jupyter_code>env.render()  # Old state of the game
env.close()
state = env.reset()<jupyter_output><empty_output><jupyter_text>Now we can again take a look at the game<jupyter_code>for _ in range(1000):
    env.render(mode="human")  # display the current state
    random_action = env.action_space.sample()  # get the random action
    observation, reward, done, info = env.step(random_action) # perform the action the current state of the environment
    print(f"Reward: {reward}, Done: {done}, Info: {info}")
    if done:  # with the done variable we can leave the loop if gym notifies us that the game is over
        break
    time.sleep(0.01)  # slow down the game a bit
env.close()  # dont forget to close the environment<jupyter_output>Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: False, Info: {'ale.lives': 5}
Reward: 0.0, Done: F[...]<jupyter_text>As you might have noticed, there are always two versions of an environment:1. RAM Version2. Normal Version / Image based versionThose version differ in the observation which gym returns to you. The Ram Version only returns some properties (like the coordinates of the ball), while the normal version returns images (i.e exact the image that render() shows us).<jupyter_code>env_name_ram = "Breakout-ram-v0"  # use the RAM version
env_ram = gym.make(env_name_ram)  # create the RAM version of the environment
env_ram.render(mode="human*")
env_ram.close()
env_ram.reset().shape # you can use the reset function to inspect the initial state of your environment.
                      # There are only 128 values instead of the whole image 
env.reset().shape  # Our original environment returns an image as observation<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11174751607247224, 'fraction_numerical': 0.019988310929281123, 'mean_word_length': 4.628947368421053, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '348382', 'license': 'MIT', 'n_tokens_mistral': 2824, 'n_tokens_neox': 2661, 'n_words': 1173, 'path': 'practical_ai/archive/05-Introduction-to-gym/01-OpenAi-Gym-Basics.ipynb', 'repo_name': 'Anshul-GH/machine_learning_advanced', 'size': '8561'}