<jupyter_start><jupyter_text><a href="https://colab.research.google.com/github/christianhidber/easyagents/blob/master/jupyter_notebooks/intro_cartpole.ipynb"    target="_parent">     CartPole Gym environment with TfAgents  Install packages (gym, tfagents, tensorflow,....)  suppress package warnings, prepare matplotlib, if in colab: load additional packages for rendering<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import sys
import warnings
warnings.filterwarnings('ignore')

if 'google.colab' in sys.modules:
    !apt-get update >/dev/null
    !apt-get install xvfb >/dev/null
    !pip install pyvirtualdisplay >/dev/null    
    
    from pyvirtualdisplay import Display
    Display(visible=0, size=(960, 720)).start()    
else:
    #  for local installation
    sys.path.append('..')<jupyter_output><empty_output><jupyter_text>install easyagents<jupyter_code>import sys
if 'google.colab' in sys.modules:
    !pip install easyagents >/dev/null<jupyter_output><empty_output><jupyter_text>The fc_layers argument defines the policy's neural network architecture. Here we use 3 fully connected layerswith 100 neurons in the first, 50 in the second and 25 in the final layer. By default fc_layers=(75,75) is used.The first argument of the train method is a list of callbacks. Through callbacks we define the plots generated during training, the logging behaviour or control training duration. By passing [plot.State(), plot.Loss(), plot.Actions(), plot.Rewards()] we add in particular the State() plot, depicting the last observation state of the last evaluation episode. plot.Actions() displays a histogram of the actions taken for each episode played during the last evaluation period. Besides num_iterations there are quite a few parameters to specify the exact training duration (e.g. num_episodes_per_iteration, num_epochs_per_iteration, max_steps_per_episode,...).  Switching the algorithmSwitching from Ppo to Dqn is easy, essentially just replace PpoAgent with DqnAgent (the evaluation may take a fewminuites):<jupyter_code>from easyagents.agents import DqnAgent
from easyagents.callbacks import plot
%%time

dqnAgent = DqnAgent('CartPole-v0', fc_layers=(100, ))
dqnAgent.train([plot.State(), plot.Loss(), plot.Actions(), plot.Rewards()], 
               num_iterations=20000, num_iterations_between_eval=1000)<jupyter_output>Wall time: 4min 50s
{'fraction_non_alphanumeric': 0.08680994521702487, 'fraction_numerical': 0.014749262536873156, 'mean_word_length': 5.330666666666667, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 17, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '12242', 'license': 'MIT', 'n_tokens_mistral': 740, 'n_tokens_neox': 685, 'n_words': 234, 'path': 'jupyter_notebooks/scratch/intro_cartpole.ipynb', 'repo_name': 'doMConSwiss/easyagents', 'size': '2373'}