<jupyter_start><jupyter_text>Instalando e importando as bibliotecas necessárias<jupyter_code>!pip install -q -r 'requirements.txt'
import numpy as np
import gym

from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Activation
from keras.layers import Flatten
from keras.optimizers import Adam

from rl.agents.dqn import DQNAgent
from rl.policy import BoltzmannQPolicy, EpsGreedyQPolicy, LinearAnnealedPolicy
from rl.memory import SequentialMemory
from rl.callbacks import FileLogger, ModelIntervalCheckpoint<jupyter_output><empty_output><jupyter_text>Criando o ambiente e extraindo o número de inputs<jupyter_code>env = gym.make('CartPole-v0')
np.random.seed(123)
env.seed(123)
nb_actions = env.action_space.n
print('Número de inputs: {}'.format(nb_actions))<jupyter_output>Número de inputs: 2<jupyter_text>Testando o ambiente com inputs randômicos<jupyter_code>episodes = 10
for episode in range(1, episodes+1):
    state = env.reset()
    done = False
    score = 0 
    
    while not done:
        # env.render()
        action = env.action_space.sample()
        n_state, reward, done, info = env.step(action)
        score+=reward
    print('Episódio: {} Score: {}'.format(episode, score))
env.close()<jupyter_output>Episódio: 1 Score: 26.0
Episódio: 2 Score: 16.0
Episódio: 3 Score: 32.0
Episódio: 4 Score: 28.0
Episódio: 5 Score: 14.0
Episódio: 6 Score: 29.0
Episódio: 7 Score: 45.0
Episódio: 8 Score: 16.0
Episódio: 9 Score: 17.0
Episódio: 10 Score: 23.0<jupyter_text>Construindo a model<jupyter_code>model = Sequential()
model.add(Flatten(input_shape=(1,) + env.observation_space.shape))
model.add(Dense(units = 128, activation = 'relu'))
model.add(Dense(units = 64, activation = 'relu'))
model.add(Dense(units = 32, activation = 'relu'))
model.add(Dense(units = nb_actions, activation = 'linear'))
print(model.summary())<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten (Flatten)            (None, 4)                 0         
_________________________________________________________________
dense (Dense)                (None, 128)               640       
_________________________________________________________________
dense_1 (Dense)              (None, 64)                8256      
_________________________________________________________________
dense_2 (Dense)              (None, 32)                2080      
_________________________________________________________________
dense_3 (Dense)              (None, 2)                 66        
=================================================================
Total params: 11,042
Trainable params: 11,042
Non-trainable params: 0
____________________________________________________[...]<jupyter_text>Configurando e compilando o Agent  Parâmetros:    - memory: Estrutura de dados para o armazenamento das experiências do agente. É necessário especificar o tamanho máximo para esse objeto de memória, que é um hiperparâmetro. Quando novas experiências são adicionadas à essa memória e ela se enche, experiências passadas começam a ser esquecidas.    - policy: Usada para o balanceamento da exploration e exploitation.    - agent: Com o modelo, a memória e a policy definidas, podemos criar o agente e atribuir a esse agente os parâmetros definidos.<jupyter_code>memory = SequentialMemory(limit=50000, window_length=1)
policy = EpsGreedyQPolicy()
dqn = DQNAgent(model=model, nb_actions=nb_actions, memory=memory, nb_steps_warmup=10,
               target_model_update=1e-2, policy=policy)
dqn.compile(Adam(lr=1e-4), metrics=['mae'])
dqn.fit(env, nb_steps=10000, visualize=False, verbose=2, nb_max_start_steps=750)<jupyter_output>Training for 10000 steps ...<jupyter_text>Avaliação do treinamento em 20 episódios<jupyter_code>dqn.test(env, nb_episodes=20, visualize=False)<jupyter_output>Testing for 20 episodes ...
Episode 1: reward: 200.000, steps: 200
Episode 2: reward: 200.000, steps: 200
Episode 3: reward: 200.000, steps: 200
Episode 4: reward: 200.000, steps: 200
Episode 5: reward: 200.000, steps: 200
Episode 6: reward: 200.000, steps: 200
Episode 7: reward: 200.000, steps: 200
Episode 8: reward: 200.000, steps: 200
Episode 9: reward: 200.000, steps: 200
Episode 10: reward: 200.000, steps: 200
Episode 11: reward: 200.000, steps: 200
Episode 12: reward: 200.000, steps: 200
Episode 13: reward: 200.000, steps: 200
Episode 14: reward: 200.000, steps: 200
Episode 15: reward: 200.000, steps: 200
Episode 16: reward: 200.000, steps: 200
Episode 17: reward: 200.000, steps: 200
Episode 18: reward: 200.000, steps: 200
Episode 19: reward: 200.000, steps: 200
Episode 20: reward: 200.000, steps: 200
{'fraction_non_alphanumeric': 0.11940919492406907, 'fraction_numerical': 0.07114624505928854, 'mean_word_length': 4.5842044134727065, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 20, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '208151', 'license': 'MIT', 'n_tokens_mistral': 1824, 'n_tokens_neox': 1513, 'n_words': 443, 'path': 'dqn.ipynb', 'repo_name': 'deeplearningunb/auto-pong', 'size': '4807'}