<commit_before>"""
DQN and its variants
------------------------
We implement Double DQN, Dueling DQN and Noisy DQN here.

The max operator in standard DQN uses the same values both to select and to
evaluate an action by
Q(s_t, a_t) = R_{t+1} + \gamma * max_{a}Q_{tar}(s_{t+1}, a).
Double DQN propose to use following evaluation to address overestimation problem
of max operator:
Q(s_t, a_t) = R_{t+1} + \gamma * Q_{tar}(s_{t+1}, max_{a}Q(s_{t+1}, a)).

Dueling DQN uses dueling architecture where the value of state and the advantage
of each action is estimated separately.

Noisy DQN propose to explore by adding parameter noises.


Reference:
------------------------
1. Double DQN
    <NAME>, <NAME>, <NAME>. Deep reinforcement learning with double
    q-learning[C]//Thirtieth AAAI Conference on Artificial Intelligence. 2016.
2. Dueling DQN
    <NAME>, <NAME>, <NAME>, et al. Dueling network architectures for deep
    reinforcement learning[J]. arXiv preprint arXiv:1511.06581, 2015.
3. Noisy DQN
    <NAME>, <NAME>, <NAME>, et al. Parameter space noise for
    exploration[J]. arXiv preprint arXiv:1706.01905, 2017.


Environment:
------------------------
Cartpole and Pong in OpenAI Gym


Requirements:
------------------------
tensorflow>=2.0.0a0
tensorlayer>=2.0.0


To run:
------------------------
python tutorial_DQN_variantes.py --mode=train
python tutorial_DQN_variantes.py --mode=test --save_path=c51/8000.npz
"""
import argparse<commit_msg>update annocation<commit_after>"""
DQN and its variants
------------------------
We implement Double DQN, Dueling DQN and Noisy DQN here.

The max operator in standard DQN uses the same values both to select and to
evaluate an action by
Q(s_t, a_t) = R_{t+1} + \gamma * max_{a}Q_{tar}(s_{t+1}, a).
Double DQN propose to use following evaluation to address overestimation problem
of max operator:
Q(s_t, a_t) = R_{t+1} + \gamma * Q_{tar}(s_{t+1}, max_{a}Q(s_{t+1}, a)).

Dueling DQN uses dueling architecture where the value of state and the advantage
of each action is estimated separately.

Noisy DQN propose to explore by adding parameter noises.


Reference:
------------------------
1. Double DQN
    <NAME>, <NAME>, <NAME>. Deep reinforcement learning with double
    q-learning[C]//Thirtieth AAAI Conference on Artificial Intelligence. 2016.
2. Dueling DQN
    <NAME>, <NAME>, <NAME>, et al. Dueling network architectures for deep
    reinforcement learning[J]. arXiv preprint arXiv:1511.06581, 2015.
3. Noisy DQN
    <NAME>, <NAME>, <NAME>, et al. Parameter space noise for
    exploration[J]. arXiv preprint arXiv:1706.01905, 2017.


Environment:
------------------------
Cartpole and Pong in OpenAI Gym


Requirements:
------------------------
tensorflow>=2.0.0a0
tensorlayer>=2.0.0


To run:
------------------------
python tutorial_DQN_variantes.py --mode=train
python tutorial_DQN_variantes.py --mode=test --save_path=dqn_variants/8000.npz
"""
import argparse
{'fraction_non_alphanumeric': 0.17753499487879823, 'fraction_numerical': 0.03414134516899966, 'mean_word_length': 2.9594594594594597, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5523762', 'n_tokens_mistral': 1016, 'n_tokens_neox': 899, 'n_words': 330}