<commit_before>Deep Reinforcement Learning with PyChrono and Tensorflow {#tutorial_pychrono_demo_tensorflow}
==========================

# Introduction
Deep Reinforcement Learning (DRL) consits in using Reinforcement Learning to train Deep Neural Network. In the last few years it has been applied with success to various robotic control tasks. 
The main advantage of this approach is its ability to deal with unstructured and mutable environments, while classical robotic control often fails when facing these challenges.
To train a NN with DRL several interactions with the environment are needed. For this reason physical engines offer a valuable help, allowing to train the agent in a virtual environment instead of training it directly in the real world, reducing time and risks of the operation.
With PyChrono you can easily build physical models and exchange data between the simulation and you favorite ML framework. 
We suggest two ways to get started using PyChrono for DRL:

# 1- [gym-chrono](https://github.com/projectchrono/gym-chrono)
 
If you are looking for more complex and realistic environments and you want to leverage OpenAI Gym capabilities (i.e. using [OpenAI Baselines](https://github.com/openai/baselines)) we recommend to use gym-chrono, a set of PyChrono-based OpenAI Gym environments. 
These environments provide an open-source alternative to MuJoCo environments.
# 2- Tensorflow Demos
We also provide 2 simple plug-and-play examples to kickstart you into DRL for robotic control. These demos only require Tensorflow and PyChrono to run; and are provided together with a standalon Proximal Policy optimization algorithm.

[PYTHON CODES HERE](https://github.com/projectchrono/chrono/tree/develop/src/demos/python/chrono-tensorflow)
### Goal:
We will train a Neural Network to solve robotic tasks using virtual training environment created with [PyChrono](@ref pychrono_introduction).
The demo contains virtual environments and a learning model created with Tensorflow. 

### Environments
We provide two sample environments for robotic control. 

**ChronoPendulum** 
![](http://projectchrono.org/assets/manual/Tutorial_tensorflow_pendulum.jpg)

Reverse pendulum, the goal is to balance a pole on a cart.  1 action (force along the z axis) and 4 observations (position and speed of cart and pole).

**ChronoAnt** 
![](http://projectchrono.org/assets/manual/Tutorial_tensorflow_ant.jpg)<commit_msg>Updated tensorflow demo documentation<commit_after>Deep Reinforcement Learning with PyChrono and Tensorflow {#tutorial_pychrono_demo_tensorflow}
==========================

# Introduction
Deep Reinforcement Learning (DRL) consits in using Reinforcement Learning to train Deep Neural Network. In the last few years it has been applied with success to various robotic control tasks. 
The main advantage of this approach is its ability to deal with unstructured and mutable environments, while classical robotic control often fails when facing these challenges.
To train a NN with DRL several interactions with the environment are needed. For this reason physical engines offer a valuable help, allowing to train the agent in a virtual environment instead of training it directly in the real world, reducing time and risks of the operation.
With PyChrono you can easily build physical models and exchange data between the simulation and you favorite ML framework. 
We suggest two ways to get started using PyChrono for DRL:

# 1- [gym-chrono](https://github.com/projectchrono/gym-chrono)
 
If you are looking for more complex and realistic environments and you want to leverage OpenAI Gym capabilities (i.e. using [OpenAI Baselines](https://github.com/openai/baselines)) we recommend to use gym-chrono, a set of PyChrono-based OpenAI Gym environments. 
These environments provide an open-source alternative to MuJoCo environments.
# 2- Tensorflow Demos
We also provide 2 simple plug-and-play examples to kickstart you into DRL for robotic control. These demos only require Tensorflow and PyChrono to run; and are provided together with a standalon Proximal Policy optimization algorithm.

[PYTHON CODES HERE](https://github.com/projectchrono/chrono/tree/develop/src/demos/python/chrono-tensorflow)

### Requirements:
In order to run these demos, you will need:
* Numpy
* Scikit-Learn
* Tensorflow 1

We recommend to use [Anaconda](https://www.anaconda.com/). To prepare your environment do the following:
* [Download](https://www.anaconda.com/distribution/) and install Anaconda. We suggest user-only installation and to set Anaconda as your default Python.
* Create a new Python [environment](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html). If you've built PyChrono from sources, set the Python version of your environment as the version of the Python executable you pointed to in CMake. You could also use the executable, headers and libraries of the virtual environment to build PyChrono.
* Activate the new environment 
* If you have ***not*** built from sources, install pychrono using Anaconda 
    ~~~~~~~~~~~~~{.sh}
     conda install -c projectchrono pychrono 
    ~~~~~~~~~~~~~
* Install Numpy
     ~~~~~~~~~~~~~{.sh}
     conda install numpy
    ~~~~~~~~~~~~~
* Install Scikit-Learn
    ~~~~~~~~~~~~~{.sh}
    conda install scikit-learn 
    ~~~~~~~~~~~~~
* Install Tensorflow
    ~~~~~~~~~~~~~{.sh}
    conda install tensorflow-gpu=1.14
    ~~~~~~~~~~~~~
### Goal:
We will train a Neural Network to solve robotic tasks using virtual training environment created with [PyChrono](@ref pychrono_introduction).
The demo contains virtual environments and a learning model created with Tensorflow. 

### Environments
We provide two sample environments for robotic control. 

**ChronoPendulum** 
![](http://projectchrono.org/assets/manual/Tutorial_tensorflow_pendulum.jpg)

Reverse pendulum, the goal is to balance a pole on a cart.  1 action (force along the z axis) and 4 observations (position and speed of cart and pole).

**ChronoAnt** 
![](http://projectchrono.org/assets/manual/Tutorial_tensorflow_ant.jpg)
{'fraction_non_alphanumeric': 0.08470394736842106, 'fraction_numerical': 0.002302631578947368, 'mean_word_length': 4.991133004926108, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 9, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2494113', 'n_tokens_mistral': 1629, 'n_tokens_neox': 1537, 'n_words': 731}