<commit_before>---
layout: post
comments: true
title: "Policy Gradient Algorithms"
date: 2018-04-08 00:15:06
tags: reinforcement-learning long-read
image: "A3C_vs_A2C.png"
---

> Abstract: In this post, we are going to look deep into policy gradient, why it works, and many new policy gradient algorithms proposed in recent years: vanilla policy gradient, actor-critic, off-policy actor-critic, A3C, A2C, DPG, DDPG, D4PG, MADDPG, TRPO, PPO, ACER, ACTKR, SAC and TD3.


<!--more-->

<span style="color: #286ee0;">[Updated on 2018-06-30: add two new policy gradient methods, [SAC](#sac) and [D4PG](#d4pg).]</span>
<br/>
<span style="color: #286ee0;">[Updated on 2018-09-30: add an new policy gradient method, [TD3](#td3).]</span>
<br/>
<span style="color: #286ee0;">[Updated on 2019-02-09: add [SAC with automatically adjusted temperature](#sac-with-automatically-adjusted-temperature)].</span>
<br/>
<span style="color: #286ee0;">[Updated on 2019-05-01: Thanks to Wenhao, we have a version of this post in [Chinese](https://tomaxent.com/2019/04/14/%E7%AD%96%E7%95%A5%E6%A2%AF%E5%BA%A6%E6%96%B9%E6%B3%95/)].</span>

{: class="table-of-content"}
* TOC
{:toc}


## What is Policy Gradient<commit_msg>add a link to PG post in Korean<commit_after>---
layout: post
comments: true
title: "Policy Gradient Algorithms"
date: 2018-04-08 00:15:06
tags: reinforcement-learning long-read
image: "A3C_vs_A2C.png"
---

> Abstract: In this post, we are going to look deep into policy gradient, why it works, and many new policy gradient algorithms proposed in recent years: vanilla policy gradient, actor-critic, off-policy actor-critic, A3C, A2C, DPG, DDPG, D4PG, MADDPG, TRPO, PPO, ACER, ACTKR, SAC and TD3.


<!--more-->

<span style="color: #286ee0;">[Updated on 2018-06-30: add two new policy gradient methods, [SAC](#sac) and [D4PG](#d4pg).]</span>
<br/>
<span style="color: #286ee0;">[Updated on 2018-09-30: add an new policy gradient method, [TD3](#td3).]</span>
<br/>
<span style="color: #286ee0;">[Updated on 2019-02-09: add [SAC with automatically adjusted temperature](#sac-with-automatically-adjusted-temperature)].</span>
<br/>
<span style="color: #286ee0;">[Updated on 2019-05-01: Thanks to Wenhao, we have a version of this post in [Chinese](https://tomaxent.com/2019/04/14/%E7%AD%96%E7%95%A5%E6%A2%AF%E5%BA%A6%E6%96%B9%E6%B3%95/)].</span>
<br/>
<span style="color: #286ee0;">[Updated on 2019-06-26: Thanks to Chanseok, we have a version of this post in [Korean](https://talkingaboutme.tistory.com/entry/RL-Policy-Gradient-Algorithms)].</span>

{: class="table-of-content"}
* TOC
{:toc}


## What is Policy Gradient
{'fraction_non_alphanumeric': 0.1702371843917368, 'fraction_numerical': 0.08033664881407804, 'mean_word_length': 5.226190476190476, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 32, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4324625', 'n_tokens_mistral': 1116, 'n_tokens_neox': 956, 'n_words': 278}