<story><title>Key Papers in Deep Reinforcement Learning</title><url>https://spinningup.openai.com/en/latest/spinningup/keypapers.html</url></story><parent_chain></parent_chain><comment><author>inarrears</author><text>Many of these papers had been featured on HN before:&lt;p&gt;Neural Episodic Control &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=13843282&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=13843282&lt;/a&gt;&lt;p&gt;Exploration by Random Network Distillation &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=18346943&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=18346943&lt;/a&gt;&lt;p&gt;Evolution Strategies as a Scalable Alternative to Reinforcement Learning &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=13953980&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=13953980&lt;/a&gt;&lt;p&gt;Recurrent World Models Facilitate Policy Evolution &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=16860247&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=16860247&lt;/a&gt;&lt;p&gt;Playing Atari with Deep Reinforcement Learning &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=8484313&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=8484313&lt;/a&gt;</text></comment>
<story><title>Key Papers in Deep Reinforcement Learning</title><url>https://spinningup.openai.com/en/latest/spinningup/keypapers.html</url></story><parent_chain></parent_chain><comment><author>marmaduke</author><text>The spinning up guide is neat, but it seems to assume access to fairly expensive GPU resources for running models.&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;cloud.google.com&amp;#x2F;blog&amp;#x2F;products&amp;#x2F;gcp&amp;#x2F;new-lower-prices-for-gpus-and-preemptible-local-ssds&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;cloud.google.com&amp;#x2F;blog&amp;#x2F;products&amp;#x2F;gcp&amp;#x2F;new-lower-prices-...&lt;/a&gt;&lt;p&gt;&amp;gt; In US regions, each K80 GPU attached to a VM is priced at $0.45 per hour while each P100 costs $1.46 per hour.&lt;p&gt;The $300 free tier gets you ~600 hours of K80.  The spinning up guide suggests iterating models in &amp;lt;5 min, so that&amp;#x27;s 7200 iterations.&lt;p&gt;&amp;gt; start with vanilla policy gradient (also called REINFORCE), DQN, A2C (the synchronous version of A3C), PPO (the variant with the clipped objective), and DDPG, ... VPG...&lt;p&gt;that&amp;#x27;s 6 algorithms, combined with a half a dozen tasks to try, whittles it down to a few hundred iterations per task&amp;#x2F;algo combo.&lt;p&gt;that, combined with a lot of paper-reading, and perhaps clever blogging is probably enough to get started.&lt;p&gt;Still, it seems beneficial to democratize DL by making these 5 minute iterations free, doesn&amp;#x27;t it?</text></comment>