<story><title>OpenAI Five: Goals and Progress</title><url>https://openai.com/five/</url></story><parent_chain><item><author>shawn</author><text>As someone who was once semi-pro in dota (4400 MMR, get rekt), it&amp;#x27;s &lt;i&gt;freaky&lt;/i&gt; watching these bots play. It&amp;#x27;s uncanny. Little things... Like, when the bots are taking a tower, one of them will stand in front of the tower and tank the creep wave, so that their creeps do more damage on the tower. They had to learn this.&lt;p&gt;Insta-TPing right when an enemy wastes their stun and can&amp;#x27;t cancel their TP.&lt;p&gt;Grouping up as 5 at the beginning of the game and pushing into the enemy jungle. &lt;i&gt;Pubs never do this.&lt;/i&gt;&lt;p&gt;The most interesting part is that OpenAI appears to be discovering new knowledge in the dota scene. For example, they always take the ranged barracks first, never the melee. This is exactly the opposite of what the pro scene does. Therefore, the smartest pro team should study what the bot is doing and trust that on average it&amp;#x27;s a better idea to always focus on the ranged barracks first. After all, if it was a bad idea, they probably wouldn&amp;#x27;t do that.&lt;p&gt;The most hilarious part was when OpenAI paused the game, then resumed it. This illustrates that there is still some unexplainable randomness.&lt;p&gt;Question for OpenAI: Is it more accurate to think of the bots as 5 separate minds, or a single mind controlling 5 heroes?&lt;p&gt;EDIT: By the way, TI is going on right now! &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.twitch.tv&amp;#x2F;dota2ti&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.twitch.tv&amp;#x2F;dota2ti&lt;/a&gt; If you&amp;#x27;re new to the scene, take a peek. TI is always so high energy -- even if it&amp;#x27;s hard to follow what&amp;#x27;s going on, listening to Tobi (the shoutcaster) go nuts during the game is always a highlight.&lt;p&gt;And of course, &amp;#x2F;r&amp;#x2F;dota2 has the best memes anywhere, hands-down. &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.reddit.com&amp;#x2F;r&amp;#x2F;DotA2&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.reddit.com&amp;#x2F;r&amp;#x2F;DotA2&amp;#x2F;&lt;/a&gt;</text></item></parent_chain><comment><author>zawerf</author><text>&amp;gt; Question for OpenAI: Is it more accurate to think of the bots as 5 separate minds, or a single mind controlling 5 heroes?&lt;p&gt;They released their architecture: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.reddit.com&amp;#x2F;r&amp;#x2F;MachineLearning&amp;#x2F;comments&amp;#x2F;9533g8&amp;#x2F;n_openai_five_benchmark_results&amp;#x2F;e3q0f9l&amp;#x2F;?context=1&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.reddit.com&amp;#x2F;r&amp;#x2F;MachineLearning&amp;#x2F;comments&amp;#x2F;9533g8&amp;#x2F;n_o...&lt;/a&gt;&lt;p&gt;In the above thread, a reddit user noted that 512 out of the 2048 unit input into the LSTM of each bot is shared (max pooled across players).&lt;p&gt;This means they are telepathically linked and never need to worry about communicating, disagreeing, etc. They know how the others are interpreting their local inputs because it&amp;#x27;s explicitly shared. So it&amp;#x27;s not really fair to call it 5 separate minds.&lt;p&gt;You can&amp;#x27;t really call it a single mind either because if you assume the LSTM is the &amp;quot;mind&amp;quot; (since it&amp;#x27;s the only place that has memory of previous state), that state isn&amp;#x27;t read by any other bots.</text></comment>
<story><title>OpenAI Five: Goals and Progress</title><url>https://openai.com/five/</url></story><parent_chain><item><author>shawn</author><text>As someone who was once semi-pro in dota (4400 MMR, get rekt), it&amp;#x27;s &lt;i&gt;freaky&lt;/i&gt; watching these bots play. It&amp;#x27;s uncanny. Little things... Like, when the bots are taking a tower, one of them will stand in front of the tower and tank the creep wave, so that their creeps do more damage on the tower. They had to learn this.&lt;p&gt;Insta-TPing right when an enemy wastes their stun and can&amp;#x27;t cancel their TP.&lt;p&gt;Grouping up as 5 at the beginning of the game and pushing into the enemy jungle. &lt;i&gt;Pubs never do this.&lt;/i&gt;&lt;p&gt;The most interesting part is that OpenAI appears to be discovering new knowledge in the dota scene. For example, they always take the ranged barracks first, never the melee. This is exactly the opposite of what the pro scene does. Therefore, the smartest pro team should study what the bot is doing and trust that on average it&amp;#x27;s a better idea to always focus on the ranged barracks first. After all, if it was a bad idea, they probably wouldn&amp;#x27;t do that.&lt;p&gt;The most hilarious part was when OpenAI paused the game, then resumed it. This illustrates that there is still some unexplainable randomness.&lt;p&gt;Question for OpenAI: Is it more accurate to think of the bots as 5 separate minds, or a single mind controlling 5 heroes?&lt;p&gt;EDIT: By the way, TI is going on right now! &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.twitch.tv&amp;#x2F;dota2ti&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.twitch.tv&amp;#x2F;dota2ti&lt;/a&gt; If you&amp;#x27;re new to the scene, take a peek. TI is always so high energy -- even if it&amp;#x27;s hard to follow what&amp;#x27;s going on, listening to Tobi (the shoutcaster) go nuts during the game is always a highlight.&lt;p&gt;And of course, &amp;#x2F;r&amp;#x2F;dota2 has the best memes anywhere, hands-down. &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.reddit.com&amp;#x2F;r&amp;#x2F;DotA2&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.reddit.com&amp;#x2F;r&amp;#x2F;DotA2&amp;#x2F;&lt;/a&gt;</text></item></parent_chain><comment><author>jdoliner</author><text>I&amp;#x27;m also very excited about what these bots can tell us about DOTA but I think a certain degree of skepticism is necessary due to the bots not actually having been trained truly tabula rasa. The training contained lots of little &amp;quot;nudges&amp;quot; to reward the bots for things that lead to winning. I.e. there are nudges to get them to go to lane to start, there are nudges to get them to buy the right items. I don&amp;#x27;t know for sure but I wouldn&amp;#x27;t be surprised if they kill the ranged barracks first because there&amp;#x27;s a built in reward for killing barracks that&amp;#x27;s the same for both melee and ranged barracks but the ranged ones have less health so they value that a little bit higher since it&amp;#x27;s a more consistent reward than trying to get the melee one. I hope that someday they get to a point where they can truly train from scratch with no human nudges, I&amp;#x27;m not sure how much more impressive of an ML result that would be (I&amp;#x27;d imagine meaningfully so but I&amp;#x27;m not an AI researcher) but it would certainly be a much more interesting result in terms of our understanding of DOTA.</text></comment>