<story><title>Mastering Real-Time Strategy Games with Deep RL: Mere Mortal Edition</title><url>https://clemenswinter.com/2021/03/24/mastering-real-time-strategy-games-with-deep-reinforcement-learning-mere-mortal-edition/</url></story><parent_chain><item><author>Gunax</author><text>They can train the bots based on on those games though, right? Seems more like a flaw in the training data than the principle.&lt;p&gt;I am not sure if the training is done live or not--that is does the algorithm learn based on each game against a real, live player? Or do they just train the model offline, then allow players to play against the static model?</text></item><item><author>kevinwang</author><text>&amp;lt; OpenAI&amp;#x27;s DoTA 2 system wasn&amp;#x27;t playing the full game. I think the final version could play 17 of the 117 heroes, and the opposing human players were also restricted to playing this subset of the game.&lt;p&gt;The bigger issue in my eyes was that while OpenAI 5 defeated the world champion team OG, when they let anyone in the world fight it, some ingenious players figured out a pretty robust method to consistently exploit and defeat the bot.  As I haven&amp;#x27;t heard any buzz about OpenAI 5 since then, I think it was more or less unsuccessful unless they can show that their training method produces unexploitable bots (instead of bots that are really good against certain strategies)</text></item><item><author>FartyMcFarter</author><text>&amp;gt;  This trend has culminated in the defeat of top human players in the complex real-time strategy (RTS) games of DoTA 2   [1]   and StarCraft II  [2]  in 2019.&lt;p&gt;Not quite:&lt;p&gt;- OpenAI&amp;#x27;s DoTA 2 system wasn&amp;#x27;t playing the full game. I think the final version could play 17 of the 117 heroes, and the opposing human players were also restricted to playing this subset of the game.&lt;p&gt;- DeepMind&amp;#x27;s StarCraft II system reached a level above &amp;quot;above 99.8% of officially ranked human players.&amp;quot;, so it isn&amp;#x27;t trivial to argue that this amounts to defeating top players.</text></item></parent_chain><comment><author>jrumbut</author><text>Sibling posters have pointed out technical issues but I&amp;#x27;d like to point out that while chasing after each successive exploit might let them stay on top of the Starcraft rankings, it changes the accomplishment from &amp;quot;we made a model that understands Starcraft better than almost any human&amp;quot; to &amp;quot;we made a model that memorized a selection of very good strategies and tactics.&amp;quot;&lt;p&gt;When it was first demonstrated, it really looked like it was doing very smart things (while also taking advantage of the fact that it doesn&amp;#x27;t have attention lapses and hand fatigue) and reacting well to different strategies on a level that was freaky to me.</text></comment>
<story><title>Mastering Real-Time Strategy Games with Deep RL: Mere Mortal Edition</title><url>https://clemenswinter.com/2021/03/24/mastering-real-time-strategy-games-with-deep-reinforcement-learning-mere-mortal-edition/</url></story><parent_chain><item><author>Gunax</author><text>They can train the bots based on on those games though, right? Seems more like a flaw in the training data than the principle.&lt;p&gt;I am not sure if the training is done live or not--that is does the algorithm learn based on each game against a real, live player? Or do they just train the model offline, then allow players to play against the static model?</text></item><item><author>kevinwang</author><text>&amp;lt; OpenAI&amp;#x27;s DoTA 2 system wasn&amp;#x27;t playing the full game. I think the final version could play 17 of the 117 heroes, and the opposing human players were also restricted to playing this subset of the game.&lt;p&gt;The bigger issue in my eyes was that while OpenAI 5 defeated the world champion team OG, when they let anyone in the world fight it, some ingenious players figured out a pretty robust method to consistently exploit and defeat the bot.  As I haven&amp;#x27;t heard any buzz about OpenAI 5 since then, I think it was more or less unsuccessful unless they can show that their training method produces unexploitable bots (instead of bots that are really good against certain strategies)</text></item><item><author>FartyMcFarter</author><text>&amp;gt;  This trend has culminated in the defeat of top human players in the complex real-time strategy (RTS) games of DoTA 2   [1]   and StarCraft II  [2]  in 2019.&lt;p&gt;Not quite:&lt;p&gt;- OpenAI&amp;#x27;s DoTA 2 system wasn&amp;#x27;t playing the full game. I think the final version could play 17 of the 117 heroes, and the opposing human players were also restricted to playing this subset of the game.&lt;p&gt;- DeepMind&amp;#x27;s StarCraft II system reached a level above &amp;quot;above 99.8% of officially ranked human players.&amp;quot;, so it isn&amp;#x27;t trivial to argue that this amounts to defeating top players.</text></item></parent_chain><comment><author>CyberRage</author><text>Training requires millions of games. playing against humans is only for evaluation purposes, not for training.&lt;p&gt;In both cases, it was indeed a static model but more recent work which is called MuZero is not static and achieves great results in board games and atari.</text></comment>