To perform better than the scripted baseline, we need to move beyond the performance achieved through behavioural cloning.
We propose to start from the pretrained policy network \(\pi _\theta ^{BC}\) , and optimize the policy using reinforcement learning, while avoiding a policy collapse using the behavioural cloning objective as presented in the section above.
The resulting policy \(\pi _\theta ^{MB}\)  does not only outperforms the scripted agent, but it can be further improved by using a self-playing mechanism.
In this work we play against the scripted bot and we adopt A2C [1]} as RL algorithm, however, our solution works with other and more advanced algorithms such as TRPO [2]}, PPO [3]} and several others [4]}.
