The policy network \(\pi _\theta ^{BC}\)  trained with BC cannot approximate the value function needed in an RL setting, more specifically in an actor-critic solution.
During the first training iterations using A2C, we expect to see the policy learning such value function.
However, the quality of the predicted actions is not constrained in any way, being it derived from the critic component, i.e., the approximated value.
This leads to a policy collapse, as the network is not able to retain the learned behaviour for the actor.
This is a specif case of the catastrophic forgetting [1]} that affects neural networks in sequential learning tasks.
Figure REF  show the training curves using A2C starting from the pretrained policy.
It can be seen, that, although initially the bot is performing well, it soon collapse to a bot that can neither win nor score.
<FIGURE>