Another practical advantage of our solution is that allows for efficient experimentation on reward shaping.
During the development of an agent, the designer can define a shaped reward such that the goals in the environment are more easily satisfied.
However, reward shaping, is a trial-and-error iterative process [1]}.
When the shape of the reward is changed, say to remove shaped rewards that are introduced just to accelerate the learning, the value function is changed and the policy risk to collapse.
By adopting our hybrid BC-RL training policy, one can avoid such collapse by training an agent with BC, and 'protecting' the old policy while the new value function is learned. This is a practical and easy to implement approach that allow for faster design iterations.
Although we believe that this holds true for several environments, this has to be tested on a diverse range of cases and policies.
