CGAR: Critic Guided Action Redistribution in Reinforcement Leaning
Training a game-playing reinforcement learning agent requires multiple
interactions with the environment. Ignorant random exploration may cause a
waste of time and resources. It's essential to alleviate such waste. As
discussed in this paper, under the settings of the off-policy actor critic
algorithms, we demonstrate that the critic can bring more expected discounted
rewards than or at least equal to the actor. Thus, the Q value predicted by the
critic is a better signal to redistribute the action originally sampled from
the policy distribution predicted by the actor. This paper introduces the novel
Critic Guided Action Redistribution (CGAR) algorithm and tests it on the OpenAI
MuJoCo tasks. The experimental results demonstrate that our method improves the
sample efficiency and achieves state-of-the-art performance. Our code can be
found at https://github.com/tairanhuang/CGAR.