Counterfactual regret minimization (CFR) [1]}, [2]}, [3]} is widely used in the benchmark field of the extensive-form game with imperfect information (IIEG)[4]}, since it can converge to equilibrium and gain an average strategy through iterations. But in fact, CFR needs much necessary information for calculation to work. In contrast, Reinforcement Learning (RL)[5]} algorithms provide a series of useful frameworks for making decisions in an uncertain environment. Agents constantly interact with the environment to obtain information and improve the game strategies in the end. The advantage of RL methods is that the agents can improve by themselves via setting reward function, but it is easy to make the agents fall into the local optimal state. Therefore, how to use CFR in IIEG under RL for finding a Nash equilibrium (NE) [6]}, [7]}, and design an effective exploration strategy to minimize interactions with the environment are important challenges.
