Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropy
Multi-agent deep reinforcement learning has been applied to address a variety
of complex problems with either discrete or continuous action spaces and
achieved great success. However, most real-world environments cannot be
described by only discrete action spaces or only continuous action spaces. And
there are few works having ever utilized deep reinforcement learning (drl) to
multi-agent problems with hybrid action spaces. Therefore, we propose a novel
algorithm: Deep Multi-Agent Hybrid Soft Actor-Critic (MAHSAC) to fill this gap.
This algorithm follows the centralized training but decentralized execution
(CTDE) paradigm, and extend the Soft Actor-Critic algorithm (SAC) to handle
hybrid action space problems in Multi-Agent environments based on maximum
entropy. Our experiences are running on an easy multi-agent particle world with
a continuous observation and discrete action space, along with some basic
simulated physics. The experimental results show that MAHSAC has good
performance in training speed, stability, and anti-interference ability. At the
same time, it outperforms existing independent deep hybrid learning method in
cooperative scenarios and competitive scenarios.