We can interpret the Soft Actor-Critic algorithm [1]} as a combination of PPO and DDPG algorithms. Like the PPO algorithm, it uses stochastic policy instead of a deterministic one, but unlike PPO trains it to maximize Q-function directly as in DDPG. Moreover, the authors add additional action distribution entropy loss to an actor to prevent the policy from converging to a single-value distribution seen as a deterministic policy.
