Continuous action spaces are challenging in many tasks. As mentioned before, some RL algorithms like DQN do not work well for continuous action spaces because they rely on \(\epsilon \) -greedy algorithm and the best action is required at each step. Finding the best action in a continuous space needs an optimization step for each interaction with the environment which is intractable. For discrete action spaces, a separate output is considered in the policy network for each action which is not possible when the action space is continuous as it needs infinite number of outputs. Although discretization [1]} makes the action space discrete and manageable, it is not suitable for tasks that are very sensitive to small alteration of actions.
