In many RL tasks, actions are mainly the decisions of the agent that alter the state of the environment. Different types of actions such as continuous, discrete, multi-dimensional, bounded or unbounded would entail policies with different qualities. For example, a continuous action like prices in a dynamic pricing task can be modeled as either continuous or discrete action space. On one hand, continuous actions might be more precise, however, it is not possible to model them with tabular reinforcement learning or function approximation approaches like DQN that consider an output for each action. On the other hand, although discrete actions are easier to model, modeling a continuous space as discrete might be tricky especially when small changes in the action would have large impact on the total reward. Therefore, deriving a proper action representation is very important as it is difficult to find the best representation for actions that ends up with the best policy. For this reason, automating the definition of action spaces are necessary for many tasks. Action spaces could be a combination of discrete and continuous for multi-dimensional spaces like robot joints in robot navigation problems. For many continuous control task such as pendulum or BipedalWalker [1]}, the continuous action space could be discretized to represent discrete actions. In this subsection, we review learning actions and discretizing continuous action space separately.
