When the problem is modeled as a sequential decision making problem and the components of MDP are defined, the next step in solving it with RL is to select an appropriate algorithm. One way to reduce the search space is to filter the algorithms based on the class of the problem. For example, if the states and actions are discrete and finite, tabular RL algorithms like typical Q-Learning and SARSA [1]} are suitable candidates and there is no need to search over the class of sophisticated algorithms in Deep Reinforcement Learning. Moreover, if the model of the environment is known, a wide variety of model-based algorithm like dynamic programming could be utilized. Despite having different context, algorithm selection approaches in AutoML can provide insights for AutoRL [2]}. Most of work in the domain of RL algorithm selection are intertwined with hyper-parameter optimization. For this reason, we explain these combined algorithm selection and hyper-parameter optimization in the next section and present few work that merely focus on algorithm selection in this section.
