Automated Reinforcement Learning (AutoRL) provides a framework to automatically make appropriate decisions about the settings of an RL procedure before starting the learning. In other words, RL components, including state, action and reward, algorithm selection, and hyper-parameters optimization, are determined through AutoRL, and the best configuration for each component is provided for an RL procedure to solve a task. Figure REF  shows an RL pipeline containing the RL components. To model and solve a problem using RL, the steps of this pipeline are followed, starting from MDP modeling. AutoRL aims to automate different steps of this pipeline and reduce the necessity of expert knowledge. We use AutoRL to emphasize the resemblance with Automated Machine Learning (AutoML) as a framework for automating supervised and unsupervised learning procedures. According to [1]}, an AutoML pipeline consists of the components of Machine Learning frameworks.
It starts with data preprocessing that contains data collection and data cleaning. The next step is feature engineering including feature extraction, feature construction and feature selection. After processing the data and defining the features, a model is developed and optimized to perform classification or clustering tasks. Finally, the quality of the model is determined through evaluation. In order to automate this procedure, the output of evaluation is used to configure new settings and follow the steps again. In this way, several iterations might be needed to derive the optimal settings for data preprocessing, feature engineering and model generation.
Although some components of AutoML and AutoRL are similar, AutoML methods cannot necessarily be used in AutoRL because the configuration of the problems and the complexity of the evaluation step are different.
AutoRL pipeline builds a pipeline for RL to define and select components of RL automatically. In recent years, by combining Deep Learning and RL, the need for automating the components of Deep Reinforcement Learning (DRL) is increased because many modeling decisions are made manually and even RL experts have to test several different configurations related to state, action, reward, algorithm and hyper-parameters, to obtain the best definition.
