Normally, there are three challenges in hyper-parameters optimization of DRL. First, dynamic environments require dynamic hyper-parameters and optimal hyper-parameters settings in one stage might poorly work in another stage. Second, the optimization is not sample efficient and needs a full training run to test each selected hyper-parameter. Third, dynamic modification of the neural network is not considered in literature. A joint optimization approach based on evolutionary algorithms that optimizes the agent's network and its hyper-parameters simultaneously is presented in [1]}. In the evolutionary framework, each individual is a DRL agent consisting of a policy and a value network together with RL algorithm paramteres. Rollouts of each individual is stored in a shared replay memory to be used as experiences for other agents. Each agent is evaluated by running for at least one episode in the environment and the mean reward of the agent is used as its fitness. After crossover and mutation, all the agents in the environment are trained using the experiences in the shared replay memory. This approach is applied on TD3 algorithm in MoJuCo continuous control benchmark. This integration of evolutionary algorithms and neural networks is known as neuroevolution [2]}.
