Since DNNs are usually sensitive to small modification of the weights, mutation may cause extent forgetting. To address this issue of [1]}, the evolutionary RL algorithm is modified in [2]} by adding personal replay memories and defining a crossover and a mutation operators. A small personal replay memory named genetic memory is defined for each individual to keep their most recent experiences. In the new crossover named Q-filtered distillation, the genetic memory of the child policy is filled with the latest experiences of its parents. The child policy is initialized with the weights of one of its parents and it is trained using the experiences of its genetic memory. The action selection procedure is toward minimizing a loss function that combines the policies of the parents. The new mutation named proximal mutation modifies the Gaussian perturbation of [1]} by scaling the Gaussian using the sum of gradients over a batch of transitions. This approach is evaluated on MuJoCo environments including Hopper, Ant and HalfCheetah.
