In order to address three main RL challenges including sparsity of the reward function, lack of effective exploration and brittle convergence properties, a hyper algorithm that incorporates evolutionary algorithms (EA) and DRL is introduced in [1]}. Each individual in the evolutionary algorithms represents a DNN and mutation randomly alters the weights of a DNN. These DDNs are the actors in Deep Deterministic Policy Gradient (DDPG) [2]} algorithm. Initially, a population of actors are generated and the fitness for each actor is the cumulative reward of following its policy for a particular episode. Then the actors are perturbed through mutation and crossover to create new set of actors as the next generation. Through crossover, an offspring policy is obtained by randomly exchanging segments of parents' weights. The mutation is performed by applying random Gaussian noise to the weights. The experiences of actors are stored in a replay memory and these experiences are used by another actor (apart from individuals in EA) to update its weight through DDPG algorithm and this actor is copied to the EA population periodically. The algorithm is similar to DDPG except that the main actor has access to the experiences of individuals. This approach is categorized as bootstrapping as it uses information of other policies for training the target policy.
