Plenty of the already mentioned works in gradient-free optimization specifically applied these methods to RL tasks  [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}.
Overall, the performance of ES-based methods is comparable to conventional policy gradients, especially if the number of model parameters is small [8]}, [9]}. Another advantage of ES over policy gradient is that it produces more robust and diverse policies [10]}, [6]} by eliminating the problem of delayed rewards and short length time horizons. Finally, evolutionary approaches are suitable for the problems with non-Markovian properties [7]}.
