where \(\Gamma \)  is a projection operator onto the set of distributions, \(d\)  is a distance measure, \(\mathcal {D}^{\pi }_{\mathcal {I}^\pi }\)  is a distribution over all states and actions that their advantage value is positive. In order to minimize the distance between two distributions, Implicit Quantile Network [1]} is employed by using Wasserstein distance metric. DPO algorithm is incorporated in Generative Actor Critic that contains four neural networks for policy, delayed policy, value network and action-value network.
