@inproceedings{
asadulaev2024rethinking,
title={Rethinking Optimal Transport in Offline Reinforcement Learning},
author={Arip Asadulaev and Rostislav Korst and Alexander Korotin and Vage Egiazarian and Andrey Filchenkov and Evgeny Burnaev},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=hKloKv7pR2}
}
We propose a novel algorithm for offline reinforcement learning using optimal transport. Typically, in offline reinforcement learning, the data is provided by various experts and some of them can be sub-optimal. To extract an efficient policy, it is necessary to \emph{stitch} the best behaviors from the dataset. To address this problem, we rethink offline reinforcement learning as an optimal transportation problem. And based on this, we present an algorithm that aims to find a policy that maps states to a \emph{partial} distribution of the best expert actions for each given state. We evaluate the performance of our algorithm on continuous control problems from the D4RL suite and demonstrate improvements over existing methods.
Rethinking Optimal Transport in Offline Reinforcement Learning
Arip Asadulaev Rostislav Korst Alexander Korotin Vage Egiazarian Andrey Filchenkov Evgeny Burnaev