@inproceedings{
kim2024roidice,
title={{ROIDICE}: Offline Return on Investment Maximization for Efficient Decision Making},
author={Woosung Kim and Hayeong Lee and Jongmin Lee and Byung-Jun Lee},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=6Kg26g1quR}
}
In this paper, we propose a novel policy optimization framework that maximizes Return on Investment (ROI) of a policy using a fixed dataset within a Markov Decision Process (MDP) equipped with a cost function. ROI, defined as the ratio between the return and the accumulated cost of a policy, serves as a measure of efficiency of the policy. Despite the importance of maximizing ROI in various applications, it remains a challenging problem due to its nature as a ratio of two long-term values: return and accumulated cost. To address this, we formulate the ROI maximizing reinforcement learning problem as a linear fractional programming. We then incorporate the stationary distribution correction (DICE) framework to develop a practical offline ROI maximization algorithm.
Our proposed algorithm, ROIDICE, yields an efficient policy that offers a superior trade-off between return and accumulated cost compared to policies trained using existing frameworks.
ROIDICE: Offline Return on Investment Maximization for Efficient Decision Making
Woosung Kim Hayeong Lee Jongmin Lee Byung-Jun Lee