@inproceedings{
rodriguez-soto2024an,
title={An Analytical Study of Utility Functions in Multi-Objective Reinforcement Learning},
author={Manel Rodriguez-Soto and Juan Antonio Rodriguez Aguilar and Maite L{\'o}pez-S{\'a}nchez},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=K3h2kZFz8h}
}
Multi-objective reinforcement learning (MORL) is an excellent framework for multi-objective sequential decision-making. MORL employs a utility function to aggregate multiple objectives into one that expresses a user's preferences. However, MORL still misses two crucial theoretical analyses of the properties of utility functions: (1) a characterisation of the utility functions for which an associated optimal policy exists, and (2) a characterisation of the types of preferences that can be expressed as utility functions. As a result, we formally characterise the families of preferences and utility functions that MORL should focus on: those for which an optimal policy is guaranteed to exist. We expect our theoretical results to promote the development of novel MORL algorithms that exploit our theoretical findings.
An Analytical Study of Utility Functions in Multi-Objective Reinforcement Learning
Manel Rodriguez-Soto Juan Antonio Rodriguez Aguilar Maite López-Sánchez