In this paper, we present a link between preference-based and multiobjective sequential decision-making. While transforming a multiobjective problem to a preference-based one is quite natural, the other direction is a bit less obvious. We present how this transformation (from preference-based to multiobjective) can be done under the classic condition that preferences over histories can be represented by additively decomposable utilities and that the decision criterion to evaluate policies in a state is based on expectation. This link yields a new source of multiobjective sequential decision-making problems (i.e., when reward values are unknown) and justifies the use of solving methods developed in one setting in the other one.
Multiobjective MDP (MOMDP) @cite_11 is an MDP @math where the reward function is redefined as @math with @math being the number of objectives. The value function @math of a policy @math is now vectorial and can be computed as the limit of the vectorial version of ) and ): @math , In MOMDP, the value function of policy @math Pareto-dominates that of another policy @math if in every state @math , @math is not smaller than @math on every objective and @math is greater than @math on at least one objective. By extension, we say that @math Pareto-dominates @math if value function @math Pareto-dominates value function @math . A value function (resp. policy) is Pareto-optimal if it is not Pareto-dominated by any other value function (resp. policy). Due to incomparability of vectorial value functions, there are generally many Pareto-optimal value functions (and therefore policies), which constitutes the main difficulty of the multiobjective setting.
@cite_11
2401796785
This paper is devoted to fair optimization in Multiobjective Markov Decision Processes (MOMDPs). A MOMDP is an extension of the MDP model for planning under uncertainty while trying to optimize several reward functions simultaneously. This applies to multiagent problems when rewards define individual utility functions, or in multicriteria problems when rewards refer to different features. In this setting, we study the determination of policies leading to Lorenz-non-dominated tradeoffs. Lorenz dominance is a refinement of Pareto dominance that was introduced in Social Choice for the measurement of inequalities. In this paper, we introduce methods to efficiently approximate the sets of Lorenz-non-dominated solutions of infinite-horizon, discounted MOMDPs. The approximations are polynomial-sized subsets of those solutions.