@inproceedings{
martin2023modelfree,
title={Model-Free Preference Elicitation},
author={Carlos Martin and Craig Boutilier and Ofer Meshi},
booktitle={NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World},
year={2023},
url={https://openreview.net/forum?id=yvWlYTAkl3}
}
Elicitation of user preferences is an effective way to improve the quality of recommendations, especially when there is little or no user history. In this setting, a recommendation system interacts with the user by asking questions and recording the responses. Various criteria have been proposed for optimizing the sequence of queries to improve understanding of user preferences, and thereby the quality of downstream recommendations. A compelling approach is \emph{expected value of information (EVOI)}, a Bayesian approach which computes the expected gain in user utility for possible queries. Previous work on EVOI has focused on probabilistic models of user preferences and responses to compute posterior utilities. By contrast, in this work, we explore model-free variants of EVOI which rely on function approximation to obviate the need for strong modeling assumptions. Specifically, we propose to learn a user response model and user utility model from existing data, which is often available in real-world systems, and to use these models in EVOI in place of the probabilistic models. We show promising empirical results on a preference elicitation task.
Model-Free Preference Elicitation
Carlos Martin Craig Boutilier Ofer Meshi