@inproceedings{
sun2024identifying,
title={Identifying Latent State-Transition Processes for Individualized Reinforcement Learning},
author={Yuewen Sun and Biwei Huang and Yu Yao and Donghuo Zeng and Xinshuai Dong and Songyao Jin and Boyang Sun and Roberto Legaspi and Kazushi Ikeda and Peter Spirtes and Kun Zhang},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=kREpCQtHdN}
}
In recent years, the application of reinforcement learning (RL) involving interactions with individuals has seen significant growth. These interactions, influenced by individual-specific factors ranging from personal preferences to physiological differences, can causally affect state transitions, such as the health conditions in healthcare or learning progress in education. Consequently, different individuals may exhibit different state-transition processes. Understanding these individualized state-transition processes is crucial for optimizing individualized policies. In practice, however, identifying these state-transition processes is challenging, especially since individual-specific factors often remain latent. In this paper, we establish the identifiability of these latent factors and present a practical method that effectively learns these processes from observed state-action trajectories. Our experiments on various datasets show that our method can effectively identify the latent state-transition processes and help learn individualized RL policies.
Identifying Latent State-Transition Processes for Individualized Reinforcement Learning
Yuewen Sun Biwei Huang Yu Yao Donghuo Zeng Xinshuai Dong Songyao Jin Boyang Sun Roberto Legaspi Kazushi Ikeda Peter Spirtes Kun Zhang