@inproceedings{
yalcinkaya2023automata,
title={Automata Conditioned Reinforcement Learning with Experience Replay},
author={Beyazit Yalcinkaya and Niklas Lauffer and Marcell Vazquez-Chanlatte and Sanjit Seshia},
booktitle={NeurIPS 2023 Workshop on Goal-Conditioned Reinforcement Learning},
year={2023},
url={https://openreview.net/forum?id=ktyvmR8btd}
}
We explore the problem of goal-conditioned reinforcement learning (RL) where goals are represented using deterministic finite state automata (DFAs). Due to the sparse and binary nature of automata-based goals, we hypothesize that experience replay can help an RL agent learn more quickly and consistently in this setting. To enable the use of experience replay, we introduce a novel end-to-end neural architecture, including a graph neural network (GNN) to encode the DFA goal before passing it to a feed-forward policy network. Experimental results in a gridworld domain demonstrate the efficacy of the model architecture and highlight the significant role of experience replay in enhancing the learning speed and reducing the variance of RL agents for DFA tasks.
Automata Conditioned Reinforcement Learning with Experience Replay
Beyazit Yalcinkaya Niklas Lauffer Marcell Vazquez-Chanlatte Sanjit Seshia