Transformer models. Transformer sequence models [1]} have been successfully applied in other domains such as natural language processing [2]}, [3]}, [4]} and computer vision [5]}, [6]}. Using transformers in RL and sequential decision problems has proven difficult due to the instability of training [7]}, but recent work has investigated how to use transformers in model-based RL [8]}, motion forecasting [9]}, learning from demonstrations [10]}, and tele-operation [11]}.
