Many recent works have applied Transformers beyond the domain of natural language, such as to images [1]}, [2]}, [3]}, and other high-dimensional data [4]}, [5]}.
Transformers were successfully applied to reason about visual primitives, such as object representations [6]}, and have recently gained a lot of interest in reinforcement learning settings [7]}, [8]}, [9]}, [10]}, [11]}, [12]}.
Especially in the offline setting [13]}, [14]}, which closely relates to the setting explored here,
Transformers have shown promise in predicting actions from certain input commands in a supervised manner [15]}, [16]}.
