The Craft environment was originally introduced to evaluate the method of [1]}, however they rely on annotations of sub-routine sequences (`policy sketches') as additional supervision to the model.
In a similar way, TACO [2]} treats this setting as a sequence alignment and classification problem using an LSTM [3]} trained with a CTC loss [4]}.
In a recent work of [5]}, the primitives are learned directly from offline data for continuous control.
However, these methods learn a continuous (low-dimensional) space of primitives whereas our method represents primitives as a discrete set.
Our focus on learning useful sub-routines can also be viewed as an instance of learning temporal abstractions more broadly, such as event segments in video.
Relevant approaches propose to use recurrent latent variable models for this task [6]}, [7]}, while making stronger assumptions regarding prior knowledge about boundary locations and the existence of hierarchical structure between latent states and across time.
