Having defined the target distribution, we take the slot loss L for insertions within a particular slot to be the KL-divergence between the oracle distribution q oracle and the model distribution p. Substituting L in for the slot loss within the training framework of<cite> Stern et al. (2019)</cite> then gives the full sequence generation loss, which we can use to train an Insertion Transformer under any oracle policy rather than just the specific one they propose.