There are several limitations to our proposed consistency loss method.
First, similar to the temporal consistency loss proposed in [1]}, the cycle loss method incurs overhead at the training time, as it requires two passes of the model during training, nearly doubling the training time.
Second, some of the cycle loss parameters depend on the model architecture and dataset. For example, we find the optimal cycle loss weight is different in GOHOME-Mod and in Autobots.
Third, cycle loss requires the model to be end-to-end differentiable. Some models [2]}, [3]}, [4]}, [5]} rely on sampling a probability distribution as an intermediate step, and more sophisticated designs will be needed in order to apply cycle loss to those models.
