Test Time Training. In this setting, self-supervised tasks are introduced during training of the source models. These self-supervised tasks are later used at test time, allowing adaptation of the shared encoder between the main prediction branch and the auxiliary self-supervised task branch. TTT [1]}, one of the first works to formalize test time training, uses the self-supervised task of rotation prediction. Karani  [2]} proposed test time adaptation for medical image segmentation where instead of training an auxiliary branch, they trained a denoising autoencoder on the label space, and used its error as a self-supervised loss at test time. Other recent works [3]} extend the auxiliary branch based approach by using meta-learning to ensure that test-time training on the auxiliary branch would improve predictions from the main branch. These approaches cannot utilise any off-the-shelf model and improve its performance at test time. Instead they need to re-train the source model with hand-crafted and often complicated training strategies.
