Fully Test Time Adaptation.
These approaches have the advantage of adapting off-the-shelf models, since they do not rely on any auxiliary task.
The state-of-the-art method in this category, TENT [1]}, formalised this setting and achieved better performance than some of the methods which actually re-train the source model.
TENT runs an online optimization over a given test set, which minimizes the entropy of the predicted distribution for every incoming batch. Although the method can be made to work in the SITA setting where the model is reset after every test instance, their results are primarily focused in the online setting by gradually adapting the model over the test set.
It gives impressive performance in the online setting, but we observe significant drop in performance in the SITA setting (Figure REF ), when there is only a single test instance.
[2]} improves upon TENT by adjusting the loss function to a log-likelihood ratio instead of entropy, and maintaining a running estimate of the output distribution.
This setting, while clearly more useful than the previous one, is not as realistic as SITA because it needs to (i) accumulate and store samples to create batches, (ii) run an online optimisation under the assumption that the test samples come from the same distribution.
