Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering
Deploying models on target domain data subject to distribution shift requires
adaptation. Test-time training (TTT) emerges as a solution to this adaptation
under a realistic scenario where access to full source domain data is not
available and instant inference on target domain is required. Despite many
efforts into TTT, there is a confusion over the experimental settings, thus
leading to unfair comparisons. In this work, we first revisit TTT assumptions
and categorize TTT protocols by two key factors. Among the multiple protocols,
we adopt a realistic sequential test-time training (sTTT) protocol, under which
we further develop a test-time anchored clustering (TTAC) approach to enable
stronger test-time feature learning. TTAC discovers clusters in both source and
target domain and match the target clusters to the source ones to improve
generalization. Pseudo label filtering and iterative updating are developed to
improve the effectiveness and efficiency of anchored clustering. We demonstrate
that under all TTT protocols TTAC consistently outperforms the state-of-the-art
methods on five TTT datasets. We hope this work will provide a fair
benchmarking of TTT methods and future research should be compared within
respective protocols. A demo code is available at
https://github.com/Gorilla-Lab-SCUT/TTAC.