BatchNorm Adaptation. Recent literature on domain-adaptation
proposes adapting the statistics of only the Batch Normalization layers for adjusting to the test distribution.
Though these works are not targeted towards the TTA task, they suggest that adapting normalization statistics could provide significant performance gains, without requiring hand-crafted loss functions.
Prediction Time Normalization (PTN) [1]} uses the mean and variance of the current test batch as the statistics in the batch norm layer, instead of using the accumulated statistics of the source data.
This works reasonably well if the test batch size is large enough to provide a good estimate.
Schneider  [2]} use a similar approach with focus on improving robustness to corruption.
The source model's accumulated statistics are combined with statistics accumulated over all the available test images to reach a reliable estimate.
They show usefulness of the approach in two settings: (i) full adapt where the entire test set is available for offline adaptation and (ii) partial adapt where a subset of the test set is available for offline adaptation.
A special case of the partial adapt setting, where only one image is available at a time can be considered as test time adaptation.
While Schneider  [2]} do not report results on test time adaptation, they provide suggested hyper-parameter values when only one image is available.
Thus, we include this method as a relevant baseline.
Our AugBN algorithm is similar in spirit to these works, designed specifically to provide a reliable batch norm statistics estimate at test time, within the constraints of the proposed SITA setting. Our method improves test time statistics estimation by including augmented samples around the current test image.
