Implementation details: We conduct experiments on two widely used backbones: 4-layer ConvNet (Conv-4) and ResNet-12.
Same as [1]}, [2]}, Conv-4 consists of 4 consecutive 64-channel convolution blocks that each downsample by a factor of 2.
The shape of the output feature maps for input images of size 84\(\times \) 84 is thus 64\(\times \) 5\(\times \) 5. For ResNet-12, we use the same implementation as [1]}, [4]}, [2]}.
The input image size is the same as Conv-4 and the output feature map shape is 640\(\times \) 5\(\times \) 5. During training, we use the standard data augmentation as in [1]}, [7]}, [8]}, [9]}, which includes random crop, right-left flip and color jitter.
Further training details can be found in SM Sec. .
