Training Configurations Our model is optimized by AdamW [1]} with learning rate \(2\times 10^{-3}\)  and batch size 1024. Weight decay is set to be 0.04. We adopt learning rate warmup [2]} in the first 10 epochs, and after warmup the learning rate follows a cosine decay schedule [3]}. The model uses multi-crop similar to [4]} and data augmentations similar to [5]}. The setting of momentum, temperature coefficient, and weight decay follows [6]}. The coefficient \(\lambda _1\)  of basic instance discrimination task is set as 1.0 while the restoration task \(\lambda _2\)  is set as 0.6.
