Implementation Details:
We choose ResNet-50 [1]} as the default backbone, unless otherwise specified, for our MS-STS VIS framework. Outputs from conv\({}_3\) , conv\({}_4\)  and conv\({}_5\)  of the Resnet backbone are utilized to obtain multi-scale feature inputs to our transformer encoder-decoder, as in [2]}. Both encoder and decoder layers are set to \(N_d{=}6\) . The feature dimension \(C\)  is set to 256, while the number of instance queries is set to 300 and length of video clip \(T{=}5\) , as in [3]}. We use the AdamW optimizer with a base learning rate (LR) of 2\(\times 10^{-4}\) , (\(\beta _1\) , \(\beta _2) {=} (0.9, 0.999)\)  and a weight decay of \(10^{-4}\) . LR of linear projections of deformable attention modules and the backbone are scaled by \(0.1\) . The model is first pretrained on COCO [4]} for 24 epochs with a batch size of 2. Similar to [3]}, the pretrained weights are then used to train the model on Youtube-VIS and COCO dataset for 12 epochs with \(T{=}5\)  and batch size set to 2. The LR is scaled by a factor of \(0.1\)  at 4th and 10th epochs. The framework is trained on 8 Nvidia V100 GPUs using PyTorch-1.9 [6]}.
<TABLE>