In our adversarial formulation, the objective is to discriminate between the ground-truth mask and the predicted mask output by our MS-STS VIS. With this objective, we introduce an auxiliary discriminator network during training. The discriminator takes the input frames along with the corresponding encoder features and binary masks as its input. Here, the binary mask \(\mathbf {M}\)  is obtained either from the ground-truth or predictions, such that all object instances (category-agnostic) within a frame are marked as foreground. While the discriminator \(D\)  attempts to distinguish between ground-truth and predicted binary masks (\(\mathbf {M}_{gt}\)  and \(\mathbf {M}_{pred}\) , respectively), the encoder learns to output enhanced features \(\mathbf {E}\)  such that the predicted masks \(\mathbf {M}_{pred}\)  are close to ground-truth \(\mathbf {M}_{gt}\) .
Let \(\mathbf {F}_{gt} = \mathrm {CONCAT}(\mathbf {x},\mathbf {E},\mathbf {M}_{gt})\)  and \(\mathbf {F}_{pr} = \mathrm {CONCAT}(\mathbf {x},\mathbf {E},\mathbf {M}_{pred})\)  denote the real and fake input, respectively, to the discriminator \(D\) . Similar to [1]}, the adversarial loss is then given by
\(\min _{Enc} \max _{D} \mathbb {E}[\log D(\mathbf {F}_{gt})] + \mathbb {E}[\log (1 - D(\mathbf {F}_{pr}))] + \lambda _1 \mathbb {E}[||D(\mathbf {F}_{gt})-D(\mathbf {F}_{pr})||_1].\) 
