To achieve instance paired tracking, we build SSIS-Track
by designing a tracking branch over SSIS [1]}, [2]}.
First, we leverage the labeled images in SOBA [3]} to train the tracking branch by contrastive learning [4]}, [5]}, such that it can
learn to generate distinguishable tracking embeddings
of shadow/object instances.
Specifically, we encourage the distance between the tracking embeddings to be small when they are from the same instance.
Also, we collect
unlabeled videos in SOBA-VID to refine
the performance of SISS-Track
by exploring the temporal information in a self-supervised manner [6]}, [7]}, [8]}.
Further, we adopt a tracking-by-detection approach to match instances frame by frame to enable
instance paired tracking.
Last, we design a new retrieving mechanism to handle unpaired instances that typically happen when an object/shadow is temporarily occluded or out of view in the video.
Fig. REF  shows examples, where our SSIS-Track successfully detects various paired shadow-object associations and retrieves an
unpaired shadow instance in videos.
