First, we leverage labeled image data to train SSIS-Track.
As shown in Fig. REF , given a single image, we adopt the feature pyramid network [1]} to extract features.
Beyond the box and class towers in SSIS [2]}, [3]} to predict shadow and object instances on single images, we add a tracking tower
and a tracking controller to generate parameters of the convolutional filters for each shadow/object instance.
The generated filters are then used in the tracking head to predict the tracking embedding of each instance.
The inputs
to the tracking head are the concatenation of the mask features and relative coordinates of each instance; see [2]}, [3]} for details.
