Fusion. The simplest choice for fusing the time-shifted temporal features with the residual is summation, as done in our preliminary work gsm [1]}. However, since different channels of a feature tensor encode different level of information, fusion by summation may be sub-optimal. Instead  gsf performs weighted averaging on the two feature groups. For this, the group-shifted features and their corresponding residuals are concatenated followed by spatial average pooling operation, resulting in a tensor of shape \(2\times C\times T\) . Then a 2D convolution is applied on this tensor followed by \(sigmoid\)  non-linearity to obtain a weight map with shape \(1\times C\times T\) . This weight map is then reshaped to \(C\times T \times 1\times 1\)  and is used to fuse the time-shifted features and their residuals via weighted averaging. When the fusion weight at a feature channel is 0, only the residual features with the spatial information is retained for further processing. On the other hand, if the channel weight is 1, the network will discard the residual spatial features and propagate the time-shifted temporal features. With other weight values, the model mixes the two feature groups selectively by mixing the channels.
