U-shaped Temporal Feature Pyramid Network: In order to mitigate the problem with regular TFPNs, a U-shaped TFPN architecture was designed to connect high-level and low-level features. This idea was first introduced in Unet [1]}, FPN [2]}, and DSSD [3]} for object detection and then was generalized to temporal domain in MGG [4]}, PBRNet [5]}, RapNet [6]}, C-TCN [7]}, and MLTPN [8]}. The video representation features are extracted using off-the-shelf feature extractors. Then temporal convolution and max pooling layers are applied to reduce the temporal dimension and increase the receptive field size. This is followed by temporal deconvolution layers for upscaling. Then, high-level features are combined with corresponding low-level features with lateral connections between the convolutional and deconvolutional layers. U-shaped TFPNs have drawn much attention recently and achieved state-of-the art results for temporal action detection task.
