Temporal Feature Pyramid Network: In a temporal feature pyramid network (TFPN), the predictions are yielded from multiple resolution feature maps. This idea was first introduced in Single Shot Detector (SSD) [1]} for object detection, and then extended to temporal domain for action detection in SSAD [2]} and \(\text{S}^3\text{D}\)  [3]}. They proposed an end-to-end network where the lower-level feature maps with higher resolution and smaller receptive field are responsible to detect short action instances while the top layers with lower resolution and larger receptive field, detect long action instances. For each feature map cell, several anchor segments with multiple scales are considered around the center that are fed to convolutional layers for evaluation. The limitation of this approach is that lower layers in the pyramid are unaware of high-level semantic information, and top layers lack enough details, so they all fail to localize the actions accurately.
