Given \(t\) -th i-frame, which is noticed as \(x_t\in {\mathcal {R}^{W\times {H}\times 3}}\)  of a video clip \(X\) . Our goal is to extract semantic features from \(x_t\) , which are represented by bounding box \((a^{k}_1, b^{k}_1, a^{k}_2, b^{k}_2)\)  and class ID \(c^{k}\)  respectively for object \(k\) . Following the method in [1]}, \(x_t\)  is first fed into deep layer aggregation (DLA) network [2]} to predict a center point heatmap \(\hat{Y}\in [0,1]^{{W/R}\times {H/R}\times {C}}\) , where \(R\)  is the output stride and \(C\)  is the number of predefined object categories. In \(\hat{Y}\) , a prediction of 1 corresponds predicted center point of an object, while a prediction of 0 corresponds to predicted background. Notably, the DLA network can be other fully-convolutional encoder-decoder networks, such as stacked hourglass network [3]}, [4]} and up-convolutional residual networks (ResNet) [5]}, [6]}. Based on the predicted heatmap, a branch network is introduced to regress the size of all the objects in image \(\hat{S}\in \mathcal {R}^{{W/R}\times {H/R}\times 2}\) . When output stride \(R>1\) , another additional branch is needed to predict a local offset \(\hat{O}\in \mathcal {R}^{W/R\times {H/R}\times 2}\)  to compensate the error caused by rounding, following [1]}.
