Our per-frame 2D object center localization network is shown in the upper part of Figure REF . Our network architecture is based on PVNet [1]}. To deal with multiple instances in the scene, we first use off-the-shelf YOLOv5 [2]} to detect 2D bounding boxes of the objects. The detections are then cropped and resized to 128x128 before being fed into the network. The network predicts pixel-wise binary labels and a 2D vector field towards the object center. A RANSAC-based voting scheme is finally utilized to estimate the mean \(u_k\)  and covariance \(\Sigma _k\)  of the object center at frame \(k\) . For more details of the object center localization prediction, we refer the reader to [1]}.
