An overview of the proposed BundleTrack framework is depicted in Fig. REF . The currently observed RGB-D frame \(I_{t}\)  and the object segmentation mask computed during the last timestamp \(M_{t-1}\)  are forwarded to a video segmentation network to compute the current object mask \(M_{t}\) . Based on \(M_t\)  and \(M_{t-1}\)  respectively, the target object regions in both \(I_{t}\)  and \(I_{t-1}\)  are cropped, resized and sent to a keypoint detection network to compute keypoints and feature descriptors. A data association process consisting of feature matching and outlier pruning in the manner of RANSAC [1]} identifies feature correspondences. Based on these correspondences, a registration between \(I_{t-1}\)  and \(I_{t}\)  can be solved in closed-form, which is then used to provide a coarse estimate \(\tilde{\mathbf {T}}_t\)  for the transform between the two snapshots. The estimate \(\tilde{\mathbf {T}}_t\)  is used to initialize the current node \(\mathbf {T}_{t}\)  as part of a pose graph optimization step. To define the rest of the nodes of the pose graph, no more than \(\mathcal {K}\)  keyframes are selected from a memory pool to participate in the optimization. The choice of \(\mathcal {K}\)  is made to balance an efficiency vs. accuracy tradeoff. Pose graph edges include both feature and geometric correspondences, which are computed in parallel on GPU. Given this information, the pose graph step outputs online the optimized pose for the current timestamp \(\mathbf {T}_{t} \in SE(3)\) . If the last frame corresponds to a novel view, then it is also included in the memory pool.
