The point clouds are built from the accumulation of the 2D scans from a 2D Lidar. Each point cloud is set at the size of radius 50m, i.e. diameter 100m. The images are captured by the center camera of a Bumblebee tri-camera rig. Similar to 3DFeat-Net [1]}, 35 traversals are used for training, while 5 traversals are for testing.
In training and inference, the image-point cloud pair is selected using the following steps: a) Choose a Lidar point cloud from one of the traversals. b) Randomly select an image from the same traversal and captured within \(\pm 10\) m from the coordinate origin of the point cloud. The relative pose of the camera to the point cloud coordinate frame is \(G^p_c\) . c) Apply a random 2D rotation (around the up-axis) and translation (along the x-y plane) \(G_r\)  to the point cloud. d) The objective is to recover the ground truth transformation \(G_{gt}=G^p_cG_r^{-1}\) . There are 130,078 point clouds for training and 19,156 for testing.
