Confidence estimation.
Once multi-scale features have been extracted, a sampling process occurs at each scale (using nearest-neighbor interpolation to sample at smaller scales) to compose a feature vector of size \(64+128+256+512+512+512\)  for each depth measurement contained in the LiDAR sparse depth map. The MLP head infers confidence by processing the high-level information regarding the image context and the original sparse depth distribution. Such an estimation comes in the form of variance, similarly to predictive uncertainty strategies [1]} (the lower, the more confident). It is worth noting that a plain convolutional decoder could be used in place of an MLP. However, the specific task we are tackling does not require generating a dense output. Thus a simple MLP can estimate the confidence only for the meaningful pixels in the input depth map. We will show in Sec.  how this approach steadily improves accuracy.
