Due to the lack of appearance features, historically, the problem of 6D pose estimation for textureless objects is mainly addressed with depth data [1]}, [2]}, [3]}, [4]}, [5]} or RGB-D images [6]}, [7]}, [8]}, [9]}, [10]}. These approaches can achieve strong pose estimation performance when given high-quality depth data. Despite recent advances in depth acquisition technology, commodity-level depth cameras produce depth maps with low accuracy and missing data when surfaces are too glossy or dark, or the object is near to or far from the sensor. Hence, in the past decade, RGB-based solutions have received a lot of attention as an alternative approach [11]}, [12]}. Thanks to advances in deep learning, convolutional neural networks (CNNs) have been recently shown to significantly boost the object pose estimation performance using only RGB images [13]}, [14]}, [15]}, [16]}, [17]}. However, due to the scale, depth, and perspective ambiguities inherent to a single viewpoint, RGB-based solutions usually provide good 2D detection results but generally have low accuracy for the final 6D poses.
