We conduct experiments to demonstrate how the self-supervised learning improves the system. We first show that by using the self-annotated images to fine-tune the semantic segmentation network, we can significantly improve the segmentation accuracy compared to the original network trained with synthetic data only. Semantic segmentation suffers from the domain gap severely for objects with complex material properties, such as shininess. Fine-tuning the network can bridge the domain gap. Second, we show that fine-tuning the auto-encoders used in PoseRBPF [1]} can improve their reconstruction quality, and consequently improve the pose estimation accuracy. Finally, we conduct robot grasping experiments to illustrate that our fine-tuned system can achieve a high success rate in model-based grasping based on 6D object pose estimation.
