In this work, by leveraging the recent advances in 6D object pose estimation with deep neural networks [1]}, [2]}, [3]}, we present a novel self-supervised 6D object pose estimation system for manipulation (Fig. REF ). An active vision system is set up by mounting an RGB-D camera onto the hand of a robot manipulator. Given a set of objects in the workspace, we control the robot arm to collect images from different views. Meanwhile, the 6D poses of these objects are accurately initialized and then tracked based on the PoseRBPF framework [3]}, where we use the forward kinematics of the robot arm as a motion prior in the particle filtering. Consequently, our system is able to collect images of multiple objects from different views with accurate 6D pose annotations. We can use this data to perform self-supervised training of the deep neural networks in the system for object segmentation and pose estimation. After each iteration of the multi-view pose tracking, the robot manipulator grasps or pushes one object in the scene to generate a new scene, and then repeats the pose tracking process. In this way, our system is able to continuously generate new data for self-supervised learning, and improve its pose estimation and grasping performance in a life-long learning fashion.
<FIGURE>