Learning a domain-invariant representation for object manipulation in real-world environments with minimum supervision is a fundamental challenge in vision and robotics.
State-of-the-art learning systems rely on collecting large-scale datasets with interaction labels (e.g., success or failure) using multiple robots (e.g., KUKA robotic arms) to update the model parameters in parallel via end-to-end training [1]}.
The cost of deploying these learning systems to a new setting (e.g. a novel task under different environment) is very high and the efficiency is limited by the number of robots that can be deployed in parallel.
Therefore, existing learning based methods also use simulation data to alleviate the data collection problem. However, the existence of the sim-to-real gap imposes additional difficulty in transferring from simulation to the real world. Recent work mainly focuses on domain randomization by generating diverse configurations in simulation [2]}, [3]}, [4]}. Other approaches use pixel-level or feature-level domain adaptation [5]}, [6]}, [7]}. At best these methods still require a significant amount of unlabelled real world data, which is still costly and time-consuming to collect and may not transfer to a new task.
<FIGURE>