Methods on sim-to-real transfer aim at training neural networks on simulated data with the goal to operate on real data at inference time. This is also known as domain adaptation, where a model is trained with data points of a source domain to then generalize to a target domain [1]}, [2]}. As simulated data can be generated efficiently with ground-truth labels, a number of methods focus on sim-to-real transfer by enhancing or synthesizing images [3]}, [4]}, [5]}. More recently, Bousmalis et al. [6]} introduced a generative adversarial network pipeline to enhance simulated data to significantly reduce the number of real-world data samples for a grasping task. Fang et al. [7]} propose a multi-task domain adaptation framework composed of three grasp prediction towers for instance grasping in cluttered scenes. Finally, James et al. [8]} introduce a two-stage generator pipeline to translate simulated and real images into a canonical representation to realize sim-to-real transfer. While existing work mostly focuses on introducing architectures to operate on images, we argue that point clouds of objects serve as an effective representation to facilitate sim-to-real transfer.
<FIGURE>