A key advantage of using simulated data for computer vision is that it is much cheaper to obtain expert annotations. Annotated Objaverse objects can be rendered into images, allowing them to serve as a rich source of additional data that can be used to enhance model performance on 2D computer vision tasks. As a proof-of-concept demonstrating the effectiveness of this approach, we use segmented data from Objaverse objects as auxiliary labels for training models on the LVIS dataset for Large Scale Instance Segmentation [1]}. The LVIS dataset contains instance segmentation masks for 1200 object categories that occur throughout a set of 164k images. Recognition is especially challenging in this task due to the long tail of the object category distribution in this dataset. LVIS categories only contain an average 9 instances across the dataset, so training on simulated data is a promising approach for overcoming the challenges of learning in this low-sample regime.
