Transparent objects need to be perceived before being manipulated.
Lai et al. [1]} and Khaing et al. [2]} developed CNN models to detect transparent objects from RGB images.
Xie et al. [3]} proposed a deep segmentation model that achieved state-of-the-art segmentation accuracy.
ClearGrasp [4]} employed depth completion for use with pose estimation on robotic grasping tasks, where they trained three DeepLabv3+ [5]} models to perform image segmentation, surface normal estimation, and boundary segmentation.
Follow-on studies developed different approaches for depth completion, including implicit functions [6]}, NeRF features [7]}, combined point cloud and depth features [8]}, adversarial learning [9]}, multi-view geometry [10]}, and RGB image completion [11]}.
Without completing depth, Weng et al. [12]} proposed a method to transfer the learned grasping policy from the RGB domain to the raw sensor depth domain.
For instance-level pose estimation, Xu et al. [13]} utilized segmentation, surface normal, and image coordinate UV-map as input to a network similar to [14]} that can estimate 6 DOF object pose.
Keypose [15]} was proposed to estimate 2D keypoints and regress object poses from stereo images using triangulation.
For other special sensors, Xu et al. [16]} used light-field images to do segmentation using a graph-cut-based approach.
Kalra et al. [17]} trained Mask R-CNN [18]} using polarization images as input to outperform the baseline that was trained on only RGB images by a large margin.
Zhou et al. [19]}, [20]}, [21]} employed light-field images to learn features for robotic grasping and object pose estimation.
Along with the proposed methods, massive datasets, across different sensors and both synthetic and real-world domains, have been collected and made public for various related tasks [3]}, [4]}, [15]}, [21]}, [17]}, [27]}, [6]}, [8]}, [11]}, [31]}.
Compared with these previous works, and to the best of our knowledge we propose the first category-level pose estimation approach for transparent objects.
Notably, the proposed approach provides reliable 6D pose and scale estimates across instances with similar shapes.
