@inproceedings{
liu2024colvo,
title={Col{VO}: Colonoscopic Visual Odometry Considering Geometric and Photometric Consistency},
author={Ruyu Liu and Zhengzhe Liu and ZHANG HAOYU and Guodao Zhang and Jianhua Zhang and Bo Sun and Weiguo Sheng and Xiufeng Liu and Yaochu Jin},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=IfPaymV4oz}
}
Locating lesions is the primary goal of colonoscopy examinations. 3D perception techniques can enhance the accuracy of lesion localization by restoring 3D spatial information of the colon. However, existing methods focus on the local depth estimation of a single frame and neglect the precise global positioning of the colonoscope, thus failing to provide the accurate 3D location of lesions. The root causes of this shortfall are twofold: Firstly, existing methods treat colon depth and colonoscope pose estimation as independent tasks or design them as parallel sub-task branches. Secondly, the light source in the colon environment moves with the colonoscope, leading to brightness fluctuations among continuous frame images. To address these two issues, we propose ColVO, a novel deep learning-based Visual Odometry framework, which can continuously estimate colon depth and colonoscopic pose using two key components: a deep couple strategy for depth and pose estimation (DCDP) and a light consistent calibration mechanism (LCC). DCDP utilization of multimodal fusion and loss function constraints to couple depth and pose estimation modes ensures seamless alignment of geometric projections between consecutive frames. Meanwhile, LCC accounts for brightness variations by recalibrating the luminosity values of adjacent frames, enhancing ColVO's robustness. A comprehensive evaluation of ColVO on colon odometry benchmarks reveals its superiority over state-of-the-art methods in depth and pose estimation. We also demonstrate two valuable applications: immediate polyp localization and complete 3D reconstruction of the intestine. The code for ColVO is available at https://github.com/xxx/xxx.
ColVO: Colonoscopic Visual Odometry Considering Geometric and Photometric Consistency
Ruyu Liu Zhengzhe Liu ZHANG HAOYU Guodao Zhang Jianhua Zhang Bo Sun Weiguo Sheng Xiufeng Liu Yaochu Jin