@inproceedings{
jang2023texture,
title={Texture synthesis for realistic-looking virtual colonoscopy using mask-aware transformer},
author={Seunghyun Jang and Yisak Kim and Dongheon Lee and Chang Min Park},
booktitle={Deep Generative Models for Health Workshop NeurIPS 2023},
year={2023},
url={https://openreview.net/forum?id=VsJyA3dhLr}
}
In virtual colonoscopy, computer vision techniques focus on depth estimation, photometric tracking, and simultaneous localization and mapping (SLAM). To narrow the domain gap between virtual and real colonoscopy data, it is necessary to utilize real-world data or employ realistic-looking virtual dataset. We introduce a texture synthesis and outpainting strategy using the Mask-aware-transformer. The method can generate textures for the inner surface suitable for virtual colonoscopy, including realistic-looking, controllable, and variety of synthesized textures. We generated RGB-D dataset employing the generated virtual colonoscopy, resulting in 9 video recordings. Each sequence was generated from distinct colon models, accumulating a total of 14,120 frames, paired with ground truth depth. Evaluating the generalizability across various datasets, the depth estimation model trained on our dataset exhibited superior transfer performance.
Texture synthesis for realistic-looking virtual colonoscopy using mask-aware transformer
Seunghyun Jang Yisak Kim Dongheon Lee Chang Min Park