@inproceedings{
liang2024high,
title={High Fidelity Aggregated Planar Prior Assisted PatchMatch Multi-View Stereo},
author={Jie Liang and Rongjie Wang and Rui Peng and Zhe Zhang and Kaiqiang Xiong and Ronggang Wang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=gEI5nN0Fpx}
}
The quality of 3D models reconstructed by PatchMatch Multi-View Stereo remains a challenging problem due to unreliable photometric consistency in object boundaries and textureless areas. Since textureless areas usually exhibit strong planarity, previous methods used planar prior and significantly improved the reconstruction performance. However, their planar prior ignores the depth discontinuity at the object boundary, making the boundary inaccurate (not sharp). In addition, due to the unreliable planar models in large-scale low-textured objects, the reconstruction results are incomplete. To address the above issues, we introduce the segmentation generated from Segment Anything Model into PM pipelines for the first time. We use segmentation to determine whether the depth is continuous based on the characteristics of segmentation and depth sharing boundaries. Then we segment planes at object boundaries and enhance the consistency of planes in objects. Specifically, we construct $\textbf{Boundary Plane}$ that fits the object boundary and $\textbf{Object Plane}$ to increase consistency of planes in large-scale textureless objects. Finally, we use a probability graph model to calculate the $\textbf{Aggregated Prior guided by Multiple Planes}$ and embed it into the matching cost. The experimental results indicate that our method achieves state-of-the-art in terms of boundary sharpness on ETH3D. And it also significantly improves the completeness weakly textured objects. We also validated the generalization of our method on Tanks\&Temples.
High Fidelity Aggregated Planar Prior Assisted PatchMatch Multi-View Stereo
Jie Liang Rongjie Wang Rui Peng Zhe Zhang Kaiqiang Xiong Ronggang Wang