@inproceedings{
liu2024dgsenhancer,
title={3{DGS}-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors},
author={Xi Liu and Chaoyi Zhou and Siyu Huang},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=P4s6FUpCbG}
}
Novel-view synthesis aims to generate novel views of a scene from multiple input
images or videos, and recent advancements like 3D Gaussian splatting (3DGS)
have achieved notable success in producing photorealistic renderings with efficient
pipelines. However, generating high-quality novel views under challenging settings,
such as sparse input views, remains difficult due to insufficient information in
under-sampled areas, often resulting in noticeable artifacts. This paper presents
3DGS-Enhancer, a novel pipeline for enhancing the representation quality of
3DGS representations. We leverage 2D video diffusion priors to address the
challenging 3D view consistency problem, reformulating it as achieving temporal
consistency within a video generation process. 3DGS-Enhancer restores view-
consistent latent features of rendered novel views and integrates them with the
input views through a spatial-temporal decoder. The enhanced views are then
used to fine-tune the initial 3DGS model, significantly improving its rendering
performance. Extensive experiments on large-scale datasets of unbounded scenes
demonstrate that 3DGS-Enhancer yields superior reconstruction performance and
high-fidelity rendering results compared to state-of-the-art methods. The project
webpage is https://xiliu8006.github.io/3DGS-Enhancer-project.
3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors
Xi Liu Chaoyi Zhou Siyu Huang