@inproceedings{
zhou2024gaussian,
title={Gaussian Splatting With Neural Basis Extension},
author={Zhi Zhou and Junke Zhu and ZhangJin Huang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=Yx2HjWZcZI}
}
The 3D Gaussian Splatting(3D-GS) method has recently sparked a new revolution in novel view synthesis with its remarkable visual effects and fast rendering speed. However, its reliance on simple spherical harmonics for color representation leads to subpar performance in complex scenes, struggling with effects like specular highlights, light refraction, etc. Also, 3D-GS adopts a periodic split strategy, significantly increasing the model's disk space and hindering rendering efficiency. To tackle these challenges, we introduce Gaussian Splatting with Neural Basis Extension (GSNB), a novel approach that substantially improves the performance of 3D-GS in demanding scenes while reducing storage consumption. Drawing inspiration from basis function, GSNB employs a light-weight MLP to share feature coefficients with spherical harmonics and extends the color calculation of 3D Gaussians for more precise visual effect modeling. This combination enables GSNB to achieve impressive results in scenes with challenging lighting and reflection conditions. Moreover, GSNB utilizes pre-computation to bake the network's output, thereby alleviating inference workload and subsequent speed loss. Furthermore, to leverage the capabilities of Neural Basis Extension and eliminate redundant Gaussians, we propose a new importance criterion to prune the converged Gaussian model and obtain a more compact representation through re-optimization. Experimental results demonstrate that our method delivers high-quality rendering in most scenarios and effectively reduces redundant Gaussians without compromising rendering speed. Our code and real-time demos will be released soon.
Gaussian Splatting With Neural Basis Extension
Zhi Zhou Junke Zhu ZhangJin Huang