Neural Radiance Fields (NeRFs).
NeRFs [1]} proved to be a successful popularizing in implicit volume representation and novel view synthesis. They define an implicit field and learn an MLP that predicts the RGB and density value of that 3D field given a set of posed images. NeRFs shoot rays on the volume and integrate the predictions to obtain individual pixel values. This formulation, however, has many drawbacks including large memory and compute requirements, inability to model dynamic scenes, posed image requirements, and the limitation to small 3D objects or rooms [2]}, [3]}, [4]}, [5]}, [6]}.
To address the speed limitation, PlenOctreeNeRF [4]} stores the precomputed RGB, density, and spherical harmonics in the 3D volume as an Octree data structure for fast inference. Plenoxels [8]} optimize the density and spherical harmonics on sparse voxels with a TV loss and perform ray marching for rendering from novel views. Similarly, INGP [9]} uses multi-resolution voxel hashing to perform a real-time rendering of radiance fields, demonstrating that the redundancy of the MLP in NeRFs. We build on these observations and build the SPARF dataset of sparse voxel radiance fields in order to facilitate learning on these SRFS as a 3D data structure instead of just a side outcome of a volumetric optimization.
<FIGURE>