Various subsequent works addressed rendering speed [1]}, [2]}, [3]}, training size requirement [4]}, [5]}, [6]}, or pose requirements [7]}, [8]} for NeRF. The seminal work of Plenoxels showed that the MLP network is not necessary for quick optimization and volumetric rendering of the radiance fields. However, the current paradigm is still an optimization paradigm where a single scene representation is optimized without any generalization to new scenes/objects. In this work, we treat Sparse Radiance Fields (SRFs) as a 3D data structure and try to learn a generative model (dubbed SuRFNet) on the distribution of sparse-voxel radiance fields conditioned on a few images to generalize to unseen 3D shapes (see Figure REF ).
