There exist solutions that specifically target the problem of training NeRF when the input views are few, which are pixelNeRF [1]}, MVSNeRF [2]}, and DietNeRF [3]}.
PixelNeRF [1]} and MVSNeRF [2]} condition the NeRF's output through features computed with auxiliary deep networks.
During the rendering phase, a feature is processed for each sampled 3D point that is provided as input to the MLP for the conditioning.
Unlike these approaches, DVM does not rely on an auxiliary deep network, which has to be trained separately and ad-hoc, to process the known input images.
DietNeRF [3]} extends NeRF design by extracting feature vectors of the known images through a CLIP pre-trained image encoder.
DietNeRF training renders some random poses and processes them using the very same CLIP image encoder.
DietNeRF adds a loss component to minimise the cosine distance between CLIP feature vectors of the images rendered from random views and of ground-truth images.
Unlike DietNeRF, we do not need an additional consistency loss that is computed through the additional CLIP Vision Transformer to extract semantic representations of renderings, but we instead produce actual images from random viewpoints through view morphing.
