
Although we observe the surrounding world only as a stream of 2D images, it is undeniably 3D.
The goal of recovering this underlying 3D from 2D observations has been a longstanding goal of computer vision. The task of inverting the rendering process that creates the 2D projections we observe by trying to construct the 3D world is known as Vision as Inverse Graphics (VIG) [1]}, [2]}, [3]}, [4]}. With the emergence of deep learning applications in computer graphics and the availability of 3D datasets, several approaches address the 3D generation task directly from 3D, without relying on appearance [5]}, [6]}, [7]}, [8]}, [9]}, [10]}. However, recent developments in differentiable rendering have refueled the VIG direction, which facilitates using gradients of the rendering process to optimize for the underlying 3D setup based on image observations [11]}, [12]}, [13]}, [14]}, [15]}, [16]}, [17]}, [18]}, [19]}, [20]}, [21]}, [22]}, [23]}.
More specifically, Neural Radiance Fields (NeRFs) [22]}, [25]}, [26]}, [27]} show impressive performance on novel view synthesis by optimizing volumetric radiance fields on a large number of posed multi-view images.
<FIGURE><TABLE>