Inverse rendering and differentiable rendering have been a topic of research for some time. However, major breakthroughs have only been made in recent years due to improved hardware and advancements in deep learning [1]}.
Its application can be challenging though:
Traditional triangle rasterization with depth-testing has no analytically correct spatial derivative [2]}, [1]}.
Available systems therefore either approximate the gradient [2]}, [5]}, [6]}, [7]} or approximate the rendering itself using alpha blending along edges [8]}, [9]}, [10]}.
Volume raycasting on the other hand is differentiable by accumulating all voxel values along the ray  [1]}.
This has been used by multiple authors to build volumetric reconstruction systems [12]}, [13]}, [14]} or predict the signed distance field of an object [15]}.
Instead of a voxel grid, an implicit function can also be used inside a differentiable volumetric raycasting framework [16]}, [17]}, [18]}, [19]}.
Mildenhall et al. show with Neural Radiance Fields (NeRF) that a deep neural network can be trained by volumetric raycasting to store the view-dependent radiance of an object.
Due to the impressive results of NeRF, multiple extensions and improvements have been published in the following years [20]}, [21]}, [22]}, [23]}.
Inverse rendering has also been proposed for point cloud rasterization [1]}, [25]}.
The spatial gradients of the points can be approximated in different ways.
This includes gradient computation along silhouettes [26]}, gradients for Gaussian splatting [27]}, [28]}, [29]}, and gradient approximation using a temporary 3D volume [30]}.
A different approach is taken by Lassner et al. who render the points as small spheres instead of splats [31]}.
Our differentiable point rendering approach is similar to the differentiable splatting techniques of [27]}, [29]}.
However we render only one pixel per point, which allows our approach to be multiple magnitudes more efficient than competing methods [31]}.
