In Figure REF , we show two of the test frames synthesized by all of the previously described methods.
The bottom row is the ground truth image, which has not been seen during training.
Column two and four visualize the per-pixel error of the rendered image in comparison to the ground truth.
A dark blue color means the error is small, while a brighter, reddish color indicates a large pixel error.
When comparing the result images, we can see that Synsin, NPBG, and our approach can synthesize the reference frame very well.
The output of Pulsar and NRW is slightly worse as indicated by the blurriness of the image.
A detailed look reveals that the renderings of our methods preserve color, brightness, and sharpness better than the other approaches.
The text on the train has slightly more contrast and the reflections on the playground slide are closer to the reference reflections.
However, we also note that a visual inspection is insufficient because especially the results of NPBG are very close to ours.
Therefore, we also provide a quantitative evaluation in Table REF .
This table shows the mean VGG loss [1]}, LPIPS loss [2]}, and peak signal-to-noise ratio (PSNR) over all test images.
All approaches were trained by minimizing the VGG loss.
On three of the four scenes, our method achieves the best results for all three quality metrics.
We believe that the reasoning behind these results are twofold.
First, Train, Playground, and Lighthouse, were captured outdoors on a sunny day.
The dynamic range is therefore large and a physically-based tonemapper is required to abstract color response and exposure changes.
The other approaches, which do not have a learnable tonemapper, encode brightness changes directly into the neural network.
Novel views are then synthesized incorrectly based on the training frames instead of the novel view's image settings.
The second reason why our systems outperforms the competitors is the input scene refinement during training.
As mentioned earlier, we optimize all available parameters including camera pose, camera model, and point position.
Hence, we can correct small errors of the initial reconstruction, which further improves sharpness due to a more accurate point projection.
These two reasons also explain why the difference in loss of our method to NPBG is small on the M60 scene.
This scene was captured indoors and the initial COLMAP reconstruction is already very precise.
The effect of our tonemapper, as well as, the scene refinement is therefore reduced.
However, our approach still achieves the best scores compared to the other point-based neural rendering approaches.
