A survey of photon mapping state-of-the-art research and future challenges <s> Summary <s> With the advancements in realtime ray tracing and new global illumination algorithms we are now able to render the most important illumination effects at interactive rates. One of the major remaining issues is the fast and efficient simulation of caustic illumination, such as e.g. the illumination from a car headlight. The photon mapping algorithm is a simple and robust approach that generates high-quality results and is the preferred algorithm for computing caustic illumination. However, photon mapping has a number of properties that make it rather slow on today's processors. Photon mapping has also been notoriously difficult to parallelize efficiently. ::: ::: In this paper, we present a detailed analysis of the performance issues of photon mapping together with significant performance improvements for all aspects of the photon mapping technique. The solution forms a complete framework for realtime photon mapping that efficiently combines realtime ray tracing, optimized and improved photon mapping algorithms, and efficient parallelization across commodity PCs. The presented system achieves realtime photon mapping performance of up to 22 frames per second on non-trivial scenes, while still allowing for interactively updating all aspects of the scene, including lighting, material properties, and geometry. <s> BIB001 </s> A survey of photon mapping state-of-the-art research and future challenges <s> Summary <s> We present a modified photon mapping algorithm capable of running entirely on GPUs. Our implementation uses breadth-first photon tracing to distribute photons using the GPU. The photons are stored in a grid-based photon map that is constructed directly on the graphics hardware using one of two methods: the first method is a multipass technique that uses fragment programs to directly sort the photons into a compact grid. The second method uses a single rendering pass combining a vertex program and the stencil buffer to route photons to their respective grid cells, producing an approximate photon map. We also present an efficient method for locating the nearest photons in the grid, which makes it possible to compute an estimate of the radiance at any surface location in the scene. Finally, we describe a breadth-first stochastic ray tracer that uses the photon map to simulate full global illumination directly on the graphics hardware. Our implementation demonstrates that current graphics hardware is capable of fully simulating global illumination with progressive, interactive feedback to the user. <s> BIB002 </s> A survey of photon mapping state-of-the-art research and future challenges <s> Summary <s> A number of popular global illumination algorithms uses density estimation to approximate indirect illumination. The density estimate is performed on finite points -- particles -- generated by a stochastic sampling of the scene. In the course of the sampling, particles, representing light, are stochastically emitted from the light sources and reflected around the scene. The sampling induces noise, which in turn is handled by the density estimate during the illumination reconstruction. Unfortunately, this noise reduction imposes a systematic error (bias), which is seen as a blurring of prominent illumination features. This is often not desirable as these may lose clarity or vanish altogether. We present an accurate method for reconstruction of indirect illumination with photon mapping. Instead of reconstructing illumination using classic density estimation on finite points, we use the correlation of light footprints, created by using Ray Differentials during the light pass. This procedure gives a high illumination accuracy, improving the trade-off between bias and variance considerable as compared to traditional particle tracing algorithms. In this way we preserve structures in indirect illumination. <s> BIB003 </s> A survey of photon mapping state-of-the-art research and future challenges <s> Summary <s> We present a photon mapping technique capable of computing high quality global illumination at interactive frame rates. By extending the concept of photon differentials to efficiently handle diffuse reflections, we generate footprints at all photon hit points. These enable illumination reconstruction by density estimation with variable kernel bandwidths without having to locate the k nearest photon hits first. Adapting an efficient BVH construction process for ray tracing acceleration, we build photon maps that enable the fast retrieval of all hits relevant to a shading point. We present a heuristic that automatically tunes the BVH build's termination criterion to the scene and illumination conditions. As all stages of the algorithm are highly parallelizable, we demonstrate an implementation using NVidia's CUDA manycore architecture running at interactive rates on a single GPU. Both light source and camera may be freely moved with global illumination fully recalculated in each frame. <s> BIB004 </s> A survey of photon mapping state-of-the-art research and future challenges <s> Summary <s> Blue noise sampling is an important component in many graphics applications, but existing techniques consider mainly the spatial positions of samples, making them less effective when handling problems with non-spatial features. Examples include biological distribution in which plant spacing is influenced by non-positional factors such as tree type and size, photon mapping in which photon flux and direction are not a direct function of the attached surface, and point cloud sampling in which the underlying surface is unknown a priori. These scenarios can benefit from blue noise sample distributions, but cannot be adequately handled by prior art. Inspired by bilateral filtering, we propose a bilateral blue noise sampling strategy. Our key idea is a general formulation to modulate the traditional sample distance measures, which are determined by sample position in spatial domain, with a similarity measure that considers arbitrary per sample attributes. This modulation leads to the notion of bilateral blue noise whose properties are influenced by not only the uniformity of the sample positions but also the similarity of the sample attributes. We describe how to incorporate our modulation into various sample analysis and synthesis methods, and demonstrate applications in object distribution, photon density estimation, and point cloud sub-sampling. <s> BIB005
All of the methods summarized above have introduced the special calculation on radiance estimation. These methods improve photon mapping in calculation to balance bias and noise. However, there is a general problem that noise affects the choice, and on a light smooth surface it is difficult to remove noise. Most of the improvements in this section are concerned with the details. It is still difficult to directly remove the influence of noise in the estimation. At the same time, improving the photon tracing process is a good idea. Most of the improved radiance estimation methods take more computational time than the standard photon mapping because of the accurate computation, e.g., the error estimate for density estimation at the shading point. In the ray map method, the density estimation with ray maps is 2.1-4.7 times slower than the direct visualization from photon maps. The diffusion based photon mapping method  ) uses 1.1-1.5 times computational time totally. The photon differentials method BIB003 costs more time in ray differentials' tracing and it demands 24 extra bytes per photon differential. As the core part of the photon mapping method, radiance estimation is worthy of further enhancement, and in recent years researchers have proposed to analyze the kernels used in photon density estimation. Photon relaxation as a pre-pass between photon tracing and illumination reconstruction removes noise directly rather than improves radiance estimation. This method uses a small number of emissions in photon tracing and a low bandwidth, which will lead to a significant reduction in rendering cost. However, the feature detection and maintenance are difficult, making this method currently available and suitable only for the caustic photon map with obvious features. The bilateral method BIB005 can render global illumination to a certain extent by adding arbitrary attributes to the relaxation. Fig. 7 shows the comparison with the bilateral blue noise method BIB005 , which adds color attributes. The results of the parameterized method are shown in Fig. 8 . However, the parameters are difficult to organize in global illumination. It should have a great capacity for having a great development in the aspects of feature keeping and global illumination. At the same time, the methods increase calculation in the relaxation step. Otherwise, calculation on radiance estimation is simple due to the small photon collection. The relaxation takes 1.5-5.7 times the time that the rendering step requires in the original method. The progressive photon relaxation can render a similarly accurate result by using the same amount of time as the progressive photon mapping in the relaxation step, i.e., about 1 h. However, the rendering time is less than 2 min. Therefore, the photon relaxation method is extremely suitable for a roaming scenario. Progressive photon mapping can increase the number of photon emissions by iteration without any increase in memory pressure. During the iteration process, the search radius is gradually attenuated, which can lead to noise reduction in photon distribution and bias elimination in radiance estimation. Although global illumination can be reconstructed by using the photon map directly in the PPM algorithm, thousands of iterations should be considered via iterative calculation to obtain an accurate result. The improvement on each step is conducive to reduce the number of iterations effectively. Improved PPM methods render a smooth noise-free result at the same time. The relation of the intermediate data is also very important in the progressive process. PPM is a variation of the photon mapping algorithm, and all aspects of the improvement on the photon mapping algorithm can be applied into the PPM algorithm. The adaptive photon tracing algorithm can handle illumination reconstruction in the scenes such as outdoor scenes, close-ups of a small part of an illuminated region, and illumination coming through a small gap. Selectively emitted photon can obtain better photon distribution for an accurate result. This kind of method not only expands the application field but also optimizes the rendering results. However, the enhanced tracing process introduces the cost of time and space. Because of the demand of the data for guiding photon tracing, the method generally includes multiple photon emissions. It is more convenient to apply it to the progressive photon mapping method. Global illumination algorithms are timeconsuming, and the most effective solution is parallel implementations on graphics hardware. The photon mapping algorithm has made some progress in the parallel field. However, as a two-pass method, ray tracing-based photon shooting and radiance estimation should be equally important parts in photon mapping. With the continuous development of computing hardware, the parallel efficiency of the algorithm has been further enhanced. At the moment, many methods are at interactive frame rates, and some ways have achieved the real-time rates. The grid-based method BIB002 has a 1.4-1.9 times speedup with Stencil Routing, while the KD-tree method  has a speedup of 8.9-11.1 times on the GPU. The interactive global photon mapping BIB004 ) maintains 3-4 frames/s. The distributed caustic photon mapping BIB001 achieves frame rates of up to 22 frames/s at video resolution (640 × 480). The appearance of different processing units and the use of clusters have diversified the methods. In particular, the vector calculation units should be considered to ensure the maximum utilization of the hardware's computing power.