A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> ``Bump'''' mapping is a variant of texture mapping where the texture information is used to alter the surface normal. Current techniques to pre-filter textures are all relying on the fact that the texture information can be linearly "factored out" of the shading equation, and therefore can be pre-averaged in some way. This is not the case with bump maps, and those techniques fail to filter them correctly. We propose here a technique to pre-filter bump maps by building a pyramid where each level stores distributions of normal vectors reconstructed from the distribution given by the underlying bump map. The distributions are represented as sums of a small number of Phong-like spreads of normal vectors. The technique, besides allowing an effective and smooth transition between a bump map and a single surface description, gives rise to the concept of a multiple surface, where each point of the single surface is characterized by more than one normal vector. This allows the description of visually complex surfaces by a trivial modification of current local illumination models. When a surface has an underlying microstructure, masking and self-shadowing are important factors in its appearance. Along with the filtering of normals we include the filtering of the masking and self-shadowing information. This is accomplished by computing the limiting angles of visibility and their variance along the two texture axes for the reconstructed distribution of normals. These techniques allow the modeling of any surface whose microstructure we can model geometrically. This includes complex but familiar surfaces such as anisotropic surfaces, many woven cloth, and stochastic surfaces. <s> BIB001 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> : ::: A new BRDF model is presented which can be viewed as an kind of intermediary model between empirism and theory. Main results of physics are observed (energy conservation, reciprocity rule, microfacet theory) and numerous phenomena involved in light reflection are accounted for, in a physically plausible way (incoherent and coherent reflection, spectrum modifications, anisotropy, self-shadowing, multiple surface and subsurface reflection, differences between homogeneous and heterogeneous materials). The model has been especially intended for computer graphics applications and therefore includes two main features: simplicity (a small number of intuitively understandable parameters controls the model) and efficiency (the formulation provides adequation to Monte-Carlo rendering techniques and/or hardware implementations). <s> BIB002 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> A growing number of real-time applications need graphics with photorealistic quality, especially in the field of training (virtual operation, driving and flightsimulation), but also in the areas of design or ergonomic research. We take a closer look at main deficiencies of today's real time graphics hardware and present solutions for several of the identified problems in the areas of antialiasing and texture-, bumpand reflection mapping. In the second part of the paper, a new method for antialiasing bump maps is explained in more detail. CR Categories and Subject Desscriptors: I.3.1 [Computer Graphics]: Hardware Architecture – Graphics Processors, Raster Display Devices; I.3.3 [Computer Graphics]: Picture/Image generation. Additional <s> BIB003 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> Complex repetitive scenes containing forests, foliage, grass, hair, or fur, are challenging for common modeling and rendering tools. The amount of data, the tediousness of modeling and animation tasks, and the cost of realistic rendering have caused such kind of scene to see only limited use even in high-end productions. The author describes how the use of volumetric textures is well suited to such scenes. These primitives can greatly simplify modeling and animation tasks. More importantly, they can be very efficiently rendered using ray tracing with few aliasing artifacts. The main idea, initially introduced by Kajiya and Kay (1989), is to represent a pattern of 3D geometry in a reference volume, that is tiled over an underlying surface much like a regular 2D texture. In our contribution, the mapping is independent of the mesh subdivision, the pattern can contain any kind of shape, and it is prefiltered at different scales as for MIP-mapping. Although the model encoding is volumetric, the rendering method differs greatly from traditional volume rendering. A volumetric texture only exists in the neighborhood of a surface, and the repeated instances (called texels) of the reference volume are spatially deformed. Furthermore, each voxel of the reference volume contains a key feature which controls the reflectance function that represents aggregate intravoxel geometry. This allows for ray tracing of highly complex scenes with very few aliasing artifacts, using a single ray per pixel (for the part of the scene using the volumetric texture representation). The major technical considerations of our method lie in the ray-path determination and in the specification of the reflectance function. <s> BIB004 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> Many researchers have been arguing that geometry, bump maps, and BRDFs present a hierarchy of detail that should be exploited for efficient rendering purposes. In practice however, this is often not possible due to inconsistencies in the illumination for these different levels of detail. For example, while bump map rendering often only considers direct illumination and no shadows, geometry-based rendering and BRDFs will mostly also respect shadowing effects, and in many cases even indirect illumination caused by scattered light. In this paper, we present an approach for overcoming these inconsistencies. We introduce an inexpensive method for consistently illuminating height fields and bump maps, as well as simulating BRDFs based on precomputed visibility information. With this information we can achieve a consistent illumination across the levels of detail. The method we propose offers significant performance benefits over existing algorithms for computing the light scattering in height fields and for computing a sampled BRDF representation using a virtual gonioreflectometer. The performance can be further improved by utilizing graphics hardware, which then also allows for interactive display. Finally, our method also approximates the changes in illumination when the height field, bump map, or BRDF is applied to a surface with a different curvature. <s> BIB005 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> Physically-based reflectance models typically represent light scattering as a function of surface geometry at the pixel level. With changes in viewing resolution, the geometry imaged within a pixel can undergo significant variations that result in changing reflectance characteristics. To address these transformations, we present a multiresolution reflectance framework based on microfacet normal distributions within a pixel over different scales. Since these distributions must be efficiently determined with respect to resolution, they are recorded at multiple resolution levels in mipmaps. The main contribution of this work is a real-time mipmap filtering technique for these distribution-based parameters that not only provides smooth reflectance transitions in scale, but also minimizes aliasing. With this multiresolution reflectance technique, our system can rapidly and accurately incorporate fine reflectance detail that is customarily disregarded in multiresolution rendering methods. <s> BIB006 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> Bump mapping produces realistic shading by perturbing normal vectors to a surface, but does not show the shadows that the bumps cast on nearby parts of the same surface. In this paper, these shadows are found from precomputed tables of horizon angles, listing, for each position entry, the elevation of the horizon in a sampled collection of directions. These tables are made for bumps on a standard flat surface, and then a transformation is developed so that the same tables can be used for an arbitrary curved parametrized surface patch. This necessitates a new method for scaling the bump size to the patch size. Incremental calculations can be used in a scan line algorithm for polygonal surface approximations. The errors in the bump shadows are discussed, as well as their anti-aliasing. (An earlier version of this article appeared as Max [10].) <s> BIB007 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> Shadow maps are a widely used shadowing technique in real time graphics. One major drawback of their use is that they cannot be filtered in the same way as color textures, typically leading to severe aliasing. This paper introduces variance shadow maps, a new real time shadowing algorithm. Instead of storing a single depth value, we store the mean and mean squared of a distribution of depths, from which we can efficiently compute the variance over any filter region. Using the variance, we derive an upper bound on the fraction of a shaded fragment that is occluded. We show that this bound often provides a good approximation to the true occlusion, and can be used as an approximate value for rendering. Our algorithm is simple to implement on current graphics processors and solves the problem of shadow map aliasing with minimal additional storage and computation. <s> BIB008 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> Filtering is critical for representing detail, such as color textures or normal maps, across a variety of scales. While MIP-mapping texture maps is commonplace, accurate normal map filtering remains a challenging problem because of nonlinearities in shading---we cannot simply average nearby surface normals. In this paper, we show analytically that normal map filtering can be formalized as a spherical convolution of the normal distribution function (NDF) and the BRDF, for a large class of common BRDFs such as Lambertian, microfacet and factored measurements. This theoretical result explains many previous filtering techniques as special cases, and leads to a generalization to a broader class of measured and analytic BRDFs. Our practical algorithms leverage a significant body of work that has studied lighting-BRDF convolution. We show how spherical harmonics can be used to filter the NDF for Lambertian and low-frequency specular BRDFs, while spherical von Mises-Fisher distributions can be used for high-frequency materials. <s> BIB009 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> The reflectance of a surface depends upon the resolution at which it is imaged. In this work, we propose to represent this resolution-dependent reflectance as a mixture of multiple conventional reflectance models and present a framework for efficiently rendering the reflectance effects of such mixture models over different resolutions. To rapidly determine reflectance at runtime with respect to resolution, we record the mixture model parameters at multiple resolution levels in mipmaps and propose a technique to minimize aliasing in the filtering of these mipmaps. This framework can be applied to several widely used parametric reflectance models and can be implemented in graphics hardware for real-time processing, using a presented hardware-accelerated technique for nonlinear filtering of mixture model parameters. In addition, shadowing and masking effects can be included into this framework to increase the realism of rendering. With this mixture model filtering and rendering framework, our system can efficiently render the fine reflectance detail that is customarily disregarded in conventional rendering methods. <s> BIB010 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> Rendering high-quality shadows in real-time is a challenging problem. Shadow mapping has proved to be an efficient solution, as it scales well for complex scenes. However, it suffers from aliasing problems. Filtering the shadow map alleviates aliasing, but unfortunately, native hardware-accelerated filtering cannot be applied, as the shadow test has to take place beforehand. ::: ::: We introduce a simple approach to shadow map filtering, by approximating the shadow test using an exponential function. This enables us to pre-filter the shadow map, which in turn allows for high quality hardware-accelerated filtering. Compared to previous filtering techniques, our technique is faster, consumes less memory and produces less artifacts. <s> BIB011 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> Shadow maps are commonly used in real-time rendering, but they cannot be filtered linearly like standard color, resulting in severe aliasing. Variance shadow maps resolve this problem by representing the depth distribution using moments, which can be linearly filtered. However, variance shadow maps suffer from "light bleeding" artifacts and require high-precision texture filtering hardware. ::: ::: We introduce layered variance shadow maps, which provide simultaneous solutions to both of these limitations. By partitioning the shadow map depth range into multiple layers, we eliminate all light bleeding between different layers. Using more layers increases the quality of the shadows at the expense of additional storage. Because each of these layers covers a reduced depth range, they can be stored in lower precision than would be required with typical variance shadow maps, enabling their use on a much wider range of graphics hardware. We also describe an iterative optimization algorithm to automatically position layers so as to maximize the utility of each. ::: ::: Our algorithm is easy to implement on current graphics hardware and provides an efficient, scalable solution to the problem of shadow map filtering. <s> BIB012 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> An ever-growing number of real-world computer vision applications require classification, segmentation, retrieval, or realistic rendering of genuine materials. However, the appearance of real materials dramatically changes with illumination and viewing variations. Thus, the only reliable representation of material visual properties requires capturing of its reflectance in as wide range of light and camera position combinations as possible. This is a principle of the recent most advanced texture representation, the bidirectional texture function (BTF). Multispectral BTF is a seven-dimensional function that depends on view and illumination directions as well as on planar texture coordinates. BTF is typically obtained by measurement of thousands of images covering many combinations of illumination and viewing angles. However, the large size of such measurements has prohibited their practical exploitation in any sensible application until recently. During the last few years, the first BTF measurement, compression, modeling, and rendering methods have emerged. In this paper, we categorize, critically survey, and psychophysically compare such approaches, which were published in this newly arising and important computer vision and graphics area. <s> BIB013 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> We present a real-time method for rendering global illumination effects from large area and environmental lights on dynamic height fields. In contrast to previous work, our method handles inter-reflections (indirect lighting) and non-diffuse surfaces. To reduce sampling, we construct one multi-resolution pyramid for height variation to compute direct shadows, and another pyramid for each indirect bounce of incident radiance to compute interreflections. The basic principle is to sample the points blocking direct light, or shedding indirect light, from coarser levels of the pyramid the farther away they are from a given receiver point. We unify the representation of visibility and indirect radiance at discrete azimuthal directions (i.e., as a function of a single elevation angle) using the concept of a "casting set" of visible points along this direction whose contributions are collected in the basis of normalized Legendre polynomials. This analytic representation is compact, requires no precomputation, and allows efficient integration to produce the spherical visibility and indirect radiance signals. Sub-sampling visibility and indirect radiance, while shading with full-resolution surface normals, further increases performance without introducing noticeable artifacts. Our method renders 512x512 height fields (> 500K triangles) at 36Hz. <s> BIB014 </s> A Survey of Nonlinear Prefiltering Methods for Efficient and Accurate Surface Shading <s> Hypotheses <s> We introduce Linear Efficient Antialiased Normal (LEAN) Mapping, a method for real-time filtering of specular highlights in bump and normal maps. The method evaluates bumps as part of a shading computation in the tangent space of the polygonal surface rather than in the tangent space of the individual bumps. By operating in a common tangent space, we are able to store information on the distribution of bump normals in a linearly-filterable form compatible with standard MIP and anisotropic filtering hardware. The necessary textures can be computed in a preprocess or generated in real-time on the GPU for time-varying normal maps. The method effectively captures the bloom in highlight shape as bumps become too small to see, and will even transform bump ridges into anisotropic shading. Unlike even more expensive methods, several layers can be combined cheaply during surface rendering, with per-pixel blending. Though the method is based on a modified Ward shading model, we show how to map between its parameters and those of a standard Blinn-Phong model for compatibility with existing art assets and pipelines, and demonstrate that both models produce equivalent results at the largest MIP levels. <s> BIB015
The linear terms in the prefiltering equation, such as the color map k x , can be linearly prefiltered if . the parallax offset, implicit in (6), is either null or can be handled separately-see Section 8, and . the parallax Jacobian nv=ng is quasiconstant or uncorrelated with k x , and thus cancels when moved outside both integrals in the fraction (6). If these two hypotheses hold, the average color becomes a simple weighted average of the color map k x over A: Note that when the coarse mesh curvature is small ðg % csteÞ, the above hypotheses hold if the view is nearly vertical (v % g), or if the surface bumps are small ðn % gÞ. The normal map prefiltering methods BIB001 ,  , BIB003 ,  , BIB004 ,  , BIB006 , BIB009 , BIB010 , BIB015 , presented below, start from Hypothesis 1. Then, (7) becomes: Then, the methods cited above assume that the 4D BRDF x ðl; vÞ has some symmetries allowing it to be represented with a 2D function f x ðhÞ of the half-vector h between l and v. More precisely, they assume, most often implicitly, that x has the following separable form BIB009 : where . rðl; vÞ is a function independent of x and n, which can represent, for instance, a Fresnel reflectance coefficient R þ ð1 À RÞð1 À vhÞ 5 BIB002 ; . ðl; vÞ is a function independent of x and n, which generally represents the half-vector h (this is not mandatory); and . f x ð Þ is a distribution function of microscale normals, such as microfacet normals in microfacet-based BRDF models. This microscale NDF, or NDF, depends on x, at least via its normal: we have f x ð Þ ¼ fðn x ; . . . ; Þ, with optional maps defining at each point the tangent vectors, the surface roughness, etc. This general form is true, for instance, for the Lambertian BRDF L ¼ 1 and for the Blinn-Phong  lobe BP ¼ ðnhÞ nl . However, this is not true for many BRDF models (see Section 3.3). The methods cited above can now be divided into direct and convolution methods. The horizon map prefiltering methods BIB001 , BIB010 start from Hypothesis 1. Then, the average visibility V A , given by BIB013 , becomes The visibility function V x is initially represented with a horizon map Â x ðÞ (see Fig. 3 ): where Â x ðÞ is the horizon angle in azimuth , and H is the Heaviside function. Several strategies are used to model the azimuthal dependence of Â x . For instance, using a fixed set of azimuthal directions 1 ; . . . ; n , with a piecewise linear interpolation BIB001 , BIB010 , or a B-spline interpolation BIB014 . It is also possible to fit the 2D curve ðcos Â x ðÞ; Þ in polar coordinates with an ellipse BIB005 . As said above, because the Heaviside function in (24) is nonlinear, the horizon map Â x ðÞ cannot be linearly prefiltered. To solve this problem, Fournier BIB001 and Tan et al. BIB010 reformulate this equation by using a horizon distribution function, noted p x ð; Þ and equal to ð À Â x ðÞÞ: The advantage of their reformulation is that the visibility function V x , which was a nonlinear function of the horizon map Â x , is now a linear function of the HDF p x . These distributions can thus be linearly prefiltered, using p A ð; Þ ¼ def R Aŵ ðxÞp x ð; Þdx, in order to compute the proportion of texels that are visible in the footprint A, V A ðvÞ ¼ def R Aŵ ðxÞV x ðvÞdx: This gives a method to prefilter the denominator of (23). The same principles can be used for the numerator, but then we get a 4D function p A ð; ; 0 ; 0 Þ equal to R Aŵ ðxÞp x ð; Þp x ð 0 ; 0 Þdx. Encoding a 4D function in each texel is very costly, so Fournier BIB001 and Tan et al. BIB010 approximate (23)  This allows them to use only 1D HDF distributions p A ð; i Þ, i ¼ 1 . . . n, while still being able to take into account some correlation effects between l and v in (23), discussed below. The problem is then to find compact, accurate, and linearly interpolable HDF representations. We now present the existing solutions to this problem. The shadow map prefiltering methods BIB008 ,  , BIB011 , BIB012 , presented below and illustrated in Fig. 5 , start from Hypothesis 1. Then, the average visibility U A , given by (9), becomes where we dropped l (for a punctual light source l only depends on x, and so U x ðlðxÞÞ can be noted U x ). When using shadow maps, U x is not stored explicitly, and so cannot be directly prefiltered with  . Instead, it is represented with a binary depth test applied to a depth map Z s . More precisely, by noting lðxÞ and ðxÞ the depth of x and its projection in the shadow map, respectively (see Fig. 4 ), U x is computed with and (31) becomes where A is the projection of the footprint A in the shadow map. The lðxÞ term in this equation prevents any prefiltering of the shadow map alone. To solve this problem, the shadow map prefiltering methods assume that lðxÞ can be approximated with its average l, which gives Still, as said above, because the Heaviside function in BIB007 and (33) is nonlinear, the depth map Z s cannot be linearly prefiltered. To solve this, one solution is to use depth distribution functions. We present this method and other solutions below. 