Ray tracing is a lighting technique that brings an extra level of realism to games. It emulates the way light reflects and refracts in the real world, providing a more believable environment than what’s typically seen using the static lighting in more traditional games. But what is ray tracing, exactly? And more importantly, how does it work? A good graphics card can use ray tracing to enhance immersion, but not all GPUs can handle this technique. Read on to decide if ray tracing is essential to your gaming experience and if it justifies spending hundreds on an upgraded GPU. Virtual photons To understand just how ray tracing’s revolutionary lighting system works, we need to step back and understand how games previously rendered light and what needs to be emulated for a photorealistic experience. Games without ray tracing rely on static “baked in” lighting. Developers place light sources within an environment that emit light evenly across any given view. Moreover, virtual models like NPCs and objects don’t contain any information about any other model, requiring the GPU to calculate light behavior during the rendering process. Surface textures can reflect light to mimic shininess, but only light emitted from a static source. Take the comparison of reflections in GTA V below as an example. Overall, the GPU’s evolution has helped this process become more realistic in appearance over the years, but games still aren’t photorealistic in terms of real-world reflections, refractions, and general illumination. To accomplish this, the GPU needs the ability to trace virtual rays of light. In the real world, visible light is a small part of the electromagnetic radiation family perceived by the human eye. It contains photons that behave both as a particle and as a wave. Photons have no real size or shape — they can only be created or destroyed. That said, light could be identified as a stream of photons. The more photons you have, the brighter the perceived light. Reflection occurs when photons bounce off a surface. Refraction occurs when photons — which travel in a straight line — pass through a transparent substance and the line is redirected, or “bent.” Destroyed photons can be perceived as “absorbed.” Ray tracing in games attempts to emulate the way light works in the real world. It traces the path of simulated light by tracking millions of virtual photons. The brighter the light, the more virtual photons the GPU must calculate, and the more surfaces it will reflect, refract, and scatter off and from. The process isn’t anything new. CGI has used ray tracing for decades, though the process required farms of computers in the early days to generate a full movie given a single frame could take hours or even days to render. Now home PCs can emulate ray-traced graphics in real time, leveraging hardware acceleration and clever lighting tricks to limit the number of rays to a manageable number. But here’s the real eye-opener. Like any movie or TV show, scenes in CGI animation are typically “shot” using different angles. For each frame, you can move a camera to capture the action, zoom in, zoom out, or pan an entire area. And like animation, you must manipulate everything on a frame-by-frame basis to emulate movement. Piece all the footage together and you have a flowing story. In games, you control a single camera that’s always in motion and always changing the viewpoint, especially in fast-paced games. In both CGI and ray-traced games, the GPU not only must calculate how light reflects and refracts in any given scene, but it also must calculate how it’s captured by the lens — your viewpoint. For games, that’s an enormous amount of computational work for a single PC or console. Unfortunately, we still don’t have consumer-level PCs that can truly render ray-traced graphics at high framerates. Instead, we now have hardware that can cheat effectively. Let’s get real Ray tracing’s fundamental similarity to real life makes it an extremely realistic 3D rendering technique, even making blocky games like Minecraft look near photo-realistic in the right conditions. There’s just one problem: It’s extremely hard to simulate. Recreating the way light works in the real world is complicated and resource-intensive, requiring masses of computing power. That’s why existing ray-tracing options in games, like Nvidia’s RTX-driven ray tracing, aren’t true to life. They’re not true ray tracing, whereby every point of light is simulated. Instead, the GPU “cheats” by using several smart approximations to deliver something close to the same visual effect, but without being quite as taxing on the hardware. This will likely change in future GPU generations, but for now, this is a step in the right direction. Most ray tracing games now use a combination of traditional lighting techniques, typically called rasterization, and ray tracing on specific surfaces such as reflective puddles and metalwork. Battlefield V is a great example of that. You see the reflection of troops in water, the reflection of terrain on airplanes, and the reflection of explosions across a car’s paint. It’s possible to show reflections in modern 3D engines, but not at the level of detail shown in games like Battlefield V when ray tracing is enabled. Ray tracing can also be leveraged for shadows to make them more dynamic and realistic looking. You’ll see that used to great effect in Shadow of the Tomb Raider. Ray-traced lighting can create much more realistic shadows in dark and bright scenes, with softer edges and greater definition. Achieving that look without ray tracing is extraordinarily hard. Developers can only fake it through careful, controlled use of preset, static light sources. Placing all these “stage lights” takes a lot of time and effort — and even then, the result isn’t quite right. Some games go the whole hog and use ray tracing for global illumination, effectively ray-tracing an entire scene. But that’s the most computationally expensive and needs the most powerful of modern graphics cards to run effectively. Metro Exodus uses this technique but the implementation isn’t perfect. Because of that, half-measures like only ray-tracing shadows or reflective surfaces are popular. Other games leverage Nvidia technologies like denoising and Deep Learning Super Sampling to improve performance and to cover up some of the visual hiccups that occur from rendering fewer rays than would be necessary to create a truly ray-traced scene. Those are still reserved for pre-rendered screenshots and movies where high-powered servers can spend days rendering single frames. The hardware behind the rays To handle even these relatively modest implementations of ray tracing, Nvidia’s RTX 20-series graphics cards introduced hardware specifically built for ray tracing. Nvidia’s Turing architecture — featured on 20-series GPUs — introduced RT cores alongside Nvidia’s CUDA and Tensor cores. RT cores are solely there to handle real-time ray tracing. In Turing cards, the RT cores performed decently, but it wasn’t until the recent Ampere launch where we saw them shine. Nvidia released a breakdown of generating a single frame of Metro Exodus, where it showed how the rendering pipeline is laid out and how it is affected by ray tracing. While an RTX 2080 and GTX 1080 Ti might be roughly comparable in performance for non-ray-traced games, when ray tracing is applied to a scene, it can take much longer for the 1080 Ti, without the dedicated RT cores, to generate the same image. The dedicated RT cores in RTX 20-series GPUs were a big selling point, but they didn’t quite deliver the performance Nvidia suggested. Even last-gen’s 2080 Ti struggled in supported ray tracing titles upon launch. The new RTX 3080 and 3090 feature newer RT cores, however, and the performance improvement is clear. Not only are these cards faster than their last-gen counterparts, but the new RT cores are faster, too. In many ways, the RTX 30-series cards feel like the GPUs Nvidia was promising with RTX in the first place. Nvidia’s ray tracing method isn’t the only option available, however. There are also Reshade “path tracing” post-processing effects that deliver comparable visuals without anything like the same performance hit. AMD has options for ray tracing now, too, which we’ll get to next.<doc-sep>The developers behind Shadow of the Tomb Raider responded to concerns about the game running poorly on Nvidia’s just-revealed GeForce RTX 2080 Ti graphics card with ray tracing enabled. While we were dazzled by the ray tracing support in the upcoming game during a demo showcased on Monday, August 20, the developers say it’s still a work in progress and the feature will be made available in a post-launch update. “The Nvidia Ray Tracing technology currently being shown in Shadow of the Tomb Raider is an early work in progress version,” the developers state via Twitter. “As a result, different areas of the game have received different levels of polish while we work toward complete implementation of this new technology.” During Nvidia’s big GeForce RTX 20 Series reveal, we viewed one specific crowded environment pulled from the game to show how ray tracing effects the way Nvidia’s hardware handles the area. The developers merely toggled the feature on and off to illustrate how ray tracing renders shadows, reflections and area illumination in real time differently than current rendering techniques. Naturally, Nvidia and Eidos provided demos for the press to view after the presentation. The machine running the Tomb Raider demo included the just-revealed RTX 2080 TI while the game itself was only set to a 1,920 x 1,080 resolution. With FRAPS running in the background, ray tracing turned on and the game set to its highest detail settings, the framerate would fluctuate between 30 and 70 frames per second. You can view an off-screen video capture of the gameplay right here. That fluctuating framerate for a game running in Full HD apparently generated concern. After all, the RTX 2080 Ti will have a starting price of $1,000 when it arrives later this month. It packs 4,352 CUDA cores with a maximum speed of 1,545MHz and 11GB of onboard GDDR6 memory clocked at 14Gbps. The framerate should be way higher than that, right? In this case, no. As the developer points out, there is still work that needs to be done to get the ray tracing portion running on all cylinders. Despite the underlying hardware support from the card’s dedicated RT Cores and AI-focused Tensor Cores, optimizations still need to be made on the software side, hence the demo’s current framerate. The big selling point with Nvidia’s new RTX 20 Series cards is the real-time ray tracing aspect. Typically, this rendering process takes high amounts of computing power and long wait times to render each frame, but the resulting scene can be photo-realistic. While a single graphics card still can’t produce that level of output in real time, Nvidia provides a “cheat” that involves artificial intelligence so we’re not waiting another 10 years. Given real-time ray tracing is new to PC gaming, it’s a work in progress. The cards aren’t available nor is the content to support their ray tracing aspect. The best way to gauge performance is to wait and see both in action when they’re actually available rather than jump to conclusions based on pre-release demos. Editors' Recommendations<doc-sep>The announcement today (August 20) of Nvidia's new GPUs with integrated acceleration of raytracing makes Microsoft's plans for DirectX even more relevant. Raytracing gives developers access to a wide range of effects that the current mainstream approach (rasterization) handles poorly. Shadows, reflections, and glass are all set to look much more realistic. To outline the benefits, we're resurfacing this raytracing explainer that originally ran in March 2018. At GDC, Microsoft announced a new feature for DirectX 12: DirectX Raytracing (DXR). The new API offers hardware-accelerated raytracing to DirectX applications, ushering in a new era of games with more realistic lighting, shadows, and materials. One day, this technology could enable the kinds of photorealistic imagery that we've become accustomed to in Hollywood blockbusters. Whatever GPU you have, whether it be Nvidia's monstrous $3,000 Titan V or the little integrated thing in your $35 Raspberry Pi, the basic principles are the same; indeed, while many aspects of GPUs have changed since 3D accelerators first emerged in the 1990s, they've all been based on a common principle: rasterization. Here’s how things are done today A 3D scene is made up of several elements: there are the 3D models, built from triangles with textures applied to each triangle; there are lights, illuminating the objects; and there's a viewport or camera, looking at the scene from a particular position. Essentially, in rasterization, the camera represents a raster pixel grid (hence, rasterization). For each triangle in the scene, the rasterization engine determines if the triangle overlaps each pixel. If it does, that triangle's color is applied to the pixel. The rasterization engine works from the furthermost triangles and moves closer to the camera, so if one triangle obscures another, the pixel will be colored first by the back triangle, then by the one in front of it. This back-to-front, overwriting-based process is why rasterization is also known as the painter's algorithm; bring to mind the fabulous Bob Ross, first laying down the sky far in the distance, then overwriting it with mountains, then the happy little trees, then perhaps a small building or a broken-down fence, and finally the foliage and plants closest to us. Much of the development of the GPU has focused on optimizing this process by cutting out the amount that has to be drawn. For example, objects that are outside the field of view of the viewport can be ignored; their triangles can never be visible through the raster grid. The parts of objects that lie behind other objects can also be ignored; their contribution to a given pixel will be overwritten by a pixel that's closer to the camera, so there's no point even calculating what their contribution would be. Advertisement GPUs have become more complicated over the last two decades, with vertex shaders processing the individual triangles, geometry shaders to produce new triangles, pixel shaders modifying the post-rasterization pixels, and compute shaders to perform physics and other calculations. But the basic model of operation has stayed the same. Rasterization has the advantage that it can be done fast; the optimizations that skip triangles that are hidden are effective, greatly reducing the work the GPU has to do, and rasterization also allows the GPU to stream through the triangles one at a time rather than having to hold them all in memory at the same time. But rasterization has problems that limit its visual fidelity. For example, an object that lies outside the camera's field of view can't be seen, so it will be skipped by the GPU. However, that object could still cast a shadow within the scene. Or it might be visible from a reflective surface within the scene. Even within a scene, white light that's bounced off a bright red object will tend to color everything struck by that light in red; this effect isn't found in rasterized images. Some of these deficits can be patched up with techniques such as shadow mapping (which allows objects from outside the field of view to cast shadows within it), but the result is that rasterized images always end up looking different from the real world. Fundamentally, rasterization doesn't work the way that human vision works. We don't emanate a grid of beams from our eyes and see which objects those beams intersect. Rather, light from the world is reflected into our eyes. It may bounce off multiple objects on the way, and as it passes through transparent objects, it can be bent in complex ways. Enter raytracing Raytracing is a technique for producing computer graphics that more closely mimics this physical process. Depending on the exact algorithm used, rays of light are projected either from each light source or from each raster pixel; they bounce around the objects in the scene until they strike (depending on direction) either the camera or a light source. Projecting rays from each pixel is less computationally intensive, but projecting from the light sources produces higher quality images that replicate certain optical effects accurately. Raytracing can produce substantially more accurate images; advanced raytracing engines can yield photorealistic imagery. This is why raytracing is used for rendering graphics in movies: computer images can be integrated with live-action footage without looking out of place or artificial. Advertisement But raytracing has a problem: it is enormously computationally intensive. Rasterization has been extensively optimized to try to restrict the amount of work that the GPU must do; in raytracing, all that effort is for naught, as potentially any object could contribute shadows or reflections to a scene. Raytracing has to simulate millions of beams of light, and some of that simulation may be wasted, reflected off-screen, or hidden behind something else. This isn't a problem for films; the companies making movie graphics will spend hours rendering individual frames, with vast server farms used to process each image in parallel. But it's a huge problem for games, where you only get 16 milliseconds to draw each frame (for 60 frames per second) or even less for VR. However, modern GPUs are very fast these days. And while they're not fast enough—yet—to raytrace highly complex games with high refresh rates, they do have enough compute resources that they can be used to do some bits of raytracing. This is where DXR comes in. DXR is a raytracing API that extends the existing rasterization-based Direct3D 12 API. The 3D scene is arranged in a manner that's amenable to raytracing, and with the DXR API, developers can produce rays and trace their path through the scene. DXR also defines new shader types that allow programs to interact with the rays as they interact with objects in the scene. Because of the performance demands, Microsoft expects that DXR will be used, at least for the time being, to fill in some of the things that raytracing does very well and that rasterization doesn't: things like reflections and shadows. DXR should make these things look more realistic. We might also see simple, stylized games using raytracing exclusively. The company says that it has been working on DXR for close to a year, and Nvidia in particular has plenty to say about the matter. Nvidia has its own raytracing engine designed for its Volta architecture (though currently, the only video card shipping with Volta is the Titan V, so the application of this is likely limited). When run on a Volta system, DXR applications will automatically use that engine. Microsoft says vaguely that DXR will work with hardware that's currently on the market and that it will have a fallback layer that will let developers experiment with DXR on whatever hardware they have. Should DXR be widely adopted, we can imagine that future hardware may contain features tailored to the needs of raytracing. On the software side, Microsoft says that EA (with the Frostbite engine used in the Battlefield series), Epic (with the Unreal engine), Unity 3D (with the Unity engine), and others will have DXR support soon.<doc-sep>
Ray tracing is a term you’re going to hear a lot now that Nvidia has announced professional and consumer graphics cards that use this technique to produce some of the most life-like simulations possible in games and other animations. So, what is ray tracing exactly, and how does it differ from current graphics rendering techniques? The oversimplified answer is that ray tracing models the behavior of light in real time as it intersects objects in a scene. It’s a feature that could lead to spectacular new graphics, but has been very hard to pull off because of the computational requirements. But Nvidia is tackling several issues facing ray tracing with a new graphics architecture known as Turing. Turing is the biggest jump in graphics since CUDA arrived in 2006 First, it’s tackling the problem of ushering in the next generation of computer graphics. Ray tracing is only one of many rendering techniques, but it’s where Nvidia is pushing hard because it’s especially suited for adding realistic, real-time lighting and effects. The second issue is computational cost: the best Turing card for professional production costs $10,000, but it was even costlier to use ray tracing before. What’s new here is Nvidia is ready to bring ray-tracing tech to consumer-level GPUs; that hasn’t been done before. Nvidia’s current graphics tech — and most of the industry’s — simulates light and how light behaves in a given scene in a much simpler way, using something called rasterization. Like a painter painting layers upon a canvas, objects are rendered from back to front, so those in the front obscure the objects in the back. This makes it hard to model a mirror, for example, because rasterization techniques can’t track and model light itself. It’s used often in real-time scenes because current-generation hardware can’t keep up with the demands of simulating a complex scene in motion for something that requires it (say, a game or 3D animation). Ray tracing models the behavior of light as it intersects objects This next generation of light simulations can model light in much more detail, without as much computational cost as before. Ray tracing models the behavior of light as it intersects with surfaces, materials, and moving objects. A path of light that travels through a scene can be rendered more intricately now. With ray tracing, you could simulate how rays of light interact with objects, producing realistic reflection, refraction, and scattering effects in real time. Ray tracing can even detect and render mirrors, refract glass, figure out where light in a scene originates, and even determine the color of light as it passes through objects. On paper, it sounds like the rendering technique is almost too realistic, but it’s not exactly new. Ray tracing has been used in the professional industry for years. It’s already been employed in popular media, like Pixar’s Monsters University and Marvel’s Iron Man movies. What makes the announcement exciting is it will finally be available for consumer hardware; a feat that was too difficult and expensive before. Here’s an example of ray tracing working in real time during a Star Wars demo, using Nvidia’s professional Volta RTX graphics cards: What kept Nvidia back from accomplishing this sort of feat for consumers is that ray tracing requires an incredible amount of computational power. Nvidia CEO Jensen Huang said this is “the single greatest leap that we have ever made in one generation.” It makes sense, considering the new Turing architecture used in Nvidia’s new GPUs is designed to address the processing issue. Dedicated ray-tracing cores work in conjunction with Tensor Cores — they use AI to infer the “real-time” part — to produce simulations six times faster than the previous Pascal platform (GTX 1080Ti, etc.). Ray tracing has use everywhere in computer graphics It’s a big jump in graphics tech and is exciting when you think about the studios and individuals that will find new uses for ray tracing in animation, games, and science simulations. Even if the new hardware Nvidia produces will only be available for desktops first, laptops with ray-tracing tech will start coming to market next year, so they’re not too far off. In fact, certain upcoming games like Metro: Exodus already have Nvidia RTX demos showing off real-time ray tracing doing its thing.