Nvidia introduced its Deep Learning Super Sampling 3 (DLSS 3) not too long ago, but the feature is already getting a major update. DLSS 3.5 will launch this fall, seemingly alongside Cyberpunk 2077: Phantom Liberty, and it adds something totally new to Nvidia’s storied RTX feature.
Ray Reconstruction is what’s new. At a high level, Ray Reconstruction enables greater levels of ray tracing quality without hurting your performance (in some cases, it can even improve performance). Nvidia is billing this as an image quality improvement over traditional ray tracing methods, not as a way to improve performance, however.
Nvidia says DLSS 3.5 will come first to Cyberpunk 2077: Phantom Liberty, Portal RTX, and Alan Wake 2 (which was recently delayed to late October). It lives alongside other DLSS features like Super Resolution and Frame Generation. However, unlike Frame Generation, Ray Reconstruction works on all RTX graphics cards. In addition, Nvidia tells me it will be a separate setting in the graphics menu, so you’ll be able to turn off Ray Reconstruction if you wish.
With DLSS 3.5, Nvidia has bunded four different features under the brand. Super Resolution, Deep Learning Anti-Aliasing, and Ray Reconstruction will work with all RTX GPUs in DLSS 3.5 games, while Frame Generation is exclusive to RTX 40-series graphics cards like the RTX 4070.
Like other DLSS features, Ray Reconstruction is powered by AI run on the Tensor cores on RTX graphics cards. The way it works is a little more in-depth than the other DLSS features we’ve seen, though.
How DLSS 3.5 works
Ray Reconstruction’s goal is to remove the detail that’s lost during the denoising process when ray tracing is turned on. When rendering a ray-traced game, there are only a few samples taken per pixel. This leads to noise, similar to a grainy film camera or noisy digital camera, due to the fact that some pixels don’t have any lighting information. The solution is to take more samples per pixel, but that’s too demanding for ray tracing to be possible while playing a game. The solution is denoising; cleaning up the image after you have enough samples.
Nvidia has a very good explainer on how denoising works and why it’s necessary, so I recommend watching that if you don’t understand the process. The story here isn’t what denoising is, but rather the shortcomings of it. With spatial denoising, the game engine uses nearby pixels to fill in the missing detail and clean up the image. The problem is that fine details don’t translate. If elements like a fence or lamp post aren’t sampled, they won’t show up, leading to a blurring effect you can spot in most games with ray tracing right now.
Nvidia also provided the example of temporal (timed-based) denoising, and how it can cause instability. This works by comparing two frames in order to fill in missing detail, but it can lead to strange outcomes. On fast-moving objects, you may see ghosting, for example, and on stationary objects, reflections may shimmer as the pool of samples changes.
There are other examples. Walls reflect ambient light, for instance, but if there aren’t enough samples on that wall, the reflection may not be as intense as it should be after denoising. You could go on and on, but Ray Reconstruction promises to make ray tracing more accurate by getting around denoising entirely.
It’s effectively an AI-powered denoiser. Instead of going through fixed algorithms with parameters set by the developer, Ray Reconstruction can recognize what type of lighting it’s dealing with — hard or soft reflections, global illumination, shadows, etc. — and adapt the denoising process to fit the scene you’re in.
Being able to recognize the scene is what seems to set Ray Reconstruction apart. Nvidia says this feature was trained on five times as much data as DLSS 3, allowing it to use more data from the game engine, recognize different lighting effects, and maintain detail that’s required for high-quality upscaling.
At least based on what Nvidia has shown, it seems to do all of that, as well. Nvidia demoed Portal RTX and Cyberpunk 2077, showcasing just how much more realistic the lighting conditions were with Ray Reconstruction turned on. It doesn’t hurt performance, either. Nvidia says that, in some cases, it can even improve performance slightly if the traditional denoising is particularly demanding.
As with all new features, we have to wait until DLSS 3.5 is in our hands to see if it holds up to the hype. If Nvidia’s materials are accurate, though, Ray Reconstruction looks mighty impressive. For now, all we can do is wait until fall when DLSS 3.5 releases.
