Investigate: DLSS Ray Reconstruction / FSR ray regeneration as an optional neural denoise + upscale path

sky_line commented

2026-10-19 14:36:54 +00:00

Owner

Investigate (later, not now) vendor neural denoise + upscale for ray-traced lighting as an optional upgrade path on supported hardware: NVIDIA DLSS Ray Reconstruction (DLSS 3.5+, via Streamline) and AMD FSR "Redstone" ray regeneration. Our own denoiser (#85) stays the baseline for every other GPU and for anyone who turns these off -- this is an add-on like the existing upscaler options, never a requirement.

What it is

One trained network replaces the hand-written per-signal denoisers and TAA / the upscaler. Input each frame: the un-denoised lit frame plus guide buffers (depth, motion vectors, normals, roughness, diffuse + specular albedo, specular hit distance / specular motion). Output: the denoised, upscaled, anti-aliased frame. Better detail retention and temporal stability than hand-tuned filters, especially for reflections and many-light ReSTIR noise (#869).

Questions to answer

Licensing: can an LGPL viewer ship / load the vendor binaries (DLSS SDK + Streamline; FidelityFX / FSR ML runtime)? NRD and RTXDI were already ruled out on licence grounds -- check these separately and document the answer.
Platform support: Vulkan and Linux availability for each (DLSS RR on Linux/Vulkan; Redstone ray regeneration beyond DX12). Windows-only would still be worth it if licensing allows.
Hardware: DLSS RR = RTX (tensor cores); ray regeneration = RDNA 4. Intel: no equivalent known -- recheck.
Integration shape: which passes it replaces when active (RT shadow denoiser, god-ray denoiser, future radiance denoiser, TAA / FSR), and what guide buffers we must produce that we don't today.
Interaction with our signals: does it want the RT shadow mask raw or applied; how the deterministic RT reflection resolve and probes feed in; cost vs our denoiser + FSR at the same render scale.

Design implication (applies now, cheap)

Keep ray-traced signals available as raw, un-denoised inputs with demodulated diffuse / specular and hit distance -- which #85 phase 2 needs anyway -- so plugging in a reconstruction network later is an input-routing change, not a pipeline rework.

Output

A short findings note (licence verdict per vendor, platform matrix, required buffers, rough cost) and, if viable, an implementation ticket.

Related: #85 (denoiser), #869 (ReSTIR DI), #87 (RT GI), #859 / #860 (RT reflections).

Investigate (later, not now) vendor **neural denoise + upscale** for ray-traced lighting as an optional upgrade path on supported hardware: **NVIDIA DLSS Ray Reconstruction** (DLSS 3.5+, via Streamline) and **AMD FSR "Redstone" ray regeneration**. Our own denoiser (#85) stays the baseline for every other GPU and for anyone who turns these off -- this is an add-on like the existing upscaler options, never a requirement. ## What it is One trained network replaces the hand-written per-signal denoisers **and** TAA / the upscaler. Input each frame: the *un-denoised* lit frame plus guide buffers (depth, motion vectors, normals, roughness, diffuse + specular albedo, specular hit distance / specular motion). Output: the denoised, upscaled, anti-aliased frame. Better detail retention and temporal stability than hand-tuned filters, especially for reflections and many-light ReSTIR noise (#869). ## Questions to answer - **Licensing:** can an LGPL viewer ship / load the vendor binaries (DLSS SDK + Streamline; FidelityFX / FSR ML runtime)? NRD and RTXDI were already ruled out on licence grounds -- check these separately and document the answer. - **Platform support:** Vulkan and Linux availability for each (DLSS RR on Linux/Vulkan; Redstone ray regeneration beyond DX12). Windows-only would still be worth it if licensing allows. - **Hardware:** DLSS RR = RTX (tensor cores); ray regeneration = RDNA 4. Intel: no equivalent known -- recheck. - **Integration shape:** which passes it replaces when active (RT shadow denoiser, god-ray denoiser, future radiance denoiser, TAA / FSR), and what guide buffers we must produce that we don't today. - **Interaction with our signals:** does it want the RT shadow mask raw or applied; how the deterministic RT reflection resolve and probes feed in; cost vs our denoiser + FSR at the same render scale. ## Design implication (applies now, cheap) Keep ray-traced signals available as **raw, un-denoised inputs** with demodulated diffuse / specular and hit distance -- which #85 phase 2 needs anyway -- so plugging in a reconstruction network later is an input-routing change, not a pipeline rework. ## Output A short findings note (licence verdict per vendor, platform matrix, required buffers, rough cost) and, if viable, an implementation ticket. Related: #85 (denoiser), #869 (ReSTIR DI), #87 (RT GI), #859 / #860 (RT reflections).

sky_line added the

labels

2026-10-19 14:36:54 +00:00

sky_line added this to the Slipstream project

2026-10-19 14:36:54 +00:00

sky_line referenced this issue from a commit

2026-10-23 04:58:08 +00:00