Epic: frame generation -- FSR 3 frame generation first, DLSS frame generation later (deferred) #1114

New issue

Open

opened 2026-10-26 19:15:08 +00:00 by sky_line · 0 comments

sky_line commented

2026-10-26 19:15:08 +00:00

Owner

Status: deferred (owner, 2026-10-26). Order: FSR 3 frame generation first, DLSS frame generation later.

Goal: optional frame generation, with generated frames inserted between rendered ones for smoother motion on high-refresh displays. It must look clean with Second Life's heavy UI: nametags, hover text, floaters and chat must never smear.

Phase 1: groundwork both generators need

UI separation: a hudless copy of the world (after post / upscale, before nametags, hover text, HUDs, floaters and the cursor), plus the UI as its own layer or with its alpha available. This is the hard part, since today the UI is drawn straight over the world.
Inputs per frame: depth, motion vectors and camera matrices. These already exist for DLSS RR; check that they are valid with the upscaler off and at every upscaler setting.
Present and pacing: the generator owns the present of the in-between frames. It has to cooperate with our frame limiter, background/minimised throttling, swapchain recreation on resize and vsync, and the async-compute queue layout.
Rules:
- off in VR, which is inert per the VR hard rule;
- off, with a note, when the frame rate is already capped at the display rate;
- a camera cut or teleport resets history (no generated frames across a cut);
- a latency warning in the setting's tooltip.

Phase 2: FSR 3 frame generation (first)

AMD FidelityFX frame interpolation and frame-generation swapchain. Open source (MIT), Vulkan, any vendor, Linux included. We already ship FSR3 shaders (vk_fsr3_shaders).
One setting (off by default), shown in Preferences > Graphics next to the upscaler, available on every GPU that runs the Vulkan path.
Measure on the owner's 5090, the 3060 and the GTX 1080: frame pacing (not only FPS), latency, and UI artefacts while flying, orbiting and typing.

Phase 3: DLSS frame generation (later)

NVIDIA Multi Frame Generation (DLSS 4/4.5: 2x on RTX 40, up to 6x on RTX 50) comes only through the Streamline SDK, which hooks present and requires Reflex latency markers every frame.
Licensing, as DLSS RR (option B): ship no NVIDIA binaries; the user supplies the Streamline frame-generation plugin and DLLs, like RenderVKDlssShimPath. Never build or ship the NVIDIA parts in CI.
Platform: Streamline frame generation appears to be Windows-only (older Streamline notes required DX12 for frame generation). Confirm Vulkan and Linux support in NVIDIA's current docs before starting.
It reuses Phase 1's inputs and pacing; mostly a different generator behind the same setting.

Not in scope: DLSS 5's generative "neural rendering" (a different feature); evaluate separately when its SDK is public.

**Status: deferred** (owner, 2026-10-26). Order: FSR 3 frame generation first, DLSS frame generation later. **Goal:** optional frame generation, with generated frames inserted between rendered ones for smoother motion on high-refresh displays. It must look clean with Second Life's heavy UI: nametags, hover text, floaters and chat must never smear. ## Phase 1: groundwork both generators need - **UI separation:** a hudless copy of the world (after post / upscale, before nametags, hover text, HUDs, floaters and the cursor), plus the UI as its own layer or with its alpha available. This is the hard part, since today the UI is drawn straight over the world. - **Inputs per frame:** depth, motion vectors and camera matrices. These already exist for DLSS RR; check that they are valid with the upscaler off and at every upscaler setting. - **Present and pacing:** the generator owns the present of the in-between frames. It has to cooperate with our frame limiter, background/minimised throttling, swapchain recreation on resize and vsync, and the async-compute queue layout. - **Rules:** - off in VR, which is inert per the VR hard rule; - off, with a note, when the frame rate is already capped at the display rate; - a camera cut or teleport resets history (no generated frames across a cut); - a latency warning in the setting's tooltip. ## Phase 2: FSR 3 frame generation (first) - AMD FidelityFX frame interpolation and frame-generation swapchain. Open source (MIT), Vulkan, any vendor, Linux included. We already ship FSR3 shaders (`vk_fsr3_shaders`). - One setting (off by default), shown in Preferences > Graphics next to the upscaler, available on every GPU that runs the Vulkan path. - Measure on the owner's 5090, the 3060 and the GTX 1080: frame pacing (not only FPS), latency, and UI artefacts while flying, orbiting and typing. ## Phase 3: DLSS frame generation (later) - NVIDIA Multi Frame Generation (DLSS 4/4.5: 2x on RTX 40, up to 6x on RTX 50) comes only through the **Streamline** SDK, which hooks present and requires **Reflex** latency markers every frame. - **Licensing, as DLSS RR (option B):** ship no NVIDIA binaries; the user supplies the Streamline frame-generation plugin and DLLs, like `RenderVKDlssShimPath`. Never build or ship the NVIDIA parts in CI. - **Platform:** Streamline frame generation appears to be Windows-only (older Streamline notes required DX12 for frame generation). Confirm Vulkan and Linux support in NVIDIA's current docs before starting. - It reuses Phase 1's inputs and pacing; mostly a different generator behind the same setting. **Not in scope:** DLSS 5's generative "neural rendering" (a different feature); evaluate separately when its SDK is public.

sky_line added the

labels

2026-10-26 19:15:08 +00:00

sky_line self-assigned this

2026-10-26 19:15:08 +00:00

sky_line added this to the Slipstream project