#1544: FSR 3.1 frame generation, a generator choice, one graphics queue and AMD Anti-Lag #1555

Closed
TheSlowly wants to merge 0 commits from TheSlowly/Slipstream:fsr-framegen into main
Collaborator

Closes #1544 (part of #1114, follow-up of #1517 and #1536): frame generation for GPUs that cannot run DLSS frame generation, through AMD's FSR 3.1 frame interpolation and optical flow. AMD's GLSL is vendored next to the FSR upscaler's; the host code is ours, so there is no AMD runtime or DLL and it runs on Windows and Linux.

  • One generator interface: DLSS frame generation behind it unchanged, FSR beside it. The present thread, pacer, interface layer (HDR10 and SDR), composites and moving name tags are shared.
  • The viewer picks DLSS where it runs, else FSR. Preferences gets a Generator list (Automatic, plus DLSS / FSR only where they can run); switching is live. FSR is 2x only, and the multiplier list shows only what the generator in use can run. The status line and log name the generator ("FSR 2x").
  • Either upscaler (DLSS or FSR) feeds either generator. Inputs are stated once in the renderer's units; each generator derives its own conventions (tested host maths for FSR).
  • GPUs with one graphics queue (AMD, Intel): the present thread shares the renderer's queue under one lock, IMMEDIATE present mode only, Reflex off there. RenderVKFrameGenSharedQueue forces it for testing.
  • AMD Anti-Lag (VK_AMD_anti_lag) where the driver offers it, with or without frame generation; RenderVKAntiLag 0 turns it off. Nothing changes on other GPUs.
  • Debug: RenderVKFrameGenFsrDebug (AMD's generated-frame and reset markers).
  • New checks framegen-generator, framegen-fsr, framegen-queue; docs updated, test plan doc/testplans/1544-frame-generation-fsr.md.

Tested in world on Windows (RTX 5070 Ti, exclusive fullscreen): FSR 2x in HDR10 and SDR (colours right on a BGRA8 chain), generated-only view, live DLSS <-> FSR switching, quality-mode change while running, teleport, DLSS unchanged (old-conventions A/B), Preferences lists, debug markers, shared-queue mode with DLSS and FSR (same frame rate as separate queues; alt-tab, minimise, resize, fullscreen, teleport, snapshot, torn-off floater, quit).

Not tested: Linux, AMD and Intel GPUs (FSR on them, the real single-queue drivers, Anti-Lag), older NVIDIA GPUs picking FSR automatically. The last two fixes (a live switch to FSR admits its images again; a quit no longer waits on a present thread left behind holding the shared queue) pass their checks but were not run in world.

Known: on a shared queue the renderer's submits wait for the present thread's IMMEDIATE present. If a driver ever holds such a present, the viewer stalls until it returns (logged); a bounded design (the main thread presenting in shared mode) would be a follow-up.

Closes #1544 (part of #1114, follow-up of #1517 and #1536): frame generation for GPUs that cannot run DLSS frame generation, through AMD's FSR 3.1 frame interpolation and optical flow. AMD's GLSL is vendored next to the FSR upscaler's; the host code is ours, so there is no AMD runtime or DLL and it runs on Windows and Linux. - One generator interface: DLSS frame generation behind it unchanged, FSR beside it. The present thread, pacer, interface layer (HDR10 and SDR), composites and moving name tags are shared. - The viewer picks DLSS where it runs, else FSR. Preferences gets a Generator list (Automatic, plus DLSS / FSR only where they can run); switching is live. FSR is 2x only, and the multiplier list shows only what the generator in use can run. The status line and log name the generator ("FSR 2x"). - Either upscaler (DLSS or FSR) feeds either generator. Inputs are stated once in the renderer's units; each generator derives its own conventions (tested host maths for FSR). - GPUs with one graphics queue (AMD, Intel): the present thread shares the renderer's queue under one lock, IMMEDIATE present mode only, Reflex off there. `RenderVKFrameGenSharedQueue` forces it for testing. - AMD Anti-Lag (`VK_AMD_anti_lag`) where the driver offers it, with or without frame generation; `RenderVKAntiLag` 0 turns it off. Nothing changes on other GPUs. - Debug: `RenderVKFrameGenFsrDebug` (AMD's generated-frame and reset markers). - New checks `framegen-generator`, `framegen-fsr`, `framegen-queue`; docs updated, test plan `doc/testplans/1544-frame-generation-fsr.md`. Tested in world on Windows (RTX 5070 Ti, exclusive fullscreen): FSR 2x in HDR10 and SDR (colours right on a BGRA8 chain), generated-only view, live DLSS <-> FSR switching, quality-mode change while running, teleport, DLSS unchanged (old-conventions A/B), Preferences lists, debug markers, shared-queue mode with DLSS and FSR (same frame rate as separate queues; alt-tab, minimise, resize, fullscreen, teleport, snapshot, torn-off floater, quit). Not tested: Linux, AMD and Intel GPUs (FSR on them, the real single-queue drivers, Anti-Lag), older NVIDIA GPUs picking FSR automatically. The last two fixes (a live switch to FSR admits its images again; a quit no longer waits on a present thread left behind holding the shared queue) pass their checks but were not run in world. Known: on a shared queue the renderer's submits wait for the present thread's IMMEDIATE present. If a driver ever holds such a present, the viewer stalls until it returns (logged); a bounded design (the main thread presenting in shared mode) would be a follow-up.
Nothing above the generator names the DLSS add-on any more. The render
loop fills llfg::GenInputs (llvkframegengenerator.h) in the renderer's own
terms: the images, and the camera as the upscaler that ran wrote it
(projection, view and reprojection as they are, the raw jitter and the
sign the upscaler applied, the render size, what the inputs pass writes
for motion and depth, the upscaler's reset). LLVKFrameGen holds a
LLVKFrameGenGenerator; the present thread's job carries GenInputs.

LLVKFrameGenDlss (llvkframegendlss.*) is the only caller of the add-on's
frame generation entry points and owns the one conversion to its
conventions (toEval: matrices copied as they are, motion in pixels of the
render size, RenderVKFrameGenConventionDebug). No behaviour change: the
add-on gets the same values as before.

New check framegen-generator: a host test compares toEval field by field
with a frozen copy of the earlier derivation (two cameras, every
ConventionDebug value) and covers init / prepare / release / evaluate
against a fake add-on; source rules NEUTRAL, OWNER and ONE-SITE. The rules
of framegen-direct, -policy, -sdr and -stall that pinned the moved code
now read it where it lives.
Rules (llvkframegenpolicy.h, tested): llfg::chooseGenerator picks DLSS
where it runs, else FSR, unless RenderVKFrameGenGenerator (new, hidden:
0 automatic, 1 DLSS, 2 FSR) names one; when neither runs the reason shown
is DLSS's if the player can act on it, else FSR's. FSR's caps are one
generated frame per rendered frame and never driver pacing; a stored
multiplier above 2x says "2x (FSR generates one frame per rendered
frame)". The add-on's blocks are DLSS's only. The scratch image's format
per generator (FSR: RGBA8 on a BGRA8 chain), the generator's own images in
the one-set admission, the status prefix ("FSR 2x").

Availability: "needs DLSS upscaling" becomes "needs DLSS or FSR
upscaling" (Context::inputsOn, Block::NeedsUpscaler). The FSR upscaler
hands frame generation its depth and motion vectors (the same inputs pass
as DLSS's) and the main view's camera, and the frame generation chain is
built for RenderVKUpscaler 1-11. So DLSS frame generation now also runs
with FSR upscaling.

LLVKFrameGen asks both generators at startup and keeps each one's answer.
The FSR generator (llvkframegenfsr.*) is a placeholder that reports it
cannot run, so the choice is DLSS or none, as before; the setting is read
at startup.

Checks: framegen-policy (new cases, rule GENERATOR), framegen-generator
(both upscalers as camera producers, rule INPUTS), framegen-direct and
framegen-linux follow the moved code.
AMD's frame interpolation and optical flow GPU sources (FidelityFX SDK 1.1.4, MIT, the same commit
as the vendored FSR upscaler) are vendored next to the upscaler: the 11 frame interpolation and 7
optical flow Vulkan entry points and the headers they include, without the HLSL callbacks. The
shader build compiles 24 variants into the FSR shader header, fixed to frame generation's inputs
(render-size unjittered motion vectors, reversed-Z, fp32), with the output in HDR10's and SDR's
format and SPD fallbacks for devices without subgroup quad operations.

One local patch, marked and listed in the README: the output image takes an explicit storage
format, so no device needs format-less storage writes. The licence notices now cover frame
generation. A new framegen-fsr check keeps binaries and HLSL files out, the MIT notices in, and
every pass built.

Nothing uses the shaders yet; the host code that drives them comes next.
llvkframegenfsrmath.h holds, without Vulkan or viewer headers, everything
the FSR generator's host code computes: the constant buffers of AMD's frame
interpolation and optical flow shaders and what goes into them, the optical
flow image sizes, every pass's dispatch size, which ping-pong image a pass
binds, the pass order, the reset rule and the memory the images take.

The motion vector scale is derived from the motion image's own units (UV,
previous minus current), never from another generator's scale; the jitter is
the one the frame was rendered with.

The FSR upscaler takes its device-depth factors from the same function
(llfsr::deviceToViewDepth): same values, one derivation.

framegen-fsr compiles and runs scripts/check/test_framegen_fsr.cpp: layouts,
contents, sizes, dispatch counts, the SDK's A / B selection, the pass order,
the reset rule, the memory figures, and a camera pan whose vector must come
out as the right number of render pixels -- and must not when it is stored
in other units or with the other sign.
LLVKFrameGenFsr is the host side of FSR 3.1 frame generation, as
LLCleanFSR3VK is the upscaler's: its own images, constant ring, descriptor
sets and pass sequence, no FidelityFX runtime.

- init: what the device must offer, checked against the compiled modules --
  the storage and filterable formats, subgroup arithmetic in compute with a
  subgroup size of 32 or 64 (the optical flow search), compute on the present
  queue's family. Missing: "unsupported" with the missing thing named.
  Else one generated frame per rendered frame and the viewer's own pacer.
- prepare (main thread, present thread drained): pipelines the first time,
  the images at exact sizes, every descriptor set. The present slots' images
  are part of what a feature is made for (GenSize), so no set is ever written
  on the present thread.
- evaluate (present thread): one call records prepare, optical flow and
  interpolation for a rendered frame, in the tested order of
  llfsrfg::passPlan, with the tested dispatch sizes and constants, a memory
  barrier before every step. The jitter is the one rendered with; the motion
  scale comes from the motion image's own units.
- A reset clears every image and the counters: it is also what follows a
  frame whose commands never ran.
- The output variant (rgb10_a2 / rgba8) is chosen from the output image's
  format in one place; a storage image is made in the format its GLSL
  declaration names (rule STORAGE).

A generator may keep objects for the session (its pipelines): the interface
gains shutdown(), called once after release().

Until the frame loop hands over the render size, the slot images and the
RGBA8 output image (the next change), prepare refuses and FSR generation
does not start.

framegen-fsr gains the rules THREAD, ORDER, UNITS, GPU-DRIVEN, CONST, FORMAT
and STORAGE.
- The generator's depth / motion copies and its feature are made after one drain, copies first:
  the feature is given the render size and every present slot's images (fillGenSize), counted
  by imagesSerial whenever they are made again. Another quality mode makes FSR's feature again;
  DLSS ignores the render size. No feature is made before the upscaler's inputs exist.
- The scratch image is in the generator's format (llfg::scratchFormat: RGBA8 for FSR on a BGRA8
  chain); the storage check tests it, and a generated frame that could not be composed is
  blitted instead of copied when the formats differ.
- The generator's own images join the present slots' memory admission (generatorBytes).
- fillGenInputs (was fillFrameGenEval) also hands the display's peak, the time since the frame
  before and the frame's counter.
- RenderVKFrameGenGenerator is live: a change asks for a swapchain rebuild, where applyGenerator
  releases the old feature (present thread drained, device idle), adopts the new generator's
  caps and asks about driver pacing again.
- The 5 s log line starts with the generator and multiplier ("FSR 2x", "DLSS 3x").
- The Frame generation tooltip and RenderVKFrameGen's comment: DLSS frame generation needs a
  GeForce RTX 40 or 50 and the DLSS add-on, FSR frame generation runs on other GPUs at 2x; both
  need DLSS or FSR upscaling.
- The tooltip's status starts with the generator in use and its multiplier ("FSR 2x", "DLSS 3x").
- Check framegen-fsr gains rule TEXT.
RenderVKFrameGenFsrDebug (hidden, live, not saved) hands AMD's own debug flags to the inpainting
pass: +1 a green bar down the left edge of every generated frame, +2 a red bar on a frame generated
after a history reset and a blue one when the optical flow saw a scene change.
doc/frame-generation.md and doc/frame-generation-design.md lose "DLSS" from their titles and say
which generator runs where, how FSR is built (AMD's shaders vendored, the viewer's own host code),
the inputs' units, pacing and memory with FSR, the settings and what is not done yet (one graphics
queue, other GPUs). New doc/testplans/1544-frame-generation-fsr.md; the FidelityFX README notes the
unused debug view variants.
- New Generator list next to Frame generation (RenderVKFrameGenGenerator, live): Automatic, plus
  DLSS and FSR only where they can run on this GPU.
- The multiplier list holds exactly what the generator in use can run (FSR: Off and 2x) instead of
  greying 3x and 4x; a saved higher multiplier shows as the highest one offered.
- Check framegen-fsr gains rule PREFS; framegen-policy's TOP rule follows the list.
Frame generation's present thread needed a second queue of the graphics
family, and was refused ("cannot present generated frames") where the
family has only one: Mesa's RADV and ANV state exactly one in their
source, and the same is reported for current Radeon drivers on Windows.
FSR frame generation is meant for those GPUs.

Where the family has no queue to spare the present thread now shares
the renderer's queue (llfg::presentQueueFor, VkDeviceContext::
presentQueueShared):

- Every driver call on that queue goes through one lock,
  VkDeviceContext::QueueLock: the submit and queue-wait wrappers take it
  for every caller, the two vkQueuePresentKHR sites take it themselves,
  and the instant replay encoder submits through the viewer's wrapper
  (a hook) instead of its own entry point. On a device whose present
  thread has its own queue the lock takes nothing.
- The lock holds one driver call and nothing else. It raises nothing;
  a wait for it is logged after 5 s and ends without the lock only on a
  device lost meanwhile (no driver call is made then).
- A shared queue presents generated frames in IMMEDIATE mode only, so no
  present sits in the compositor with the lock. A display that offers no
  IMMEDIATE mode gets a normal chain and the reason
  Block::NeedsImmediate; the bounded slot wait stays the main thread's
  only wait for the present thread.
- No submit waits for the pacer on a shared queue (the renderer's
  submits would sit behind it): the present thread waits each gate
  itself, at most llfg::kGateWaitBoundMs, then presents.
- A device wait made while the present thread exists waits the shared
  queue too, inside the lock.
- Reflex stays off on a shared queue (it cannot be marked out of band).
- Block::NoPresentQueue is left for a device with no usable queue; a
  missing pacer clock is Block::Unsupported.

RenderVKFrameGenSharedQueue (hidden, read at startup) forces the shared
queue on a GPU that has a spare one, for testing.

New check framegen-queue (HOOK, LOCK, OWN, DEVICE, BLOCK, CHAIN, GATE,
SETTING); policy and swapchain mock tests for the queue choice, the
present mode, the gate wait and the lock scopes.
VK_AMD_anti_lag with its feature, enabled where the driver offers both: each frame marks its input
stage after the frame wait (where it may sleep, so input reaches the screen sooner) and its present
stage before the present or the hand-over to frame generation's present thread, with the same frame
index; never in VR. RenderVKAntiLag 0 turns it off. Nothing changes on other GPUs. Check
framegen-fsr gains rule ANTILAG. Not tested on AMD hardware.
How the present thread shares the renderer's queue on a GPU with one graphics queue (IMMEDIATE only,
Reflex off, the gates waited on the CPU), AMD Anti-Lag, their settings, and the test plan's steps
for both.
- Switching from DLSS to FSR at the same scratch format admitted nothing again, so FSR's own
  images were allocated outside the memory admission. The presenter now admits the whole set again
  whenever the generator's own figure changes.
- FSR's frame counter counted skipped frames too, so a skipped frame reset its history where DLSS
  keeps it. It now counts only rendered frames that generate.
On a shared queue, a present thread left behind inside a driver call may hold the queue's lock for
good; the quit's queue waits then waited for it forever. A wait for the lock now gives up (no driver
call, as on a lost device) once the thread was left behind.
Author
Collaborator

Rebased onto main b282ecfff1 (GPU-loss recovery). The conflicts were the queue / present calls #1546 wraps in VkDriverCallScope: both kept (the queue lock, then the driver-call scope). New commit a2d80e0d14: #1547 boundedIdleProof submits on the shared present queue under its lock (framegen-queue). After a recovery FSR frame generation stays off like DLSS (shutdownDirect, no initDirect), as #1550 says. Source checks pass (76).

Rebased onto main b282ecfff1 (GPU-loss recovery). The conflicts were the queue / present calls #1546 wraps in VkDriverCallScope: both kept (the queue lock, then the driver-call scope). New commit a2d80e0d14: #1547 boundedIdleProof submits on the shared present queue under its lock (framegen-queue). After a recovery FSR frame generation stays off like DLSS (shutdownDirect, no initDirect), as #1550 says. Source checks pass (76).
Author
Collaborator

Merged to main in 2834e00bf5; closing.

Merged to main in 2834e00bf5; closing.
TheSlowly closed this pull request 2026-11-18 01:32:07 +00:00

Pull request closed

Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
sky_line/Slipstream!1555
No description provided.