Snapshots on the main render path: grab the live frame (downscale) or jittered accumulation (upscale) instead of an offscreen re-render #873

New issue

Open

opened 2026-10-19 16:26:21 +00:00 by sky_line · 0 comments

sky_line commented

2026-10-19 16:26:21 +00:00

Owner

Rewritten 2026-10-19 (owner direction): instead of teaching the offscreen snapshot renderer each feature (RT reflections were the trigger: rtReflWanted() refuses mOffscreen, and split lighting is on-screen only), take snapshots from the main render path: grab the finished live frame and resample it to the requested size. Deferred for now.

Why

Exact parity for free: RT reflections, weather, volumetrics, TAA's converged image, every current and future effect -- the snapshot is what the user sees. No per-feature snapshot work ever again.
No hitch: today's offscreen shot re-creates the G-buffer (and would need the split intermediates) at the shot size behind vkDeviceWaitIdle, then again for the next live frame.
VRAM: giant snapshots currently allocate full render targets at the shot size (an 8K G-buffer + intermediates is hundreds of MB, risky on 8 GB cards, #164 / #282). Main-path capture only needs the output image(s) at the shot size (plus an accumulation buffer for upscaled shots), never a second full set of render targets.

Design

At or below window size: grab + downscale. Copy the scene image (post-tonemap, before UI composite so "without interface" works; HUD composited separately when requested) and resample with a high-quality filter (Lanczos / similar, sRGB-correct). Downscaling from the window is real supersampling: better than rendering small.
Above window size: jittered temporal accumulation (photo-mode style). Hold the camera, render N live frames (16-64) with distinct sub-pixel jitter offsets, and reconstruct at 2x / 4x into a shot-size accumulation buffer (same principle as TAA / FSR, much higher sample count). Real detail, not hallucinated; capture takes ~0.5-1 s while the live view keeps running.
- Motion: moving avatars, particles, rain, foliage, flexi ghost across samples. Offer "freeze scene during capture" (reuse the existing freeze, Ctrl+Shift+F), default on for 2x and above; otherwise reject moving pixels by motion vectors (fewer samples there).
- Optional, clearly labelled: an ML single-frame upscaler for even larger output (invents detail; not the default).
Aspect ratio: shots with a different aspect from the window are cropped (show the crop frame in the snapshot preview) instead of re-rendering with another FOV.
Depth snapshot: read the live depth buffer (linearised) and resample.
Offscreen renderer stays only for 360 snapshots (six directions can't come from one frame) and as a fallback; everything else drops it.

Acceptance

Snapshots match the live view exactly (RT reflections, weather, volumetrics, post), with and without UI / HUD.
No visible hitch or history reset in the live view when taking any snapshot.
A 2x / 4x snapshot of a still scene (frozen) shows genuinely finer detail than the window (compare against a true high-res render), no ghosting with freeze on.
Peak VRAM during a 4x snapshot is far below today's offscreen path (measure both).
360 snapshots unchanged.

**Rewritten 2026-10-19 (owner direction):** instead of teaching the offscreen snapshot renderer each feature (RT reflections were the trigger: `rtReflWanted()` refuses `mOffscreen`, and split lighting is on-screen only), **take snapshots from the main render path**: grab the finished live frame and resample it to the requested size. Deferred for now. ## Why - **Exact parity for free:** RT reflections, weather, volumetrics, TAA's converged image, every current and future effect -- the snapshot is what the user sees. No per-feature snapshot work ever again. - **No hitch:** today's offscreen shot re-creates the G-buffer (and would need the split intermediates) at the shot size behind `vkDeviceWaitIdle`, then again for the next live frame. - **VRAM:** giant snapshots currently allocate full render targets at the shot size (an 8K G-buffer + intermediates is hundreds of MB, risky on 8 GB cards, #164 / #282). Main-path capture only needs the output image(s) at the shot size (plus an accumulation buffer for upscaled shots), never a second full set of render targets. ## Design 1. **At or below window size: grab + downscale.** Copy the scene image (post-tonemap, **before** UI composite so "without interface" works; HUD composited separately when requested) and resample with a high-quality filter (Lanczos / similar, sRGB-correct). Downscaling from the window is real supersampling: better than rendering small. 2. **Above window size: jittered temporal accumulation (photo-mode style).** Hold the camera, render N live frames (16-64) with distinct sub-pixel jitter offsets, and reconstruct at 2x / 4x into a shot-size accumulation buffer (same principle as TAA / FSR, much higher sample count). Real detail, not hallucinated; capture takes ~0.5-1 s while the live view keeps running. - **Motion:** moving avatars, particles, rain, foliage, flexi ghost across samples. Offer **"freeze scene during capture"** (reuse the existing freeze, Ctrl+Shift+F), default on for 2x and above; otherwise reject moving pixels by motion vectors (fewer samples there). - Optional, clearly labelled: an ML single-frame upscaler for even larger output (invents detail; not the default). 3. **Aspect ratio:** shots with a different aspect from the window are **cropped** (show the crop frame in the snapshot preview) instead of re-rendering with another FOV. 4. **Depth snapshot:** read the live depth buffer (linearised) and resample. 5. **Offscreen renderer stays only for 360 snapshots** (six directions can't come from one frame) and as a fallback; everything else drops it. ## Acceptance - Snapshots match the live view exactly (RT reflections, weather, volumetrics, post), with and without UI / HUD. - No visible hitch or history reset in the live view when taking any snapshot. - A 2x / 4x snapshot of a still scene (frozen) shows genuinely finer detail than the window (compare against a true high-res render), no ghosting with freeze on. - Peak VRAM during a 4x snapshot is far below today's offscreen path (measure both). - 360 snapshots unchanged.

sky_line added the

rendering

labels

2026-10-19 16:26:21 +00:00

sky_line added this to the Slipstream project

2026-10-19 16:26:21 +00:00

sky_line changed title from ~~Snapshots: render RT reflections (currently skipped on offscreen renders)~~ to Snapshots on the main render path: grab the live frame (downscale) or jittered accumulation (upscale) instead of an offscreen re-render

2026-10-19 16:32:07 +00:00

sky_line added the

status

in-progress

label

2026-10-20 23:06:21 +00:00

sky_line referenced this issue from a commit

2026-10-21 02:45:33 +00:00

#873: clean_snap_live.comp -- live-frame snapshot resample / depth / jittered accumulation

sky_line referenced this issue from a commit

2026-10-21 02:45:33 +00:00

#873: LLCleanSnapshotVK -- snapshots from the main render path (renderer + context)