VR M2: layered stereo -- N-view rendering (multiview raster, compute over view layers, shared once-per-frame work) #1075

New issue

Open

opened 2026-10-25 20:31:36 +00:00 by sky_line · 1 comment

sky_line commented

2026-10-25 20:31:36 +00:00

Owner

Hard rule: the non-VR path must not lose quality or FPS; desktop runs the same shaders at 1 view/layer and every step is gated by a desktop frame-hash + timer A/B.

Revised 2026-10-26 (owner): no serial "render the frame twice" mode. Design directly for the end state:

render targets as N-layer image arrays (desktop N=1, VR N=2);
raster passes via multiview (gl_ViewIndex selects the eye's matrices);
compute passes (deferred lighting, RT, GI, post) dispatched over the view layer;
per-view data indexed by view (matrices, previous matrices, jitter, histories as layers); one-shot flags become per-frame events;
view-independent work once per frame (skinning, snow, light lists, TLAS/BLAS, shadows, probes, sims, exposure); one union cull;
per-view upscalers (DLSS / FSR context per eye).
This absorbs the old M4 (#1077) multiview / shared-pass work. VR preset initially: RT-capable GPUs, RT shadows on, RT reflections optional (replacing planar mirrors), GI / volumetric fog & clouds / particle fog / god rays / lens flare off until their per-view histories land.

Design: VR_LAYERED_STEREO_DESIGN.md (team research repo, in progress), built on the view-state inventory in VR_PER_VIEW_REFACTOR.md. Measured: ~7.5 ms fixed per view + 0.66 ms/MP -- sharing per-view fixed cost is the point.

Part of #1072.

**Hard rule:** the non-VR path must not lose quality or FPS; desktop runs the same shaders at 1 view/layer and every step is gated by a desktop frame-hash + timer A/B. **Revised 2026-10-26 (owner):** no serial "render the frame twice" mode. Design directly for the end state: - render targets as N-layer image arrays (desktop N=1, VR N=2); - raster passes via multiview (gl_ViewIndex selects the eye's matrices); - compute passes (deferred lighting, RT, GI, post) dispatched over the view layer; - per-view data indexed by view (matrices, previous matrices, jitter, histories as layers); one-shot flags become per-frame events; - view-independent work once per frame (skinning, snow, light lists, TLAS/BLAS, shadows, probes, sims, exposure); one union cull; - per-view upscalers (DLSS / FSR context per eye). This absorbs the old M4 (#1077) multiview / shared-pass work. VR preset initially: RT-capable GPUs, RT shadows on, RT reflections optional (replacing planar mirrors), GI / volumetric fog & clouds / particle fog / god rays / lens flare off until their per-view histories land. Design: VR_LAYERED_STEREO_DESIGN.md (team research repo, in progress), built on the view-state inventory in VR_PER_VIEW_REFACTOR.md. Measured: ~7.5 ms fixed per view + 0.66 ms/MP -- sharing per-view fixed cost is the point. Part of #1072.

sky_line added the

rendering

feature

labels

2026-10-25 20:31:36 +00:00

sky_line added this to the Slipstream project

2026-10-25 20:31:36 +00:00

sky_line added a new dependency

2026-10-25 20:31:36 +00:00

#1072 [Epic] VR support (OpenXR) -- headline feature for 1.6

sky_line changed title from ~~VR M2: true stereo rendering with per-eye state~~ to VR M2: layered stereo -- N-view rendering (multiview raster, compute over view layers, shared once-per-frame work)

2026-10-26 02:50:28 +00:00

sky_line referenced this issue

2026-10-26 02:50:28 +00:00

VR M4: performance -- multiview, shared passes, foveation #1077

sky_line referenced this issue

2026-10-26 02:50:28 +00:00

[Epic] VR support (OpenXR) -- headline feature for 1.6 #1072

sky_line self-assigned this

2026-10-26 02:58:27 +00:00

sky_line commented

2026-10-26 03:24:32 +00:00

Author

Owner

End-state design decided (design doc: research repo vr/VR_LAYERED_STEREO_DESIGN.md).

Approach: the world is recorded once per frame for both eyes. Screen targets become image arrays (one layer per eye), raster passes use Vulkan multiview, screen-space compute dispatches z = N, hit-list kernels (GI hits, #1039 queue) tag records with the view and keep one walk. Shared once per frame: union cull, probe capture, shadow maps, history validity. Per eye: only the DLSS / FSR instances.

Decisions:

Multiview feature enabled on the desktop device (one feature bit; no pipeline uses a view mask on desktop).
One SPIR-V per shader, arrayed images at 1 layer on desktop, view count as a specialization constant. Every conversion commit must keep the desktop frame hash identical and pass a GPU timer A/B; a desktop twin of a shader only on a reproducible loss.
Second-layer VRAM accepted (estimate +0.6-0.8 GB per eye pair at DLSS/FSR Quality, the VR default); VRAM ladder and governor count it.
Same TAA jitter in both eyes (comfort first).
Occlusion culling off in VR until the union-cull step, which adds the either-eye test.
Per-view arrays sized for 4 views (quad-view headsets later), ship 2.

Also: desktop window shows the left eye with the normal UI overlaid; VR snapshots copy the left eye; name tags / hover text become world-space billboards (M3); cluster mesh path falls back to its vertex tier without multiview mesh-shader support.

Migration: 17 steps, each gated on desktop frame hash + timers + sync validation. Steps 1-7 change no image types (timestamp guard, feature bit + spec plumbing, per-view data arrays), then targets convert to arrays one family per commit, then a desktop two-view debug mode (IPD 0: both layers must equal the single-view hash), then union cull, per-eye upscalers, view-tagged records, XR array swapchain.

End-state design decided (design doc: research repo `vr/VR_LAYERED_STEREO_DESIGN.md`). **Approach:** the world is recorded once per frame for both eyes. Screen targets become image arrays (one layer per eye), raster passes use Vulkan multiview, screen-space compute dispatches z = N, hit-list kernels (GI hits, #1039 queue) tag records with the view and keep one walk. Shared once per frame: union cull, probe capture, shadow maps, history validity. Per eye: only the DLSS / FSR instances. **Decisions:** 1. Multiview feature enabled on the desktop device (one feature bit; no pipeline uses a view mask on desktop). 2. One SPIR-V per shader, arrayed images at 1 layer on desktop, view count as a specialization constant. Every conversion commit must keep the desktop frame hash identical and pass a GPU timer A/B; a desktop twin of a shader only on a reproducible loss. 3. Second-layer VRAM accepted (estimate +0.6-0.8 GB per eye pair at DLSS/FSR Quality, the VR default); VRAM ladder and governor count it. 4. Same TAA jitter in both eyes (comfort first). 5. Occlusion culling off in VR until the union-cull step, which adds the either-eye test. 6. Per-view arrays sized for 4 views (quad-view headsets later), ship 2. Also: desktop window shows the left eye with the normal UI overlaid; VR snapshots copy the left eye; name tags / hover text become world-space billboards (M3); cluster mesh path falls back to its vertex tier without multiview mesh-shader support. **Migration:** 17 steps, each gated on desktop frame hash + timers + sync validation. Steps 1-7 change no image types (timestamp guard, feature bit + spec plumbing, per-view data arrays), then targets convert to arrays one family per commit, then a desktop two-view debug mode (IPD 0: both layers must equal the single-view hash), then union cull, per-eye upscalers, view-tagged records, XR array swapchain.

sky_line referenced this issue

2026-10-26 03:24:34 +00:00

Probe capture: render all 6 cube faces in one multiview pass #1101

sky_line referenced this issue from a commit

2026-10-26 20:49:28 +00:00

#1076 VR: remove the frozen-anchor diagnostic (SlipstreamVRDebugLayers 4)