VRAM pressure: reflection probes as a ladder step (resolution, then count), probe budget, no hard OOM aborts (#294)

sky_line commented

2026-10-04 12:58:51 +00:00

Owner

From the crash in #294 (RTX 4070 12 GB, 3440x1348, probes Raytraced / Full scene / 512 px x 256).

What happened:

The #226 startup check allowed the 4.1 GB probe array: "requested 512^2 x 256 needs 4149 MB; free 9413 MB -> fits". It's empty-card free memory at login.
Once the scene loaded, VRAM hit 90%. The #282 pressure ladder walked every texture step down to mip bias 3: resident textures 90 MB, heap still 89%, because textures weren't the consumer.
Probes are not a ladder step and RT was never reached.
Ten minutes later a hard createBuffer allocation failed (VK_ERROR_OUT_OF_DEVICE_MEMORY, vk_device.cpp:1842) and the viewer aborted.

Do

Probes as a pressure step. Step down RenderReflectionProbeResolution first (512 -> 256 -> 128), then RenderReflectionProbeCount (256 -> 128 -> 64).
- Reuse the #226 live reallocation path (it already frees and reallocates the probe arrays and handles OOM).
- The setting is written, latched and notified ("To fit your graphics card's memory: Reflection probe resolution 512 -> 256").
- Recovery restores it only if the user's original value fits again with margin (or never auto-raises, like the other latched settings: decide).
Ladder ordering by what can actually free memory. When the excess is much larger than what the remaining texture steps could free (the ladder already has per-step byte sums), skip to the large fixed consumers instead of grinding textures to nothing. Ordering after the cheap invisible texture steps: probes (big and least visible) -> RT steps -> deeper texture bias.
A probe budget, not just a fit check. At startup and on change, cap the probe arrays at a share of the VRAM budget (e.g. 15%; about 1.5 GB on 12 GB), stepping resolution then count down, with the same notice. Users can still pick 512 on big cards; the label now shows the cost.
No hard OOM aborts for growable allocations. The createBuffer call at vk_device.cpp:1842 was a hard VK_CHECK. Find growth/staging/store allocations that can run mid-session and make them soft: fail → enter pressure → retry next frame / keep the old size. Log the caller.

Done when

The #294 settings on a 12 GB card (RenderVKDebugFakeVramBudgetMB can simulate it) load a busy sim without crashing. Probes drop with a notice, textures stay sharp, and no hard OOM abort is possible from growable buffers.

From the crash in #294 (RTX 4070 12 GB, 3440x1348, probes Raytraced / Full scene / **512 px x 256**). **What happened:** - The #226 startup check allowed the 4.1 GB probe array: "requested 512^2 x 256 needs 4149 MB; free 9413 MB -> fits". It's empty-card free memory at login. - Once the scene loaded, VRAM hit 90%. The #282 pressure ladder walked every texture step down to mip bias 3: resident textures **90 MB**, heap still 89%, because textures weren't the consumer. - Probes are not a ladder step and RT was never reached. - Ten minutes later a hard `createBuffer` allocation failed (`VK_ERROR_OUT_OF_DEVICE_MEMORY`, `vk_device.cpp:1842`) and the viewer aborted. ### Do 1. **Probes as a pressure step.** Step down `RenderReflectionProbeResolution` first (512 -> 256 -> 128), then `RenderReflectionProbeCount` (256 -> 128 -> 64). - Reuse the #226 live reallocation path (it already frees and reallocates the probe arrays and handles OOM). - The setting is written, latched and notified ("To fit your graphics card's memory: Reflection probe resolution 512 -> 256"). - Recovery restores it only if the user's original value fits again with margin (or never auto-raises, like the other latched settings: decide). 2. **Ladder ordering by what can actually free memory.** When the excess is much larger than what the remaining texture steps could free (the ladder already has per-step byte sums), skip to the large fixed consumers instead of grinding textures to nothing. Ordering after the cheap invisible texture steps: probes (big and least visible) -> RT steps -> deeper texture bias. 3. **A probe budget, not just a fit check.** At startup and on change, cap the probe arrays at a share of the VRAM budget (e.g. 15%; about 1.5 GB on 12 GB), stepping resolution then count down, with the same notice. Users can still pick 512 on big cards; the label now shows the cost. 4. **No hard OOM aborts for growable allocations.** The `createBuffer` call at `vk_device.cpp:1842` was a hard VK_CHECK. Find growth/staging/store allocations that can run mid-session and make them soft: fail → enter pressure → retry next frame / keep the old size. Log the caller. ### Done when The #294 settings on a 12 GB card (`RenderVKDebugFakeVramBudgetMB` can simulate it) load a busy sim without crashing. Probes drop with a notice, textures stay sharp, and no hard OOM abort is possible from growable buffers.

sky_line added the

rendering

perf

bug

labels

2026-10-04 12:58:51 +00:00

sky_line referenced this issue

2026-10-05 03:08:10 +00:00

Epic: RAM/VRAM management -- one budget, tiered texture storage, run on a stated minimum #405

sky_line commented

2026-10-05 03:08:10 +00:00

Author

Owner

Part of #405 (RAM/VRAM management epic).

sky_line added a new dependency

2026-10-05 03:08:10 +00:00

#405 Epic: RAM/VRAM management -- one budget, tiered texture storage, run on a stated minimum

sky_line referenced this issue from a commit

2026-10-05 08:37:46 +00:00

VRAM ladder wired: effective caps, no setting writes under ordinary pressure; hard step last (#405, #227, #226)

sky_line added this to the Slipstream project

2026-10-06 15:04:14 +00:00

sky_line added the

status

testing

label

2026-10-09 04:40:38 +00:00

sky_line commented

2026-10-10 04:53:19 +00:00

Author

Owner

Verified 2026-10-10: the VRAM ladder engaged 'reflection probe resolution 1/2' then 'probe count 1/4' under a 2000 MB fake budget and restored both on recovery. Closing.

sky_line

2026-10-10 04:53:19 +00:00