VRAM: reflection probe resolution can OOM-crash -- pre-check setting-driven allocations, revert + notify on failure

sky_line commented

2026-10-01 19:01:46 +00:00

Owner

Problem

Raising RenderReflectionProbeResolution (and/or RenderReflectionProbeCount) can crash the viewer with a VRAM out-of-memory error on cards with less memory. Reproduced on an RTX 3060 (~12 GB): changing the probe texture size setting crashed the viewer.

Reflection probes are already the largest single VRAM consumer at max settings. Measured 2026-10-01 on a 5090 in a dense sim: probes 4,133 MB, against textures at 3,640 MB, out of 11.3 GB tracked by the renderer. Allocation happens in LLCleanRendererVK probe setup (vkgfx/llcleanrenderervk.cpp, around mProbeRes / mProbeSlotsN). The cost grows with res² × count × mip chain (radiance + capture arrays), so one step of resolution is ~4×.

Wanted behaviour

Allocations driven by user settings must never crash on failure:

Pre-check before (re)allocating: estimate the bytes the new probe arrays need. Compare against the device heap budget (VK_EXT_memory_budget: budget − current usage, plus whatever the old arrays free).
Fail gracefully if it won't fit, or if vkAllocateMemory / image creation still returns VK_ERROR_OUT_OF_DEVICE_MEMORY:
- keep (or restore) the previous allocation;
- set the setting back to its previous value;
- post a system message to the user, e.g. "Your graphics card doesn't have enough video memory for reflection probe resolution N (needs ~X MB, Y MB available). The setting was reverted."
Apply the same pattern to other large setting-driven allocations: shadow map / projector atlas sizes, RT buffers, render-target scale / resolution. A reusable "try-allocate-or-revert" helper is preferable to per-setting special cases.
Optionally clamp what the preferences UI offers, based on the GPU tier / VRAM.

Notes

Related: the GPU tier classification in llappviewer.cpp (VRAM-based presets), the RAM/VRAM tickets #159–#164, and the RT VRAM-pressure ticket filed alongside this one.
Test on an 8 GB and a 12 GB card. Step the probe resolution up to the max and confirm the revert and the message appear instead of a crash.

## Problem Raising **RenderReflectionProbeResolution** (and/or **RenderReflectionProbeCount**) can crash the viewer with a VRAM out-of-memory error on cards with less memory. Reproduced on an RTX 3060 (~12 GB): changing the probe texture size setting crashed the viewer. Reflection probes are already the largest single VRAM consumer at max settings. Measured 2026-10-01 on a 5090 in a dense sim: **probes 4,133 MB**, against textures at 3,640 MB, out of 11.3 GB tracked by the renderer. Allocation happens in `LLCleanRendererVK` probe setup (`vkgfx/llcleanrenderervk.cpp`, around `mProbeRes` / `mProbeSlotsN`). The cost grows with res² × count × mip chain (radiance + capture arrays), so one step of resolution is ~4×. ## Wanted behaviour Allocations driven by user settings must **never crash** on failure: 1. **Pre-check** before (re)allocating: estimate the bytes the new probe arrays need. Compare against the device heap budget (VK_EXT_memory_budget: budget − current usage, plus whatever the old arrays free). 2. **Fail gracefully** if it won't fit, or if `vkAllocateMemory` / image creation still returns `VK_ERROR_OUT_OF_DEVICE_MEMORY`: - keep (or restore) the previous allocation; - **set the setting back** to its previous value; - post a **system message** to the user, e.g. "Your graphics card doesn't have enough video memory for reflection probe resolution N (needs ~X MB, Y MB available). The setting was reverted." 3. Apply the same pattern to other large setting-driven allocations: shadow map / projector atlas sizes, RT buffers, render-target scale / resolution. A reusable "try-allocate-or-revert" helper is preferable to per-setting special cases. 4. Optionally clamp what the preferences UI offers, based on the GPU tier / VRAM. ## Notes - Related: the GPU tier classification in `llappviewer.cpp` (VRAM-based presets), the RAM/VRAM tickets #159–#164, and the RT VRAM-pressure ticket filed alongside this one. - Test on an 8 GB and a 12 GB card. Step the probe resolution up to the max and confirm the revert and the message appear instead of a crash.

~~sky_line referenced this issue 2026-10-04 05:59:13 +00:00~~

Epic: VRAM pressure redesign -- texture pool absorbs pressure first, RT fits 8 GB cards, no alarming notices #282

sky_line referenced this issue from a commit

2026-10-04 06:45:16 +00:00

VRAM (#226): pre-check setting-driven allocations, step down + notify instead of OOM-abort

sky_line referenced this issue

2026-10-04 12:58:51 +00:00

VRAM pressure: reflection probes as a ladder step (resolution, then count), probe budget, no hard OOM aborts (#294) #295

sky_line referenced this issue

2026-10-05 03:08:10 +00:00

Epic: RAM/VRAM management -- one budget, tiered texture storage, run on a stated minimum #405

sky_line commented

2026-10-05 03:08:10 +00:00

Author

Owner

Part of #405 (RAM/VRAM management epic).

sky_line added a new dependency

2026-10-05 03:08:10 +00:00

#405 Epic: RAM/VRAM management -- one budget, tiered texture storage, run on a stated minimum

sky_line referenced this issue from a commit

2026-10-05 08:37:46 +00:00

VRAM ladder wired: effective caps, no setting writes under ordinary pressure; hard step last (#405, #227, #226)

sky_line referenced this issue from a commit

2026-10-05 08:37:46 +00:00

Advanced Graphics: "Maximum video memory" + a live VRAM budget readout (#405)

sky_line referenced this issue from a commit

2026-10-05 08:37:46 +00:00

VRAM budget: feed the measured fixed costs in -- texture budget capped at budget - fixed (#407)

sky_line added this to the Slipstream project

2026-10-06 15:04:15 +00:00

sky_line added the

status

testing

label

2026-10-09 04:40:37 +00:00

sky_line commented

2026-10-10 04:53:18 +00:00

Author

Owner

Verified 2026-10-10 (1.2.0 Beta 8 line): under a 2000 MB fake budget the setting-driven allocations were pre-checked (vramPick) and reduced instead of failing -- reflection probes 'requested 64^2 x 256 ... does not fit; reducing', sun cascades likewise -- no crash. Closing.

sky_line

2026-10-10 04:53:18 +00:00