VRAM footprint: 7.5 GB on a 12 GB card in a moderate sim — investigate leak / over-allocation #244

Closed
opened 2026-10-02 10:58:14 +00:00 by sky_line · 4 comments
Owner

Report

A user on a 12 GB card sees Slipstream using ~7.5 GB VRAM while EVE Online (running alongside) uses ~1.7 GB. That seems high for the sim they're in. This is the same user who reported textures dropping to thumbnail size when alt-tabbing between EVE and Slipstream.

Why it matters

  • The VK texture residency sizes itself from the per-process VK_EXT_memory_budget heap budget (texture budget = 60% of it, pressure mode at >90%). Whatever non-texture memory we hold (RT, staging, G-buffer / FSR / HiZ targets, geometry) comes straight out of what textures get.
  • With another GPU app holding VRAM, an inflated footprint pushes us into pressure mode, which raises the global mip bias. This is the thumbnail bug.
    • The mitigation is merged in 76c3e69e3d: budget/pressure bias is capped at RenderVKTexPressureMaxBias (3) unless an allocation actually fails, and the bias recovers immediately on focus regain.
    • That treats the symptom, not the footprint.

Investigate

  1. Reproduce: go to the reporter's sim/spot and measure with RenderVKSectionLog on. Capture:
    • the VKMem logMemory line (textures / RT AS+aux / geometry / staging / targets);
    • nvidia-smi --query-compute-apps;
    • the texture mip bias / VRAM pressure mode lines.
      Compare against the 2026-09-29 baseline (9.2 GB on a 5090 in a busy two-sim view: textures 3.4, RT 2.0, geometry 1.6, staging 1.5).
  2. Leak check: watch VRAM over a long session, through teleports / region crossings / alt-tab cycles. Does VKMem total or driver process usage grow without bound? Does staging stay at peak after a re-stream (#160)? Are freed images/buffers actually returned (destroyImageDeferred / retire queues, the descriptor pools)?
  3. Over-allocation check: account for every render target at the user's resolution and settings, and confirm each is freed when its feature is off. Recent additions:
    • FSR 3.1 internal (~250–330 MB at 4K)
    • TAA history kept for the glow resolve under FSR (~265 MB)
    • two-phase HiZ buffers (~115 MB at the bucket cap)
    • deferred G-buffer (20 B/px)
    • RT mask ping-pong history
    • pre-skin prev-position history
    • async per-slot command buffers
      Also check bucket caps sized for worst case rather than actual counts, RT aux / BLAS budgets (#161, #164), and texture residency over-holding (bias 0 at full res for off-screen textures).
  4. Check whether a discrepancy between our tracked bytes and the driver's process usage hides untracked allocations (e.g. swapchain, pipeline caches, driver-internal).

#159, #160, #161, #163, #164

Asks for the reporter

  • GPU model and driver
  • resolution, render scale / upscale mode
  • RT on/off and graphics preset
  • Slipstream.log from a session where this happens
## Report A user on a **12 GB card** sees Slipstream using **~7.5 GB VRAM** while EVE Online (running alongside) uses ~1.7 GB. That seems high for the sim they're in. This is the same user who reported textures dropping to thumbnail size when alt-tabbing between EVE and Slipstream. ## Why it matters - The VK texture residency sizes itself from the **per-process** `VK_EXT_memory_budget` heap budget (texture budget = 60% of it, pressure mode at >90%). Whatever non-texture memory we hold (RT, staging, G-buffer / FSR / HiZ targets, geometry) comes straight out of what textures get. - With another GPU app holding VRAM, an inflated footprint pushes us into pressure mode, which raises the global mip bias. This is the thumbnail bug. - The mitigation is merged in 76c3e69e3d: budget/pressure bias is capped at `RenderVKTexPressureMaxBias` (3) unless an allocation actually fails, and the bias recovers immediately on focus regain. - That treats the symptom, not the footprint. ## Investigate 1. **Reproduce:** go to the reporter's sim/spot and measure with `RenderVKSectionLog` on. Capture: - the `VKMem` logMemory line (textures / RT AS+aux / geometry / staging / targets); - `nvidia-smi --query-compute-apps`; - the `texture mip bias` / `VRAM pressure mode` lines. Compare against the 2026-09-29 baseline (9.2 GB on a 5090 in a busy two-sim view: textures 3.4, RT 2.0, geometry 1.6, staging 1.5). 2. **Leak check:** watch VRAM over a long session, through teleports / region crossings / alt-tab cycles. Does `VKMem` total or driver process usage grow without bound? Does staging stay at peak after a re-stream (#160)? Are freed images/buffers actually returned (`destroyImageDeferred` / retire queues, the descriptor pools)? 3. **Over-allocation check:** account for every render target at the user's resolution and settings, and confirm each is freed when its feature is off. Recent additions: - FSR 3.1 internal (~250–330 MB at 4K) - TAA history kept for the glow resolve under FSR (~265 MB) - two-phase HiZ buffers (~115 MB at the bucket cap) - deferred G-buffer (20 B/px) - RT mask ping-pong history - pre-skin prev-position history - async per-slot command buffers Also check bucket caps sized for worst case rather than actual counts, RT aux / BLAS budgets (#161, #164), and texture residency over-holding (bias 0 at full res for off-screen textures). 4. Check whether a **discrepancy between our tracked bytes and the driver's process usage** hides untracked allocations (e.g. swapchain, pipeline caches, driver-internal). ## Related #159, #160, #161, #163, #164 ## Asks for the reporter - GPU model and driver - resolution, render scale / upscale mode - RT on/off and graphics preset - `Slipstream.log` from a session where this happens
Author
Owner

Reporter follow-up: Linux on the same machine

On Linux, from a fresh start, the reporter measured ~4 GB VRAM. On Windows, with session length unknown, they saw ~7.5 GB.

The difference has two possible explanations, and we need to tell them apart:

  1. Growth over time (leak, or staging/pools held at peak, #160). A fresh start can't show this. On Linux, re-measure after an equivalent session (same sim, some teleports / region crossings / alt-tabs) and compare with the fresh-start number.
  2. Windows-specific. Possible causes:
    • The Windows driver/WDDM reports or commits memory differently. Task Manager's "dedicated GPU memory" counts committed residency, not what we allocated.
    • Our allocations behave differently on Windows (memory types, heap budget, pipeline cache).
    • To check, compare our own VKMem logMemory totals (what we allocated) in both OSes' Slipstream.log against the OS tool's number. If VKMem agrees across OSes but the tool doesn't, it's driver accounting. If VKMem itself differs, it's our allocation path.

Asks

  • Slipstream.log from the Windows session, plus a matching Linux session of similar length.
  • How long the Windows session ran, and where.
  • On Windows, nvidia-smi or Task Manager readings at start and after ~30 min.
## Reporter follow-up: Linux on the same machine On Linux, from a **fresh start**, the reporter measured **~4 GB VRAM**. On Windows, with session length unknown, they saw ~7.5 GB. The difference has two possible explanations, and we need to tell them apart: 1. **Growth over time (leak, or staging/pools held at peak, #160).** A fresh start can't show this. On Linux, re-measure after an equivalent session (same sim, some teleports / region crossings / alt-tabs) and compare with the fresh-start number. 2. **Windows-specific.** Possible causes: - The Windows driver/WDDM reports or commits memory differently. Task Manager's "dedicated GPU memory" counts committed residency, not what we allocated. - Our allocations behave differently on Windows (memory types, heap budget, pipeline cache). - To check, compare our own `VKMem` logMemory totals (what we allocated) in both OSes' `Slipstream.log` against the OS tool's number. If `VKMem` agrees across OSes but the tool doesn't, it's driver accounting. If `VKMem` itself differs, it's our allocation path. ## Asks - `Slipstream.log` from the Windows session, plus a matching Linux session of similar length. - How long the Windows session ran, and where. - On Windows, `nvidia-smi` or Task Manager readings at start and after ~30 min.
Author
Owner

Update: the texture bug looks Windows-only

The reporter (RTX 4070, 12 GB) loaded EVE alongside Slipstream on Linux and could not reproduce the thumbnail-texture bug.

Leading explanation: WDDM behaviour for background processes.

  • On Windows, the OS lowers a background process's VK_EXT_memory_budget heap budget and demotes its residency.
  • Our residency treats the shrunken budget as pressure and raises the mip bias.
  • The Linux NVIDIA driver reports a budget that tracks physical heap headroom, so the same scene stays under 90%.

What to check

  • The mitigation (76c3e69e3d): cap budget-driven bias at RenderVKTexPressureMaxBias 3 unless an allocation actually fails, and recover immediately on focus. Have the reporter test it on Windows.
  • Whether budget pressure should apply while unfocused. Consider ignoring budget shrinkage while we're unfocused unless allocations fail: on WDDM, demoted memory is paged rather than lost.
  • Windows Slipstream.log around an alt-tab: look for VRAM pressure mode ENTER and texture mip bias lines, and the heapBudget in the VKMem line, which needs SlipstreamVerboseLogging.
## Update: the texture bug looks Windows-only The reporter (RTX 4070, 12 GB) loaded EVE alongside Slipstream **on Linux** and **could not reproduce** the thumbnail-texture bug. **Leading explanation: WDDM behaviour for background processes.** - On Windows, the OS lowers a background process's `VK_EXT_memory_budget` heap budget and demotes its residency. - Our residency treats the shrunken budget as pressure and raises the mip bias. - The Linux NVIDIA driver reports a budget that tracks physical heap headroom, so the same scene stays under 90%. **What to check** - **The mitigation (76c3e69e3d):** cap budget-driven bias at `RenderVKTexPressureMaxBias` 3 unless an allocation actually fails, and recover immediately on focus. Have the reporter test it on Windows. - **Whether budget pressure should apply while unfocused.** Consider ignoring budget shrinkage while we're unfocused unless allocations fail: on WDDM, demoted memory is paged rather than lost. - **Windows `Slipstream.log` around an alt-tab:** look for `VRAM pressure mode ENTER` and `texture mip bias` lines, and the `heapBudget` in the `VKMem` line, which needs `SlipstreamVerboseLogging`.
Author
Owner

Part of #405 (RAM/VRAM management epic).

Part of #405 (RAM/VRAM management epic).
sky_line added this to the Slipstream project 2026-10-06 15:04:15 +00:00
Author
Owner

Investigation complete: every VRAM category is tracked and reported (#407), pools and lazy allocations (#411, #600, #614-#616) removed the over-allocation, and like-for-like totals are down. Remaining reductions are tracked in #614-#617 and #625. Closing.

Investigation complete: every VRAM category is tracked and reported (#407), pools and lazy allocations (#411, #600, #614-#616) removed the over-allocation, and like-for-like totals are down. Remaining reductions are tracked in #614-#617 and #625. Closing.
sky_line 2026-10-10 04:53:21 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
sky_line/Slipstream#244
No description provided.