Leak: materials, bindless texture slots and geometry of a sim you left are never freed #737

Closed
opened 2026-10-12 23:08:40 +00:00 by sky_line · 3 comments
Owner

Measured (owner, 2026-10-12, RTX, plenty of VRAM): teleported from an event sim to home at 23:01:49 and sampled every 15 s.

Event sim Peak Home, 3 min later
Objects 16,249 28,576 16,881 (old sim's objects gone by 23:02:53)
LL texture list 5,830 11,559 7,138 and falling (LL's 30 s flush works)
Material store 3,863 7,520 7,520 -- never shrinks
GPU texture slots (total_bindless) 4,818 9,601 9,601 -- never released
Texture VRAM 700 MB 1,243 MB 1,155 MB (barely moves)
Mesh slots 29,943 51,689 51,425 -- old sim's ~21.5k slots retained
Geometry VRAM 692 MB 823 MB 823 MB
CPU geometry mirror 472 MB 649 MB 646 MB

In a longer session that visited three sims, the material store reached 34,334 materials with ~18k objects loaded, and textures 53k.

Owner direction: free resources once they're no longer needed. Keeping a left sim's assets in VRAM for a possible return is not worth it: visiting several sims (especially during events) is the common pattern, and the stale data eats headroom and later forces bulk cleanup under pressure (the geometry GC thrash fixed earlier today was this).

Fix: release by liveness, not by memory pressure.

  • Materials: a material with no live instance referencing it is released (short grace, e.g. 30-60 s, to ride out LOD / re-ingest churn); its texture references drop.
  • Bindless texture slots / GPU images: freed when their texture / material is released (and when LL deletes the texture), so texture VRAM follows LL's list.
  • Geometry: mesh slots / LOD groups with no live instance are freed in the background (time-sliced), with the CPU mirror following; the GC budget stays as the pressure backstop, not the only path.
  • Correctness: refcounts / liveness must be exact (no freeing something a live or pending object uses; re-ingest re-acquires cleanly); budgeted per frame; stats: released per minute by kind, live counts, and a log line on teleport summarizing what was freed within a minute.
  • Verification: repeat the TP test -- after leaving a sim, materials, bindless slots, mesh slots, geometry VRAM and the CPU mirror return close to the new sim's own working set within ~1-2 minutes.
**Measured (owner, 2026-10-12, RTX, plenty of VRAM):** teleported from an event sim to home at 23:01:49 and sampled every 15 s. | | Event sim | Peak | Home, 3 min later | |---|---|---|---| | Objects | 16,249 | 28,576 | 16,881 (old sim's objects gone by 23:02:53) | | LL texture list | 5,830 | 11,559 | 7,138 and falling (LL's 30 s flush works) | | **Material store** | 3,863 | 7,520 | **7,520 -- never shrinks** | | **GPU texture slots (total_bindless)** | 4,818 | 9,601 | **9,601 -- never released** | | Texture VRAM | 700 MB | 1,243 MB | 1,155 MB (barely moves) | | **Mesh slots** | 29,943 | 51,689 | **51,425 -- old sim's ~21.5k slots retained** | | **Geometry VRAM** | 692 MB | 823 MB | **823 MB** | | **CPU geometry mirror** | 472 MB | 649 MB | **646 MB** | In a longer session that visited three sims, the material store reached 34,334 materials with ~18k objects loaded, and textures 53k. **Owner direction:** free resources once they're no longer needed. Keeping a left sim's assets in VRAM for a possible return is not worth it: visiting several sims (especially during events) is the common pattern, and the stale data eats headroom and later forces bulk cleanup under pressure (the geometry GC thrash fixed earlier today was this). **Fix:** release by liveness, not by memory pressure. - **Materials:** a material with no live instance referencing it is released (short grace, e.g. 30-60 s, to ride out LOD / re-ingest churn); its texture references drop. - **Bindless texture slots / GPU images:** freed when their texture / material is released (and when LL deletes the texture), so texture VRAM follows LL's list. - **Geometry:** mesh slots / LOD groups with no live instance are freed in the background (time-sliced), with the CPU mirror following; the GC budget stays as the pressure backstop, not the only path. - **Correctness:** refcounts / liveness must be exact (no freeing something a live or pending object uses; re-ingest re-acquires cleanly); budgeted per frame; stats: released per minute by kind, live counts, and a log line on teleport summarizing what was freed within a minute. - **Verification:** repeat the TP test -- after leaving a sim, materials, bindless slots, mesh slots, geometry VRAM and the CPU mirror return close to the new sim's own working set within ~1-2 minutes.
sky_line added this to the Slipstream project 2026-10-12 23:08:40 +00:00
Author
Owner

20 minutes after the teleport (23:21), nothing further was released: materials 7,520, bindless slots 9,601, mesh slots 51,425, geometry VRAM 823 MB, CPU mirror 646 MB, texture VRAM 1,156 MB. LL texture list levelled off at 7,135 (from a peak of 11,559). So the retention is permanent, not slow.

20 minutes after the teleport (23:21), nothing further was released: materials 7,520, bindless slots 9,601, mesh slots 51,425, geometry VRAM 823 MB, CPU mirror 646 MB, texture VRAM 1,156 MB. LL texture list levelled off at 7,135 (from a peak of 11,559). So the retention is permanent, not slow.
Author
Owner

Fix verified (build 70d807f1ea, pushed to main): home -> another sim -> home round trip.

home, fresh login 24 s after return 3 min after return before fix
mesh slots 23,369 45,239 24,215 51,425
geometry buffers 347 MB ~908 MB 331 MB 823 MB
CPU mirror 207 MB 649 MB 225 MB 646 MB
materials (live) 5,600 8,753 5,288 5,380
texture slots 5,904 10,313 5,863 ~5,970

released since TP: geometry 1,232 MB, mirror 461 MB, 24.6k mesh slots, 5,475 materials, 4,825 texture images. Two liveness compactions shrank the buffers 594 -> 331 MB; no hitches logged after the return. Orphan rows 0 throughout.

Correction to my earlier comment: the materials= / total_bindless= figures were high-water marks (ids are reused); the live counts were already falling. Geometry was the real leak (GC only ran above a ~2.5 GB budget). Those log fields now print live (hw).

Tradeoff: after more than 30 s away (RenderVKAssetReleaseGraceSec), a sim's assets are released and reload on return.

Fix verified (build 70d807f1ea, pushed to main): home -> another sim -> home round trip. | | home, fresh login | 24 s after return | 3 min after return | before fix | |---|---|---|---|---| | mesh slots | 23,369 | 45,239 | **24,215** | 51,425 | | geometry buffers | 347 MB | ~908 MB | **331 MB** | 823 MB | | CPU mirror | 207 MB | 649 MB | **225 MB** | 646 MB | | materials (live) | 5,600 | 8,753 | **5,288** | 5,380 | | texture slots | 5,904 | 10,313 | **5,863** | ~5,970 | `released since TP`: geometry 1,232 MB, mirror 461 MB, 24.6k mesh slots, 5,475 materials, 4,825 texture images. Two liveness compactions shrank the buffers 594 -> 331 MB; no hitches logged after the return. Orphan rows 0 throughout. Correction to my earlier comment: the `materials=` / `total_bindless=` figures were high-water marks (ids are reused); the live counts were already falling. Geometry was the real leak (GC only ran above a ~2.5 GB budget). Those log fields now print live (hw). Tradeoff: after more than 30 s away (`RenderVKAssetReleaseGraceSec`), a sim's assets are released and reload on return.
Author
Owner

Shipped in 1.3.0 Beta 8 (70d807f1ea asset lifetime by liveness; fix verified by round-trip test in the comments). Closing -- reopen or comment if you still see this.

Shipped in 1.3.0 Beta 8 (70d807f1ea asset lifetime by liveness; fix verified by round-trip test in the comments). Closing -- reopen or comment if you still see this.
sky_line 2026-10-14 06:33:45 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
sky_line/Slipstream#737
No description provided.