Track RAM + VRAM per release (baseline 2026-09-29) so regressions are caught #163

New issue

Open

opened 2026-09-29 05:49:03 +00:00 by sky_line · 1 comment

sky_line commented

2026-09-29 05:49:03 +00:00

Owner

We had VRAM strongly optimized before, and users are now complaining about RAM. Keep an eye on both with a fixed measurement.

Baseline (vulkan-port-tb @ `b346870984`, busy two-sim view, ~45 avatars, RT projector shadows on)

VRAM (process 9,175 MiB per nvidia-smi; viewer tracked 8,997 MB = 7,915 device + 1,790 host-visible):

category	MB
textures	3,381 (20,041 images; all-full would be ~95 GB)
ray tracing	1,990 (AS 944 / aux 643 / rt 403)
geometry	1,569 (vbo 709, ibo 291, skin out 317, wgt 89, tables 163)
upload staging	1,476 (regression, separate ticket)
probes / shadow / cull / particles	281 / 139 / 101 / 60
untracked (swapchain, UI, driver)	~700

RAM: RSS 28.3 GB (anon 26.2, file 2.1, swap 0):

CPU texture copies 16.7 GB (separate ticket)
mesh store 0.85 GB (33,688 assets)
geometry CPU mirrors ~0.93 GB
~7.7 GB not broken down

Do

Before each release, capture the same numbers in a standard busy spot. The VKMem line plus /proc/<pid>/smaps_rollup plus nvidia-smi --query-compute-apps is enough; a make memcheck-style script would help. Put them in the release notes / this ticket.
Add a CPU-side breakdown to the VKMem line for the "not broken down" 7.7 GB: object / avatar data, LL decode caches, mesh repo, CEF host.
Flag any category growing > 10% between releases.

Refs #157.

We had VRAM strongly optimized before, and users are now complaining about RAM. Keep an eye on both with a fixed measurement. ## Baseline (vulkan-port-tb @ b346870984, busy two-sim view, ~45 avatars, RT projector shadows on) **VRAM** (process 9,175 MiB per nvidia-smi; viewer tracked 8,997 MB = 7,915 device + 1,790 host-visible): | category | MB | |---|---| | textures | 3,381 (20,041 images; all-full would be ~95 GB) | | ray tracing | 1,990 (AS 944 / aux 643 / rt 403) | | geometry | 1,569 (vbo 709, ibo 291, skin out 317, wgt 89, tables 163) | | upload staging | 1,476 (regression, separate ticket) | | probes / shadow / cull / particles | 281 / 139 / 101 / 60 | | untracked (swapchain, UI, driver) | ~700 | **RAM**: RSS 28.3 GB (anon 26.2, file 2.1, swap 0): - CPU texture copies 16.7 GB (separate ticket) - mesh store 0.85 GB (33,688 assets) - geometry CPU mirrors ~0.93 GB - ~7.7 GB not broken down ## Do - Before each release, capture the same numbers in a standard busy spot. The VKMem line plus `/proc/<pid>/smaps_rollup` plus `nvidia-smi --query-compute-apps` is enough; a `make memcheck`-style script would help. Put them in the release notes / this ticket. - Add a CPU-side breakdown to the VKMem line for the "not broken down" 7.7 GB: object / avatar data, LL decode caches, mesh repo, CEF host. - Flag any category growing > 10% between releases. Refs #157.

sky_line referenced this issue

2026-09-29 15:55:11 +00:00

Perf: frame rate degrades after moving between sims (leaks / leftover per-frame work) #169

sky_line referenced this issue

2026-10-02 10:58:14 +00:00

VRAM footprint: 7.5 GB on a 12 GB card in a moderate sim — investigate leak / over-allocation #244

sky_line referenced this issue

2026-10-05 03:08:10 +00:00

Epic: RAM/VRAM management -- one budget, tiered texture storage, run on a stated minimum #405

sky_line commented

2026-10-05 03:08:10 +00:00

Author