#157 - Perf: busy sim is CPU-bound at ~31 fps — RT build is 13.7 ms/frame on the main thread - sky_line/Slipstream

sky_line commented

2026-09-29 03:32:46 +00:00

Owner

30 s capture (2026-09-28 23:31, build bab6bb454e + clouds WIP) in a busy sim that the user reports as hitchy. RTX-class GPU, timers and section logs on, RT projector shadows on (56 projectors).

Summary

CPU-bound, not GPU-bound. Frames average 30.6–37.5 ms (27–33 fps). The GPU needs only ~13.2 ms/frame; GPU utilization is 45–70 %, with dips to 4–5 %. The main thread (App) sits at 99 % CPU, and the worker pool is at 20–45 %.

CPU sections (avg ms/frame, last window of 169 frames)

section	ms
frame	30.6
display_total	24.8
renderFrame	20.3
vk_recordWorld	14.5
↳ vk_rtBuild	13.7
LL_idle_total	5.7
LL_objectListUpdate	3.0
ui_draw	2.9
avatarCharacter	1.8 (47 avatars, 33 visible)
vk_ingestTick	1.8 (up to 3.5)
vk_sync_total	1.7
buildWorldFrame	1.6 (bwf_lights 1.1)
vk_texUpload	1.5

RT counters: rt_blasResident=41765, rt_blasMB=525. Scene: 14,587 draws emitted from 262,144 dispatched instances; opaque 4980 / mask 1354 / alpha 1880 / rigged opaque 2652 / rigged mask 1215. Particles: ~13–14 k live from 432 emitters.

GPU (ms/frame)

total 13.2 = compute 7.5 + pass1 3.4 + pass2 0.9 + probe 0.5 + sel/hud 0.45 + tonemap/glow 0.4.
Compute breakdown: c_shadows 3.6, c_mirror ~1.8–2.2 (0 in some windows), c_preskin 1.26, c_blas_refit 1.04, c_tlas 0.72, c_cull 0.35, c_stream_inst 0.24.

Leads

LLCleanRTVK::recordBuild (vk_rtBuild, 13.7 ms CPU) is the single biggest item: 45 % of the frame, on the main thread. It walks per-frame loops over the full slot table (static BLAS selection), every instance x 4 LODs (rigged plan), the rigged entries/map, feedback and compaction. With ~42 k resident BLAS and 262 k instance capacity, these O(N) scans per frame are the likely cost. Fixes: incremental/dirty-driven planning, move selection to the GPU TLAS pass (already partly there), and/or run the plan on a worker. Profile with perf record to confirm which loop.
LL_idle / objectListUpdate (~6 ms): LL's per-object idle update over the object list.
Tex-anim face re-registration churn: 405 #VKTexAnim# registerCleanFace log lines in 30 s across 304 distinct objects, some re-registered 5-6 times. Check why texanim faces keep re-registering (sweep / dirty flag), and demote that log line; it's spam at this volume.
c_shadows 3.6 ms GPU: largest GPU item. Revisit once the CPU side is fixed.
Hitches: the section averages don't show the spikes the user feels. Add a max/p99 frame time and a top-sections dump when a frame is over ~2x the average, so the next capture catches the hitch itself.

Capture files (local): log excerpt, per-second GPU samples, per-thread top samples.

30 s capture (2026-09-28 23:31, build `bab6bb454e` + clouds WIP) in a busy sim that the user reports as hitchy. RTX-class GPU, timers and section logs on, RT projector shadows on (56 projectors). ## Summary **CPU-bound, not GPU-bound.** Frames average **30.6–37.5 ms (27–33 fps)**. The GPU needs only **~13.2 ms/frame**; GPU utilization is 45–70 %, with dips to 4–5 %. The main thread (`App`) sits at **99 % CPU**, and the worker pool is at 20–45 %. ## CPU sections (avg ms/frame, last window of 169 frames) | section | ms | |---|---| | frame | 30.6 | | display_total | 24.8 | | renderFrame | 20.3 | | **vk_recordWorld** | **14.5** | | **↳ vk_rtBuild** | **13.7** | | LL_idle_total | 5.7 | | LL_objectListUpdate | 3.0 | | ui_draw | 2.9 | | avatarCharacter | 1.8 (47 avatars, 33 visible) | | vk_ingestTick | 1.8 (up to 3.5) | | vk_sync_total | 1.7 | | buildWorldFrame | 1.6 (bwf_lights 1.1) | | vk_texUpload | 1.5 | RT counters: `rt_blasResident=41765`, `rt_blasMB=525`. Scene: 14,587 draws emitted from 262,144 dispatched instances; opaque 4980 / mask 1354 / alpha 1880 / rigged opaque 2652 / rigged mask 1215. Particles: ~13–14 k live from 432 emitters. ## GPU (ms/frame) total 13.2 = compute 7.5 + pass1 3.4 + pass2 0.9 + probe 0.5 + sel/hud 0.45 + tonemap/glow 0.4. Compute breakdown: **c_shadows 3.6**, c_mirror ~1.8–2.2 (0 in some windows), c_preskin 1.26, c_blas_refit 1.04, c_tlas 0.72, c_cull 0.35, c_stream_inst 0.24. ## Leads 1. **`LLCleanRTVK::recordBuild` (vk_rtBuild, 13.7 ms CPU)** is the single biggest item: 45 % of the frame, on the main thread. It walks per-frame loops over the full slot table (static BLAS selection), every instance x 4 LODs (rigged plan), the rigged entries/map, feedback and compaction. With ~42 k resident BLAS and 262 k instance capacity, these O(N) scans per frame are the likely cost. Fixes: incremental/dirty-driven planning, move selection to the GPU TLAS pass (already partly there), and/or run the plan on a worker. Profile with `perf record` to confirm which loop. 2. **LL_idle / objectListUpdate (~6 ms)**: LL's per-object idle update over the object list. 3. **Tex-anim face re-registration churn**: 405 `#VKTexAnim# registerCleanFace` log lines in 30 s across 304 distinct objects, some re-registered 5-6 times. Check why texanim faces keep re-registering (sweep / dirty flag), and demote that log line; it's spam at this volume. 4. **c_shadows 3.6 ms GPU**: largest GPU item. Revisit once the CPU side is fixed. 5. **Hitches**: the section averages don't show the spikes the user feels. Add a max/p99 frame time and a top-sections dump when a frame is over ~2x the average, so the next capture catches the hitch itself. Capture files (local): log excerpt, per-second GPU samples, per-thread top samples.

sky_line referenced this issue from a commit

2026-09-29 04:15:58 +00:00

RT: stop scanning every instance and the fragmented AS free list per frame

sky_line referenced this issue from a commit

2026-09-29 04:26:43 +00:00

RT: one multi-geometry BLAS per avatar (rigged groups) instead of per face

sky_line referenced this issue from a commit

2026-09-29 05:34:11 +00:00

Hitch logging: hitch lines with the section log, pacing stats, GPU worst frame

sky_line referenced this issue from a commit

2026-09-29 05:34:11 +00:00

Hitches: gate the 5 s HUGE-object scan; back off the mesh GC

sky_line referenced this issue from a commit

2026-09-29 05:34:11 +00:00

VK buffers: grow without full re-upload; staging sized to the upload

sky_line referenced this issue from a commit

2026-09-29 05:34:11 +00:00

VK UI: record texture uploads into the frame instead of submit-and-wait

sky_line referenced this issue

2026-09-29 05:49:02 +00:00

RAM: 16.7 GB of CPU texture copies kept after GPU upload #159

sky_line referenced this issue

2026-09-29 05:49:02 +00:00

VRAM: upload staging rings stay at their peak size (~1.5 GB host-visible) #160

sky_line referenced this issue

2026-09-29 05:49:02 +00:00

VRAM: ray-tracing working buffers (rt-aux 643 MB) sized for the worst case #161

sky_line referenced this issue

2026-09-29 05:49:03 +00:00

Track RAM + VRAM per release (baseline 2026-09-29) so regressions are caught #163

sky_line referenced this issue

2026-09-29 06:00:00 +00:00

VRAM: ray tracing must fit a VRAM-scaled budget, below textures (8 GB cards starve) #164

sky_line referenced this issue from a commit

2026-09-29 06:08:23 +00:00

GPU timers: sub-marks for pass 2 and the post prelude

sky_line referenced this issue

2026-09-29 15:55:11 +00:00

Perf: frame rate degrades after moving between sims (leaks / leftover per-frame work) #169

sky_line referenced this issue

2026-09-29 22:12:39 +00:00

Ray-traced sun and moon shadows (RenderVKSunShadows 2) #185

sky_line referenced this issue

2026-09-30 03:15:29 +00:00

RT: opacity micromaps (VK_EXT_opacity_micromap) for hardware alpha-tested shadows, shader alpha-test fallback #194

sky_line referenced this issue

2026-09-30 03:17:38 +00:00

RT sun shadows: further optimization (shadow-mask pass, reduced-rate tracing, hybrid edges, denoise) #195

sky_line commented

2026-10-01 01:22:47 +00:00

Author

Owner

Current numbers (2026-09-30, 4d377811f1): vk_rtBuild = 0.33 ms/frame on the main thread (64.8k TLAS instances, 15.2k resident BLAS, ~16 rigged refits/frame); the frame is GPU-bound (CPU ~9-10 ms of a 26 ms frame). Down from 13.7 ms. Closing.

Current numbers (2026-09-30, 4d377811f1): vk_rtBuild = 0.33 ms/frame on the main thread (64.8k TLAS instances, 15.2k resident BLAS, ~16 rigged refits/frame); the frame is GPU-bound (CPU ~9-10 ms of a 26 ms frame). Down from 13.7 ms. Closing.

sky_line closed this issue

2026-10-01 01:22:47 +00:00

Rows
Columns

Perf: busy sim is CPU-bound at ~31 fps — RT build is 13.7 ms/frame on the main thread #157

Summary

CPU sections (avg ms/frame, last window of 169 frames)

GPU (ms/frame)

Leads