Post-teleport hitch when old regions drop: ~7.8k objects torn down in single frames -- time-slice / batch region teardown #433

New issue

Closed

opened 2026-10-05 11:52:46 +00:00 by sky_line · 1 comment

sky_line commented

2026-10-05 11:52:46 +00:00

Owner

After a teleport, when the regions left behind are dropped, there's still a noticeable hitch. It's much better since the VRAM work (texture frees are now cheap pool frees, #411), but not gone. Related: #290 (a 2-3 s hitch ~30-60 s after a region unloads, deferred asset cleanup in one burst) and #405.

Evidence (owner's session, 2026-10-05 ~11:51Z, teleport into Cloudsdale)

11:51:07 teleport finished
11:51:19 dropRegion: Dropping region Baltimare ...: outside the ring (dropped 1, connected 3)
11:51:19 removeRegion: Removing region 208128:295168
11:51:19 clearAllMapObjectsInRegion: There are 4421 objects not removed from the deleted region!
11:51:20 dropRegion: Dropping region Horse Heights ...: unreachable (dropped 2, connected 2)
11:51:20 removeRegion: Removing region 207872:295168
11:51:20 clearAllMapObjectsInRegion: There are 3375 objects not removed from the deleted region!

~7,800 objects are torn down in two single frames. LLWorld::removeRegion → LLViewerRegion destructor / killAllObjects-style path, which per object does:

markDead;
drawable / octree removal;
the VK instance store release (LLCleanInstanceStore::releaseObject);
mesh / LOD-group refs and the GPU geometry free bookkeeping;
the name / selection / map lists (clearAllMapObjectsInRegion warns that thousands were still on the map list);
audio sources, media, and particles.

Fix direction

Time-slice the region teardown:
- On drop, immediately make the region's objects invisible and inert (cheap: stop drawing / culling them; GPU instances can be flagged hidden in one pass).
- Kill them over the next frames with a per-frame budget (count and/or ~1-2 ms), largest/oldest first.
- The region object itself goes away once its objects are gone (or hold a lightweight shell).
Batch the per-object work:
- one octree / spatial-partition teardown per region (drop whole partitions instead of removing objects one by one);
- batched instance-store releases (one dirty-range upload instead of thousands);
- batched mesh ref decrements.
Fix the "objects not removed from the deleted region" path: clearAllMapObjectsInRegion finding 4421 / 3375 leftovers means map-list removal isn't happening per object as they die (or the region is cleared before its objects). Remove the region's map entries in one sweep (the map list is now index-based after #316), or tear down in an order that avoids the warning.
Coordinate with #290: the deferred cleanup burst 30-60 s later (asset/texture/mesh frees) should also be budgeted per frame (it may be largely fixed by the texture pools; re-measure).
Measure:
- add a hitch-counter scope around removeRegion and the deferred cleanup, logging object count + ms per drop;
- teleport between busy regions and compare frame-time spikes before and after.

Done when

Dropping a region with thousands of objects causes no frame over a small budget (e.g. no spike > ~16 ms attributable to the teardown).
No "objects not removed from the deleted region" warnings in normal teleports.
#290's deferred burst re-measured and budgeted if still present.

After a teleport, when the regions left behind are dropped, there's still a **noticeable hitch**. It's much better since the VRAM work (texture frees are now cheap pool frees, #411), but not gone. Related: #290 (a 2-3 s hitch ~30-60 s after a region unloads, deferred asset cleanup in one burst) and #405. ## Evidence (owner's session, 2026-10-05 ~11:51Z, teleport into Cloudsdale) ``` 11:51:07 teleport finished 11:51:19 dropRegion: Dropping region Baltimare ...: outside the ring (dropped 1, connected 3) 11:51:19 removeRegion: Removing region 208128:295168 11:51:19 clearAllMapObjectsInRegion: There are 4421 objects not removed from the deleted region! 11:51:20 dropRegion: Dropping region Horse Heights ...: unreachable (dropped 2, connected 2) 11:51:20 removeRegion: Removing region 207872:295168 11:51:20 clearAllMapObjectsInRegion: There are 3375 objects not removed from the deleted region! ``` **~7,800 objects are torn down in two single frames.** `LLWorld::removeRegion` → `LLViewerRegion` destructor / `killAllObjects`-style path, which per object does: - markDead; - drawable / octree removal; - the VK instance store release (`LLCleanInstanceStore::releaseObject`); - mesh / LOD-group refs and the GPU geometry free bookkeeping; - the name / selection / map lists (`clearAllMapObjectsInRegion` warns that thousands were still on the map list); - audio sources, media, and particles. ## Fix direction 1. **Time-slice the region teardown:** - On drop, immediately make the region's objects **invisible and inert** (cheap: stop drawing / culling them; GPU instances can be flagged hidden in one pass). - Kill them over the next frames with a per-frame budget (count and/or ~1-2 ms), largest/oldest first. - The region object itself goes away once its objects are gone (or hold a lightweight shell). 2. **Batch the per-object work:** - one octree / spatial-partition teardown per region (drop whole partitions instead of removing objects one by one); - batched instance-store releases (one dirty-range upload instead of thousands); - batched mesh ref decrements. 3. **Fix the "objects not removed from the deleted region" path:** `clearAllMapObjectsInRegion` finding 4421 / 3375 leftovers means map-list removal isn't happening per object as they die (or the region is cleared before its objects). Remove the region's map entries in one sweep (the map list is now index-based after #316), or tear down in an order that avoids the warning. 4. **Coordinate with #290:** the deferred cleanup burst 30-60 s later (asset/texture/mesh frees) should also be budgeted per frame (it may be largely fixed by the texture pools; re-measure). 5. **Measure:** - add a hitch-counter scope around `removeRegion` and the deferred cleanup, logging object count + ms per drop; - teleport between busy regions and compare frame-time spikes before and after. ## Done when - [ ] Dropping a region with thousands of objects causes no frame over a small budget (e.g. no spike > ~16 ms attributable to the teardown). - [ ] No "objects not removed from the deleted region" warnings in normal teleports. - [ ] #290's deferred burst re-measured and budgeted if still present.

sky_line added the

perf

bug

labels

2026-10-05 11:52:46 +00:00

sky_line referenced this issue

2026-10-05 22:16:28 +00:00

Test plan: fly across all of Zindra (mainland) at max flight speed -- the region-crossing / streaming / memory endurance test #469

sky_line added this to the Slipstream project

2026-10-06 15:04:14 +00:00

sky_line added the

status