UI render backend: GPU-driven RmlUi render interface in the Vulkan renderer (glass, multi-window, VR panels)

sky_line commented

2026-10-12 09:20:21 +00:00

Owner

Decisions (Sky, 2026-10-28) — supersede the original "backend under LLView" scope

The UI moves to RmlUi (C++, MIT): a whole rewrite of the viewer UI in one coordinated effort, with a temporary "UI: Classic / New (restart required)" switch on main while both exist; screens are rebuilt fresh (not mechanically converted from the old XUI); the old toolkit is removed when the new UI is complete. Glass UI (#1242) is built on top of this.

This ticket = the GPU-driven UI render backend, implemented as RmlUi's RenderInterface inside our Vulkan renderer:

Geometry: persistent / pooled vertex+index buffers, batched draws (indirect where it pays), bindless textures (shared with the world's descriptor-indexed textures), scissor and transforms, no per-element CPU uploads when nothing changed.
RmlUi 6 layers / filters / shaders: backdrop-filter blur (frosted glass: blur of the scene behind panels, cheap, at reduced resolution), drop shadows, masks, gradients.
Fonts: GPU glyph atlas (SDF or MSDF for crisp scaling), HiDPI / per-monitor scale.
Targets: the main window, extra OS windows (one RmlUi context per detached window, using the multi-window / swapchain layer from #1013), and in-world / VR panels (render to texture).
Fixed cost, bounded memory (atlas / pool budgets, LRU), allocation failures never crash; measured with GPU timers; must not cost the world renderer frame time when the UI is static.

Related: #1242 (glass UI), #1013 (pop-out windows: multi-window layer).

Original description:

Measured (2026-10-12, perf sample, light UI): UI costs ~1.3 ms CPU per frame: ~1.0 ms walking the LLView tree into gGL (retained floater replay ~0.4, live widgets ~0.4, mini-map ~0.15) and ~0.3 ms recording one vkCmdDraw + descriptor bind per batch. Batches break on every texture / blend / scissor / matrix change, and even retained floaters are replayed vertex by vertex every frame.

Proposal: keep the LLView tree and XUI; replace the drawing layer with the world renderer's pattern.

Widgets write a few fixed-size records (rect, clip, bindless texture / atlas uv, colour, corner radius, border, kind) into a persistent UI instance buffer, only when dirty.
Text shaped once per change into glyph runs over an MSDF atlas (sharp at any UI scale).
Each floater owns a record range and a z-order; moving a floater updates one transform.
Per frame: upload dirty ranges (usually none), one compute pass clips and builds the draw list, 1-2 indirect draws.
Virtualised lists (inventory, scroll lists, chat) as a follow-up.

Expected: ~1.3 ms -> ~0.2 ms CPU per frame in a light UI, more with heavy UI; sharper text and shapes at any scale. Needs a design pass before implementation.

## Decisions (Sky, 2026-10-28) — supersede the original "backend under LLView" scope The UI moves to **RmlUi** (C++, MIT): a whole rewrite of the viewer UI in one coordinated effort, with a temporary **"UI: Classic / New (restart required)"** switch on main while both exist; screens are rebuilt fresh (not mechanically converted from the old XUI); the old toolkit is removed when the new UI is complete. Glass UI (#1242) is built on top of this. **This ticket = the GPU-driven UI render backend, implemented as RmlUi's `RenderInterface` inside our Vulkan renderer:** - Geometry: persistent / pooled vertex+index buffers, batched draws (indirect where it pays), bindless textures (shared with the world's descriptor-indexed textures), scissor and transforms, no per-element CPU uploads when nothing changed. - RmlUi 6 layers / filters / shaders: **`backdrop-filter` blur** (frosted glass: blur of the scene behind panels, cheap, at reduced resolution), drop shadows, masks, gradients. - Fonts: GPU glyph atlas (SDF or MSDF for crisp scaling), HiDPI / per-monitor scale. - Targets: the main window, **extra OS windows** (one RmlUi context per detached window, using the multi-window / swapchain layer from #1013), and **in-world / VR panels** (render to texture). - Fixed cost, bounded memory (atlas / pool budgets, LRU), allocation failures never crash; measured with GPU timers; must not cost the world renderer frame time when the UI is static. Related: #1242 (glass UI), #1013 (pop-out windows: multi-window layer). --- **Original description:** **Measured (2026-10-12, perf sample, light UI):** UI costs ~1.3 ms CPU per frame: ~1.0 ms walking the `LLView` tree into `gGL` (retained floater replay ~0.4, live widgets ~0.4, mini-map ~0.15) and ~0.3 ms recording one `vkCmdDraw` + descriptor bind per batch. Batches break on every texture / blend / scissor / matrix change, and even retained floaters are replayed vertex by vertex every frame. **Proposal:** keep the `LLView` tree and XUI; replace the drawing layer with the world renderer's pattern. - Widgets write a few fixed-size records (rect, clip, bindless texture / atlas uv, colour, corner radius, border, kind) into a persistent UI instance buffer, only when dirty. - Text shaped once per change into glyph runs over an MSDF atlas (sharp at any UI scale). - Each floater owns a record range and a z-order; moving a floater updates one transform. - Per frame: upload dirty ranges (usually none), one compute pass clips and builds the draw list, 1-2 indirect draws. - Virtualised lists (inventory, scroll lists, chat) as a follow-up. **Expected:** ~1.3 ms -> ~0.2 ms CPU per frame in a light UI, more with heavy UI; sharper text and shapes at any scale. Needs a design pass before implementation.

sky_line added the

triage

label

2026-10-12 09:20:21 +00:00

sky_line added this to the Slipstream project

2026-10-12 09:20:21 +00:00

sky_line changed title from ~~UI drawing: GPU-driven backend under LLView (instance records, bindless, indirect draws)~~ to UI render backend: GPU-driven RmlUi render interface in the Vulkan renderer (glass, multi-window, VR panels)

2026-10-29 02:50:34 +00:00

sky_line added

status

in-progress

and removed

triage

labels

2026-10-29 02:50:34 +00:00

sky_line self-assigned this

2026-10-29 02:50:34 +00:00

sky_line referenced this issue

2026-10-29 06:21:29 +00:00