Linux Wayland: ship libdecor + only prefer native Wayland when decorations are guaranteed #197

New issue

Closed

opened 2026-09-30 04:23:47 +00:00 by sky_line · 1 comment

sky_line commented

2026-09-30 04:23:47 +00:00

Owner

Implementation ticket for #196 (no window border on Ubuntu 24.04 GNOME, RX 9070 XT). The same reporter also sees poor FPS versus Alchemy, and this is probably one of the causes.

Background / root cause

Commit 6639761b61 ("High-DPI window and native Wayland", first in v0.11.0-beta.1) added SDL_SetHint(SDL_HINT_VIDEO_DRIVER, "wayland,x11") at indra/llwindow/llsdl.cpp:114 (Linux only). That forces native Wayland whenever a Wayland session exists. SDL 3.2's own default would put a GNOME 46 session on X11/XWayland, which is why Alchemy has a border on the same machine.
Reporter log: SDL 3.2.22, SDL video driver: wayland, VK_KHR_wayland_surface, Ubuntu 24.04 (kernel 7.0.0-31-generic), GNOME, RX 9070 XT (RADV GFX1201).
GNOME/Mutter does not implement zxdg_decoration_manager_v1 (server-side decorations). A native Wayland client on GNOME has to draw its own frame. SDL3 does this by dlopen'ing libdecor-0.so.0 at runtime, and libdecor in turn loads a plugin (libdecor-cairo.so or libdecor-gtk.so) from its plugin dir.
We don't ship libdecor. Our libSDL3.so.0.2.22 is built with libdecor support and dlopens libdecor-0.so.0 (confirmed via strings). The dev machine (Arch) has libdecor 0.2.5 installed, so the bug never showed locally. On Ubuntu, libdecor-0-0 / libdecor-0-plugin-1-gtk are not guaranteed to be installed. Without them SDL creates a plain xdg_toplevel with no decorations: no title bar, no border, and no move/resize handles.
Frame-pacing side effect: GNOME 46 lacks the fifo-v1 / commit-timing-v1 protocols. With RenderVSyncEnable=1 (the default) we present FIFO, and Mesa's Wayland WSI then paces on frame callbacks, which Mutter stops sending for occluded surfaces. This is one reason SDL 3.2 doesn't prefer Wayland by default on such compositors.

Workaround for users today: SDL_VIDEO_DRIVER=x11 ./slipstream. The env var overrides our hint, because we set it at normal priority.

Plan

1. Ship libdecor + cairo plugin

Add libdecor-0.so.0 and plugins/libdecor-cairo.so to the Linux package. Use a 3p package or copy from the build host in viewer_manifest.py (LinuxManifest). The cairo plugin is preferred over gtk because it only needs cairo/pango/dbus, which any desktop has, instead of GTK3.
In the launcher (indra/newview/linux_tools/wrapper.sh → packaged/slipstream), export LIBDECOR_PLUGIN_DIR="$RUN_PATH/lib/libdecor/plugins" unless the user already set it.
Make sure our bundled copy is the one SDL finds: LD_LIBRARY_PATH=./lib is already set by the launcher.
Watch the glibc floor: build or pin it so make glibc-check still passes (see the glibc_compat.h / fix_isa_note.py tooling).

2. Safety net: only prefer Wayland when a frame is guaranteed

In llsdl.cpp, before SDL_Init, replace the unconditional "wayland,x11" hint:

If SDL_VIDEO_DRIVER is set in the environment, respect it (already works).
If WAYLAND_DISPLAY is set and either (a) the compositor advertises zxdg_decoration_manager_v1 (KDE, wlroots, Hyprland, etc.), or (b) dlopen("libdecor-0.so.0") succeeds and a plugin is present, then prefer wayland,x11.
Otherwise, e.g. GNOME without libdecor, prefer x11,wayland.
For (a), a throwaway wl_display_connect + registry roundtrip is enough, or just treat XDG_CURRENT_DESKTOP containing GNOME as needing libdecor.
Log the decision on one line, e.g. Video driver: wayland (decorations: libdecor/cairo) or x11 (GNOME without libdecor), so reports make it obvious.

3. User override

Add a setting (e.g. LinuxVideoDriver: auto / wayland / x11), read before SDL init and restart-required. Also add a --x11 launcher flag.

On compositors without fifo-v1, consider MAILBOX (if available) or FIFO_RELAXED under Wayland, or at least document the occlusion stall. vk_swapchain.cpp currently uses FIFO whenever vsync is on.

Test matrix

Ubuntu 24.04 GNOME (Wayland) without libdecor installed: should get a border (bundled libdecor), or fall back to X11 if that's disabled.
Ubuntu 24.04 GNOME on X11 session: unchanged.
KDE Plasma 6 Wayland: native Wayland, server-side decorations, fractional scaling intact.
Hyprland/sway: native Wayland (tiling, decorations irrelevant).
Arch/CachyOS dev box: no regression.
Check the log line and make glibc-check.

Refs #196

Implementation ticket for #196 (no window border on Ubuntu 24.04 GNOME, RX 9070 XT). The same reporter also sees poor FPS versus Alchemy, and this is probably one of the causes. ## Background / root cause - Commit 6639761b61 ("High-DPI window and native Wayland", first in v0.11.0-beta.1) added `SDL_SetHint(SDL_HINT_VIDEO_DRIVER, "wayland,x11")` at `indra/llwindow/llsdl.cpp:114` (Linux only). That forces native Wayland whenever a Wayland session exists. SDL 3.2's own default would put a GNOME 46 session on X11/XWayland, which is why Alchemy has a border on the same machine. - Reporter log: `SDL 3.2.22`, `SDL video driver: wayland`, `VK_KHR_wayland_surface`, Ubuntu 24.04 (kernel 7.0.0-31-generic), GNOME, RX 9070 XT (RADV GFX1201). - **GNOME/Mutter does not implement `zxdg_decoration_manager_v1`** (server-side decorations). A native Wayland client on GNOME has to draw its own frame. SDL3 does this by **dlopen'ing `libdecor-0.so.0`** at runtime, and libdecor in turn loads a plugin (`libdecor-cairo.so` or `libdecor-gtk.so`) from its plugin dir. - **We don't ship libdecor.** Our `libSDL3.so.0.2.22` is built with libdecor support and dlopens `libdecor-0.so.0` (confirmed via `strings`). The dev machine (Arch) has `libdecor 0.2.5` installed, so the bug never showed locally. On Ubuntu, `libdecor-0-0` / `libdecor-0-plugin-1-gtk` are not guaranteed to be installed. Without them SDL creates a plain `xdg_toplevel` with no decorations: no title bar, no border, and no move/resize handles. - **Frame-pacing side effect:** GNOME 46 lacks the `fifo-v1` / `commit-timing-v1` protocols. With `RenderVSyncEnable=1` (the default) we present FIFO, and Mesa's Wayland WSI then paces on frame callbacks, which Mutter stops sending for occluded surfaces. This is one reason SDL 3.2 doesn't prefer Wayland by default on such compositors. **Workaround for users today:** `SDL_VIDEO_DRIVER=x11 ./slipstream`. The env var overrides our hint, because we set it at normal priority. ## Plan ### 1. Ship libdecor + cairo plugin - Add `libdecor-0.so.0` and `plugins/libdecor-cairo.so` to the Linux package. Use a 3p package or copy from the build host in `viewer_manifest.py` (LinuxManifest). The cairo plugin is preferred over gtk because it only needs cairo/pango/dbus, which any desktop has, instead of GTK3. - In the launcher (`indra/newview/linux_tools/wrapper.sh` → `packaged/slipstream`), export `LIBDECOR_PLUGIN_DIR="$RUN_PATH/lib/libdecor/plugins"` unless the user already set it. - Make sure our bundled copy is the one SDL finds: `LD_LIBRARY_PATH=./lib` is already set by the launcher. - Watch the glibc floor: build or pin it so `make glibc-check` still passes (see the glibc_compat.h / fix_isa_note.py tooling). ### 2. Safety net: only prefer Wayland when a frame is guaranteed In `llsdl.cpp`, before `SDL_Init`, replace the unconditional `"wayland,x11"` hint: - If `SDL_VIDEO_DRIVER` is set in the environment, respect it (already works). - If `WAYLAND_DISPLAY` is set and either (a) the compositor advertises `zxdg_decoration_manager_v1` (KDE, wlroots, Hyprland, etc.), or (b) `dlopen("libdecor-0.so.0")` succeeds **and** a plugin is present, then prefer `wayland,x11`. - Otherwise, e.g. GNOME without libdecor, prefer `x11,wayland`. - For (a), a throwaway `wl_display_connect` + registry roundtrip is enough, or just treat `XDG_CURRENT_DESKTOP` containing GNOME as needing libdecor. - Log the decision on one line, e.g. `Video driver: wayland (decorations: libdecor/cairo)` or `x11 (GNOME without libdecor)`, so reports make it obvious. ### 3. User override Add a setting (e.g. `LinuxVideoDriver`: auto / wayland / x11), read before SDL init and restart-required. Also add a `--x11` launcher flag. ### 4. (Related) Wayland vsync pacing On compositors without `fifo-v1`, consider MAILBOX (if available) or FIFO_RELAXED under Wayland, or at least document the occlusion stall. `vk_swapchain.cpp` currently uses FIFO whenever vsync is on. ## Test matrix - Ubuntu 24.04 GNOME (Wayland) **without** libdecor installed: should get a border (bundled libdecor), or fall back to X11 if that's disabled. - Ubuntu 24.04 GNOME on X11 session: unchanged. - KDE Plasma 6 Wayland: native Wayland, server-side decorations, fractional scaling intact. - Hyprland/sway: native Wayland (tiling, decorations irrelevant). - Arch/CachyOS dev box: no regression. - Check the log line and `make glibc-check`. Refs #196

sky_line referenced this issue from a commit

2026-10-04 06:45:16 +00:00

Launcher: point libdecor at the bundled plugin when the system has none; --x11/--wayland (#197)

sky_line referenced this issue from a commit

2026-10-04 06:45:16 +00:00

Preferences: Display backend (Auto / Wayland / X11) on Linux (#197)

sky_line referenced this issue from a commit

2026-10-04 06:45:16 +00:00

llsdl: keep the test-loaded bundled libdecor plugin loaded (#197)

sky_line added this to the Slipstream project

2026-10-06 15:04:15 +00:00

sky_line added the

status

testing

label

2026-10-09 04:40:37 +00:00

sky_line commented

2026-10-12 15:02:34 +00:00

Author

Owner

Shipped in 1.0.0 (bundled libdecor, native Wayland only when decorations are available) and in every release since, with no further Wayland decoration reports. Closing.

sky_line

2026-10-12 15:02:34 +00:00