Explore Rust in the viewer: a pilot for untrusted-input parsers behind a C ABI; wholesale rewrites later if they clearly help #1602

Open
opened 2026-11-17 23:15:55 +00:00 by sky_line · 0 comments
Owner

Exploratory. Where would Rust make the viewer safer, and how do we add it without hurting the build or the C++ codebase?

Why

Memory-safety bugs in C++ show up as crashes users can't report, for example the recent texture-residency heap fault and the startup crash where a device create info was walked after a third party had changed it. Rust's ownership and bounds checks rule out whole classes of them at compile time. AI-assisted work benefits too: the compiler catches more mistakes before they ship. Rust is not faster than C++ by itself; the case for it is safety and maintainability.

Where it makes sense first: untrusted input behind narrow interfaces

  • Network and protocol parsing: LLSD binary/XML, message decoding, the SlimeVR OSC/VMC parser (#1582).
  • Asset decoders: mesh, image formats, animation, sound.
  • File and archive handling: skin import (#1599), viewer import, notecard/script upload.
  • Data from other processes: the compile-worker protocol, and anything else read from another process.

Where it does not make sense, for now

The renderer core and anything woven through LL's object model: texture residency, the stores, avatars, the UI. Too wide a boundary for the gain; better C++ ownership, sanitizer runs and tests are the realistic fix there.

Approach

  1. Pilot: one self-contained parser as a Rust crate behind a C ABI (or cxx), for example the OSC/VMC parser or the skin import unpacker.
    • A CMake option to turn it on, built through Corrosion.
    • A fuzz target (cargo-fuzz).
    • CI on Linux and the Windows cross-build (clang-cl; cargo-xwin or equivalent).
  2. Measure the cost:
    • build time;
    • CI complexity;
    • debugging across the boundary (mixed stacks, symbols in crash reports);
    • panic handling: no unwinding across FFI, abort-or-catch policy;
    • the binary size;
    • what the other developers need to learn.
  3. Decide: if the workflow holds up, move the other untrusted-input parsers one at a time, each behind the same C ABI pattern.
  4. Later, only if time allows and it clearly helps: consider wholesale rewrites of existing systems in Rust. That fits the long-term goal of replacing LL code with our own, but is not a commitment.

Questions to answer

  • Toolchain pinning and offline builds (our 3p manifest model).
  • The Flatpak / AUR / apt packaging impact.
  • Licensing of crates: compatible with LGPL 2.1, and our rule of preferring our own implementations.
  • Rust's allocator vs ours (memory accounting in the logs).
Exploratory. Where would Rust make the viewer safer, and how do we add it without hurting the build or the C++ codebase? ## Why Memory-safety bugs in C++ show up as crashes users can't report, for example the recent texture-residency heap fault and the startup crash where a device create info was walked after a third party had changed it. Rust's ownership and bounds checks rule out whole classes of them at compile time. AI-assisted work benefits too: the compiler catches more mistakes before they ship. Rust is not faster than C++ by itself; the case for it is safety and maintainability. ## Where it makes sense first: untrusted input behind narrow interfaces - **Network and protocol parsing:** LLSD binary/XML, message decoding, the SlimeVR OSC/VMC parser (#1582). - **Asset decoders:** mesh, image formats, animation, sound. - **File and archive handling:** skin import (#1599), viewer import, notecard/script upload. - **Data from other processes:** the compile-worker protocol, and anything else read from another process. ## Where it does not make sense, for now The renderer core and anything woven through LL's object model: texture residency, the stores, avatars, the UI. Too wide a boundary for the gain; better C++ ownership, sanitizer runs and tests are the realistic fix there. ## Approach 1. **Pilot:** one self-contained parser as a Rust crate behind a C ABI (or `cxx`), for example the OSC/VMC parser or the skin import unpacker. - A CMake option to turn it on, built through Corrosion. - A fuzz target (cargo-fuzz). - CI on Linux and the Windows cross-build (clang-cl; `cargo-xwin` or equivalent). 2. **Measure the cost:** - build time; - CI complexity; - debugging across the boundary (mixed stacks, symbols in crash reports); - panic handling: no unwinding across FFI, abort-or-catch policy; - the binary size; - what the other developers need to learn. 3. **Decide:** if the workflow holds up, move the other untrusted-input parsers one at a time, each behind the same C ABI pattern. 4. **Later, only if time allows and it clearly helps:** consider wholesale rewrites of existing systems in Rust. That fits the long-term goal of replacing LL code with our own, but is not a commitment. ## Questions to answer - Toolchain pinning and offline builds (our 3p manifest model). - The Flatpak / AUR / apt packaging impact. - Licensing of crates: compatible with LGPL 2.1, and our rule of preferring our own implementations. - Rust's allocator vs ours (memory accounting in the logs).
sky_line added this to the Slipstream project 2026-11-17 23:15:56 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
sky_line/Slipstream#1602
No description provided.