Picking rigged mesh (worn clothing / bodies / animesh) almost never works, especially in edit mode

sky_line commented

2026-10-02 21:12:46 +00:00

Owner

Report

Clicking on rigged mesh (worn mesh bodies, clothing, rigged attachments, animesh) is extremely difficult, and in edit / build mode it essentially never works. You can't select or right-click the rigged item you're pointing at.

Background / suspects

LL picks rigged mesh against its skinned positions. LL's LLVOVolume::lineSegmentIntersect for rigged faces uses the CPU-skinned volume (getRiggedVolume / updateRiggedVolume) only when "pick rigged" is enabled (gPipeline / LLDrawable flag, RenderRiggedPicking / "Select rigged mesh" in the build tools, gFloaterTools "edit linked" + rigged).
Under Vulkan, skinning happens on the GPU (pre-skin compute). The CPU rigged volume may never be updated (the GL removal took out CPU skinning paths), so picking tests the bind-pose mesh. The ray misses wherever the pose differs from bind pose, which is nearly everywhere.
Edit mode: the build tools' pick path (LLToolSelect / LLToolCompTranslate / gViewerWindow->pickImmediate with pick_rigged) may additionally require mRiggedVolume to be valid, or may exclude rigged entirely unless "select rigged" is on.
Related history: #120 (only part of a mesh is clickable), and the "rigged right-click drop" memory note.
Options:
- (a) GPU pick: use the VK picking infrastructure (there's a VK pick path for HUDs, vkPickHUD, and RT primary rays exist: debug view 13/14 show per-pixel hit instance ids). A ray-query against the TLAS (which holds the skinned rig positions) at the cursor gives the exact rigged instance / face under the mouse. This is the standard GPU-driven way and matches what's drawn.
- (b) CPU: read back the pre-skinned positions for the candidate instance and intersect on the CPU.
- (a) is preferred where RT is available; (b) or a pre-skin readback as the fallback for non-RT GPUs.

Acceptance

Clicking, right-clicking and hover on worn rigged clothing / bodies and animesh selects the item under the cursor (matching what's drawn), both normally and in edit mode with "Select rigged mesh" (or the equivalent) on.
Works on avatars in non-bind poses (sitting, animating).

## Report Clicking on **rigged mesh** (worn mesh bodies, clothing, rigged attachments, animesh) is extremely difficult, and in **edit / build mode it essentially never works**. You can't select or right-click the rigged item you're pointing at. ## Background / suspects - **LL picks rigged mesh against its *skinned* positions.** LL's `LLVOVolume::lineSegmentIntersect` for rigged faces uses the CPU-skinned volume (`getRiggedVolume` / `updateRiggedVolume`) only when "pick rigged" is enabled (`gPipeline` / `LLDrawable` flag, `RenderRiggedPicking` / "Select rigged mesh" in the build tools, `gFloaterTools` "edit linked" + rigged). - **Under Vulkan, skinning happens on the GPU (pre-skin compute).** The CPU rigged volume may never be updated (the GL removal took out CPU skinning paths), so picking tests the **bind-pose** mesh. The ray misses wherever the pose differs from bind pose, which is nearly everywhere. - **Edit mode:** the build tools' pick path (`LLToolSelect` / `LLToolCompTranslate` / `gViewerWindow->pickImmediate` with `pick_rigged`) may additionally require `mRiggedVolume` to be valid, or may exclude rigged entirely unless "select rigged" is on. - Related history: #120 (only part of a mesh is clickable), and the "rigged right-click drop" memory note. - Options: - **(a) GPU pick:** use the VK picking infrastructure (there's a VK pick path for HUDs, `vkPickHUD`, and RT primary rays exist: debug view 13/14 show per-pixel hit instance ids). A ray-query against the TLAS (which holds the skinned rig positions) at the cursor gives the exact rigged instance / face under the mouse. This is the standard GPU-driven way and matches what's drawn. - **(b) CPU:** read back the pre-skinned positions for the candidate instance and intersect on the CPU. - (a) is preferred where RT is available; (b) or a pre-skin readback as the fallback for non-RT GPUs. ## Acceptance - Clicking, right-clicking and hover on worn rigged clothing / bodies and animesh selects the item under the cursor (matching what's drawn), both normally and in edit mode with "Select rigged mesh" (or the equivalent) on. - Works on avatars in non-bind poses (sitting, animating).

sky_line added the

bug

avatars

labels

2026-10-02 21:12:46 +00:00

sky_line commented

2026-10-02 21:49:01 +00:00

Author

Owner

Fix on branch rigpick (81dacb7cea, 37d4fd3b62, 4005fa4ae8, ab198588c8). Not built or run yet: syntax checks only.

Root cause

Rigged geometry is skinned on the GPU (and other avatars are GPU-animated), so LLVOVolume::lineSegmentIntersect tested the bind pose. Worn rigged items went through vkIntersectAttachmentObject: the bind-pose volume times the attach-point matrix, which is the wrong space altogether. Animesh went through a CPU LLRiggedVolume built from stale, sparsely read-back joints.
The VK attachment branch also dropped LL's rule that your own rigged items are pickable while the build floater is open. So edit mode never picked them.

Fix: GPU cursor pick (vkgfx/llcleanpickvk.*, llcleanrigpick.*, shaders clean_pick_*)

Every frame the cursor ray (built the same way as cursorIntersect) is sent to the renderer.
- RT path: when this frame's TLAS is valid, one ray-query dispatch at the end of the main view. It tests rigged instances only, with the raster's facing and alpha rules. One traversal gives both answers LLToolPie needs: nearest with alpha and nearest ignoring alpha.
- Fallback (no RT): a compute pass tests the ray against the triangles of the main view's pre-skinned vertices (cand[] plus mSkinVbo). It only looks at rigged items whose avatar's bounds the ray crosses.
Answers are read back about 2 frames later.
On a click whose ray no per-frame answer covers (camera or cursor moving), the ray is answered on the spot against the last frame's TLAS. That waits for the frames in flight, so there's a short stall on that click only.
How a hit is reported, following LL's rules:
- Worn rigged item: the item itself on right-click, or for your own items in build mode. Otherwise the wearer, so clicking visible clothing counts as clicking the avatar.
- Animesh: the object, on left-click too. This is a deliberate change from LL, where AnimatedObjectsAllowLeftClick defaults to 0.
- System body: the avatar.
- LL's selectable, click-action-ignore and hidden-self filters still apply.
Face select, touch and media UVs use the GPU hit (face, texcoord, normal, tangent).

Preference: Preferences > Advanced > "Accurate rigged mesh picking" (RenderVKPickRigged).

0 = off.
1 = with ray tracing (default). Costs about nothing: one ray per frame.
2 = also without RT. The fallback reads about 100 bytes per triangle of the rigged items along the cursor ray, every frame. My estimate is about 0.1 to 0.3 ms per avatar on a mid-range GPU, and several ms in crowds on slow or integrated GPUs, so it is opt-in.

Caveat: mode 1 needs something that uses ray tracing to be on (ray-traced sun, ray-traced projector shadows, ray-traced probes, or an RT debug view), not just RT hardware. With none of those on, the TLAS isn't built and rigged mesh isn't pickable at the default setting. That's no worse than before, when picking used the bind pose. Mode 2 covers that case.

Fix on branch `rigpick` (81dacb7cea, 37d4fd3b62, 4005fa4ae8, ab198588c8). Not built or run yet: syntax checks only. **Root cause** - Rigged geometry is skinned on the GPU (and other avatars are GPU-animated), so `LLVOVolume::lineSegmentIntersect` tested the **bind pose**. Worn rigged items went through `vkIntersectAttachmentObject`: the bind-pose volume times the attach-point matrix, which is the wrong space altogether. Animesh went through a CPU `LLRiggedVolume` built from stale, sparsely read-back joints. - The VK attachment branch also dropped LL's rule that **your own rigged items are pickable while the build floater is open**. So edit mode never picked them. **Fix: GPU cursor pick** (`vkgfx/llcleanpickvk.*`, `llcleanrigpick.*`, shaders `clean_pick_*`) - Every frame the cursor ray (built the same way as `cursorIntersect`) is sent to the renderer. - **RT path:** when this frame's TLAS is valid, one ray-query dispatch at the end of the main view. It tests rigged instances only, with the raster's facing and alpha rules. One traversal gives both answers LLToolPie needs: nearest with alpha and nearest ignoring alpha. - **Fallback (no RT):** a compute pass tests the ray against the triangles of the main view's pre-skinned vertices (`cand[]` plus `mSkinVbo`). It only looks at rigged items whose avatar's bounds the ray crosses. - Answers are read back about 2 frames later. - On a click whose ray no per-frame answer covers (camera or cursor moving), the ray is answered on the spot against the last frame's TLAS. That waits for the frames in flight, so there's a short stall on that click only. - How a hit is reported, following LL's rules: - **Worn rigged item:** the item itself on right-click, or for your own items in build mode. Otherwise the **wearer**, so clicking visible clothing counts as clicking the avatar. - **Animesh:** the object, on left-click too. This is a deliberate change from LL, where `AnimatedObjectsAllowLeftClick` defaults to 0. - **System body:** the avatar. - LL's selectable, click-action-ignore and hidden-self filters still apply. - Face select, touch and media UVs use the GPU hit (face, texcoord, normal, tangent). **Preference:** Preferences > Advanced > "Accurate rigged mesh picking" (`RenderVKPickRigged`). - 0 = off. - **1 = with ray tracing (default).** Costs about nothing: one ray per frame. - 2 = also without RT. The fallback reads about 100 bytes per triangle of the rigged items along the cursor ray, every frame. My estimate is about 0.1 to 0.3 ms per avatar on a mid-range GPU, and several ms in crowds on slow or integrated GPUs, so it is opt-in. **Caveat:** mode 1 needs something that *uses* ray tracing to be on (ray-traced sun, ray-traced projector shadows, ray-traced probes, or an RT debug view), not just RT hardware. With none of those on, the TLAS isn't built and rigged mesh isn't pickable at the default setting. That's no worse than before, when picking used the bind pose. Mode 2 covers that case.

sky_line referenced this issue from a commit

2026-10-03 01:50:57 +00:00

#255: GPU cursor pick shaders for rigged surfaces (ray query + skinned fallback)

sky_line referenced this issue from a commit

2026-10-03 01:50:57 +00:00

#255: LLCleanPickVK -- per-frame rigged cursor pick in recordWorld + sync RT pick

sky_line referenced this issue from a commit

2026-10-03 01:50:57 +00:00

#255: pick worn rigged mesh / animesh against what is drawn (GPU pick resolve)

sky_line referenced this issue from a commit

2026-10-03 01:50:57 +00:00

#255: skinned pick path: skip while the draw sets don't exist yet

sky_line referenced this issue from a commit

2026-10-03 05:26:11 +00:00

#255 follow-up: pick request/result carry a "filtered" candidate flag

sky_line referenced this issue from a commit

2026-10-03 05:26:11 +00:00

Avatar clickies: rigid clickies win through mesh bodies; touchable rigged items get hover + touch

sky_line referenced this issue

2026-10-04 01:01:10 +00:00

Picking: shoot through transparent texels (alpha mask/blend) instead of selecting the see-through face #276

sky_line commented

2026-10-04 02:45:46 +00:00

Author

Owner

Status check (verification sweep): the GPU cursor pick shipped in v1.0.0-beta.1/beta.2 (81dacb7cea, 37d4fd3b62, 4005fa4ae8, ab198588c8, 5b6784ad6f, 42e0d982c0).

Still open: with the default settings, worn rigged mesh is usually still not pickable. RenderVKPickRigged defaults to 1, which needs a TLAS, and the TLAS is only built when some RT feature is in use. RenderVKSunShadows and RenderVKProjectorShadows both default to 1 (shadow maps), so there's no TLAS, and the setting's own comment says it then tests "usually nothing". To meet the acceptance by default, mode 1 needs to build or keep a pick TLAS, or auto-fall back to the skinned compute path (mode 2) when no TLAS exists.

Status check (verification sweep): the GPU cursor pick shipped in v1.0.0-beta.1/beta.2 (81dacb7cea, 37d4fd3b62, 4005fa4ae8, ab198588c8, 5b6784ad6f, 42e0d982c0). Still open: with the default settings, worn rigged mesh is usually still not pickable. RenderVKPickRigged defaults to 1, which needs a TLAS, and the TLAS is only built when some RT feature is in use. RenderVKSunShadows and RenderVKProjectorShadows both default to 1 (shadow maps), so there's no TLAS, and the setting's own comment says it then tests "usually nothing". To meet the acceptance by default, mode 1 needs to build or keep a pick TLAS, or auto-fall back to the skinned compute path (mode 2) when no TLAS exists.

sky_line added this to the Slipstream project

2026-10-06 15:04:14 +00:00

sky_line added the

priority

high

label

2026-10-09 04:40:37 +00:00

sky_line referenced this issue

2026-10-12 09:25:12 +00:00

Hover pick costs ~3.5 ms of main-thread time per frame: move it off the CPU raycast #702

sky_line referenced this issue from a commit

2026-10-12 12:33:14 +00:00