Parcel audio from in-world speaker objects (positional, through room acoustics)

sky_line commented

2026-10-20 16:30:16 +00:00

Owner

Today the parcel music stream plays directly to the listener: it isn't positioned, and room acoustics (#683) don't apply to it.

Goal

Play the parcel audio stream through one or more in-world objects, the "speakers", as positioned sound sources, so it behaves like other in-world sound:

distance falloff;
occlusion through walls;
the room reverb from acoustics.

Open questions (feasibility)

Getting PCM out of the stream. Streaming audio currently goes through the media plugin path (LibVLC / LLStreamingAudio). Can the decoded audio be taken (for example with libvlc audio callbacks) and fed into an OpenAL streaming source with our EFX and acoustics, instead of the plugin's own audio output? What latency, buffering and resampling does that need? Does it work on Windows, Linux and macOS?
Designating speakers. Options:
- a local per-user choice (right-click > Play parcel audio here);
- a tag in the parcel description that only the owner can set, in the same style as the [slwx] weather tag;
- a tag in the description of objects owned by the parcel owner or group.
It has to work with nothing new on the server.
Multiple speakers. Several speakers playing one stream in sync.
Falling back. When no speaker is in range, or the feature is off, what happens: fall back to normal stereo, or go silent?
Other details. Volume controls, the stream's own stereo image, and cost.

Today the parcel music stream plays directly to the listener: it isn't positioned, and room acoustics (#683) don't apply to it. ## Goal Play the parcel audio stream through one or more in-world objects, the "speakers", as positioned sound sources, so it behaves like other in-world sound: - distance falloff; - occlusion through walls; - the room reverb from acoustics. ## Open questions (feasibility) - **Getting PCM out of the stream.** Streaming audio currently goes through the media plugin path (LibVLC / LLStreamingAudio). Can the decoded audio be taken (for example with libvlc audio callbacks) and fed into an OpenAL streaming source with our EFX and acoustics, instead of the plugin's own audio output? What latency, buffering and resampling does that need? Does it work on Windows, Linux and macOS? - **Designating speakers.** Options: - a local per-user choice (right-click > Play parcel audio here); - a tag in the parcel description that only the owner can set, in the same style as the `[slwx]` weather tag; - a tag in the description of objects owned by the parcel owner or group. It has to work with nothing new on the server. - **Multiple speakers.** Several speakers playing one stream in sync. - **Falling back.** When no speaker is in range, or the feature is off, what happens: fall back to normal stereo, or go silent? - **Other details.** Volume controls, the stream's own stereo image, and cost.

sky_line added the

feature

label

2026-10-20 16:30:16 +00:00

sky_line added this to the Slipstream project

2026-10-20 16:30:17 +00:00

sky_line referenced this issue

2026-10-20 16:38:43 +00:00

Explore: Steam Audio (ray-traced occlusion, reverb, pathing, HRTF) for in-world sounds #683

sky_line commented

2026-10-20 16:47:39 +00:00

Author

Owner

Feasibility study done. Design decisions:

Capture: the media plugin hands its decoded audio to the viewer instead of playing it itself. On Linux that is a GStreamer appsink; on Windows it is the libvlc audio callbacks. The audio travels over the plugin's existing shared memory, and the viewer plays it through OpenAL speaker sources that go through room acoustics.
Speakers: a speaker is marked by invisible hover text, [slspk grp=stage ch=M|L|R|S vol= r= ...], set by a script that can delete itself afterwards. The owner and group come from one properties request per tagged object.
Trust: a speaker counts if its owner is the parcel owner, it is deeded to the parcel group, or it is set to the parcel's group. Speakers are often group-set rather than deeded, so they don't get returned. The worst case is a group member placing an extra speaker, which doesn't affect existing ones.
Sync: sample-exact across speakers within one viewer. Across viewers, live streams stay as they are today (a few seconds apart); fixed clips with t0= sync to about ±1 s.
Order of work: plugin capture and speaker discovery come first. The OpenAL speaker sources and the acoustics changes come after #891, which is editing the same audio engine files.

Feasibility study done. Design decisions: - **Capture:** the media plugin hands its decoded audio to the viewer instead of playing it itself. On Linux that is a GStreamer appsink; on Windows it is the libvlc audio callbacks. The audio travels over the plugin's existing shared memory, and the viewer plays it through OpenAL speaker sources that go through room acoustics. - **Speakers:** a speaker is marked by invisible hover text, `[slspk grp=stage ch=M|L|R|S vol= r= ...]`, set by a script that can delete itself afterwards. The owner and group come from one properties request per tagged object. - **Trust:** a speaker counts if its owner is the parcel owner, it is deeded to the parcel group, **or it is set to the parcel's group**. Speakers are often group-set rather than deeded, so they don't get returned. The worst case is a group member placing an extra speaker, which doesn't affect existing ones. - **Sync:** sample-exact across speakers within one viewer. Across viewers, live streams stay as they are today (a few seconds apart); fixed clips with `t0=` sync to about ±1 s. - **Order of work:** plugin capture and speaker discovery come first. The OpenAL speaker sources and the acoustics changes come after #891, which is editing the same audio engine files.

sky_line referenced this issue

2026-10-20 21:08:07 +00:00

Investigate: speed-of-sound propagation delay for in-world sounds and speakers #896

sky_line referenced this issue

2026-10-20 21:29:33 +00:00

Acoustics: boundary bass gain (wall / edge / corner loading) for the listener and sources #897

sky_line referenced this issue

2026-10-20 22:11:28 +00:00

Parcel speakers: smooth sync during movement (no drift-then-snap) #898

sky_line referenced this issue from a commit

2026-10-20 22:27:39 +00:00

#892: parcel stream audio capture through a shared-memory ring (phase A, opt-in)