Explore: GPU (Vulkan compute) JPEG2000 decoder -- fetched J2C straight into the bindless texture store #634

New issue

Open

opened 2026-10-10 09:31:34 +00:00 by sky_line · 0 comments

sky_line commented

2026-10-10 09:31:34 +00:00

Owner

Exploration, gated on measurement (#217 step 3: OpenJPEG decode cost on first load).

Idea

Upload the fetched J2C bytes as they are, and let the GPU decode them directly into the texture's bindless slot (then through the existing GPU BC encoder). The CPU only queues work.

Pipeline

Tier-2 (packet headers): serial and branchy, but cheap. Parse on the CPU (microseconds per texture) into a code-block table (offsets, pass counts, lengths), or on the GPU with one thread or workgroup per codestream.
Tier-1 (EBCOT / MQ entropy decode): the expensive stage. Serial within each 64x64 code-block, parallel across blocks: a texture has tens to hundreds of blocks, a batch has thousands. One thread or subgroup per code-block, as in nvJPEG2000's approach.
Dequantize + inverse DWT (5/3 and 9/7) + inverse colour transform: plain compute.
Write into the slot, then run the existing GPU BC7/BC5/BC1 encode (texcomp). Mip levels come for free from the DWT resolution levels.
Discard levels / partial fetches: decode only up to the received resolution (truncated codestreams must work; see #389).

Expected wins

RAM: no CPU-side decoded pixels ever (cf. #159's 16.7 GB of CPU texture copies).
Upload bandwidth: J2C is about 10-20x smaller than RGBA through staging.
Login / first rez: decode leaves the CPU decode threads, which helps most when a busy region loads thousands of textures at once.
Fits the GPU-driven architecture: the GPU owns texture residency end to end.

Limits and risks

Cached reloads already skip decoding via the BC disk cache (#274), so this only helps the first fetch of each texture.
Correctness across ~20 years of SL content: varied encoders, layer counts, progression orders, code-block sizes, ROI and precinct use. Keep OpenJPEG as the CPU fallback for anything outside the supported subset.
Validation corpus: a large sample of real SL assets, compared against OpenJPEG (bit-exact for 5/3; tolerance for 9/7).
Engineering size: weeks, not days.

Gate

Do this only if measurement shows OpenJPEG decode dominates first-load rez or login time on busy regions.

Related: #217 (Kakadu), #389 (OpenJPEG update), #274, #159, #405, #586.

**Exploration**, gated on measurement (#217 step 3: OpenJPEG decode cost on first load). ## Idea Upload the fetched J2C bytes as they are, and let the GPU decode them directly into the texture's bindless slot (then through the existing GPU BC encoder). The CPU only queues work. ## Pipeline 1. **Tier-2 (packet headers):** serial and branchy, but cheap. Parse on the CPU (microseconds per texture) into a code-block table (offsets, pass counts, lengths), or on the GPU with one thread or workgroup per codestream. 2. **Tier-1 (EBCOT / MQ entropy decode):** the expensive stage. Serial within each 64x64 code-block, parallel across blocks: a texture has tens to hundreds of blocks, a batch has thousands. One thread or subgroup per code-block, as in nvJPEG2000's approach. 3. **Dequantize + inverse DWT (5/3 and 9/7) + inverse colour transform:** plain compute. 4. Write into the slot, then run the existing GPU BC7/BC5/BC1 encode (texcomp). Mip levels come for free from the DWT resolution levels. 5. **Discard levels / partial fetches:** decode only up to the received resolution (truncated codestreams must work; see #389). ## Expected wins - **RAM:** no CPU-side decoded pixels ever (cf. #159's 16.7 GB of CPU texture copies). - **Upload bandwidth:** J2C is about 10-20x smaller than RGBA through staging. - **Login / first rez:** decode leaves the CPU decode threads, which helps most when a busy region loads thousands of textures at once. - **Fits the GPU-driven architecture:** the GPU owns texture residency end to end. ## Limits and risks - **Cached reloads already skip decoding** via the BC disk cache (#274), so this only helps the first fetch of each texture. - **Correctness across ~20 years of SL content:** varied encoders, layer counts, progression orders, code-block sizes, ROI and precinct use. Keep OpenJPEG as the CPU fallback for anything outside the supported subset. - **Validation corpus:** a large sample of real SL assets, compared against OpenJPEG (bit-exact for 5/3; tolerance for 9/7). - **Engineering size:** weeks, not days. ## Gate Do this only if measurement shows OpenJPEG decode dominates first-load rez or login time on busy regions. Related: #217 (Kakadu), #389 (OpenJPEG update), #274, #159, #405, #586.

sky_line added the

feature

rendering

perf

labels

2026-10-10 09:31:34 +00:00

sky_line added this to the Slipstream project

2026-10-10 09:31:34 +00:00