GPU and render regression tests: headless Vulkan, compute shaders against golden outputs #1246

Open
opened 2026-10-28 20:33:40 +00:00 by sky_line · 0 comments
Owner

Part of #1243.

Scope.

  • A test runner (indra/vktest already exists; check what it provides before building anything) that creates a Vulkan instance with no surface, picks lavapipe (Mesa software ICD) on CI and any real GPU locally, loads the embedded SPIR-V the viewer ships (indra/vkclean/shaders to embedded SPIR-V), dispatches compute shaders on fixed inputs and compares with golden outputs.
  • First targets: the denoiser (small synthetic noisy frame, compare against a CPU reference within tolerance), ReSTIR reservoir maths (CPU reference implementation of reservoir update / merge, fixed RNG seeds, assert equivalence including the weight-sum and M clamping), and the weather maps (rain / snow / wet maps from a known state).
  • Tolerances, not bit equality: golden files carry per-test epsilon (absolute and relative) because lavapipe, NVIDIA and AMD differ in fused multiply-add and denormal handling; goldens are generated on lavapipe and checked against real GPUs only on the opt-in label.
  • The golden-image capture mode (#1241): once it lands, a scripted scene capture (testctl core scenario or a canned offscreen frame) compared with perceptual tolerance (e.g. per-pixel max delta plus a mean-error budget) and a tool to regenerate goldens (scripts/testing/regen_goldens.py) with a diff report artifact.
  • Labels: gpu (needs lavapipe or a GPU, runs in CI nightly and on demand, never in the default build), gpu-real (needs a physical GPU: the garbage test box, 1080 / 3060 / 5090).
  • Run under the Vulkan validation layer in CI and fail on any validation error (matches make validate).

Acceptance criteria.

  • ctest -L gpu runs on a CI machine with only mesa-vulkan-drivers installed (set VK_ICD_FILENAMES to lavapipe; document the package names for the CI box).
  • At least 3 compute-shader tests (denoiser, ReSTIR reservoir, one weather map) pass on lavapipe and on the 1080 within their tolerances; each has a CPU reference that is the documented spec of the maths.
  • Changing a constant in the denoiser shader makes the test fail with a readable diff (max error, location).
  • Total gpu label time under 60 s on lavapipe; nothing from this ticket in the default build's 5 s budget.

Effort. About 1 week for the headless runner, three shader tests and lavapipe CI; +3 days for the golden-image comparison once #1241 exists. Risk: lavapipe feature coverage (subgroup ops, bindless / descriptor indexing, REQUIRE_FULL_SUBGROUPS shaders): any shader that needs a capability lavapipe lacks gets the gpu-real label instead.

Part of #1243. **Scope.** - A test runner (`indra/vktest` already exists; check what it provides before building anything) that creates a Vulkan instance with no surface, picks lavapipe (Mesa software ICD) on CI and any real GPU locally, loads the **embedded SPIR-V** the viewer ships (`indra/vkclean/shaders` to embedded SPIR-V), dispatches compute shaders on fixed inputs and compares with golden outputs. - First targets: the denoiser (small synthetic noisy frame, compare against a CPU reference within tolerance), ReSTIR reservoir maths (CPU reference implementation of reservoir update / merge, fixed RNG seeds, assert equivalence including the weight-sum and M clamping), and the weather maps (rain / snow / wet maps from a known state). - Tolerances, not bit equality: golden files carry per-test epsilon (absolute and relative) because lavapipe, NVIDIA and AMD differ in fused multiply-add and denormal handling; goldens are generated on lavapipe and checked against real GPUs only on the opt-in label. - The golden-image capture mode (#1241): once it lands, a scripted scene capture (testctl `core` scenario or a canned offscreen frame) compared with perceptual tolerance (e.g. per-pixel max delta plus a mean-error budget) and a tool to regenerate goldens (`scripts/testing/regen_goldens.py`) with a diff report artifact. - Labels: `gpu` (needs lavapipe or a GPU, runs in CI nightly and on demand, never in the default build), `gpu-real` (needs a physical GPU: the garbage test box, 1080 / 3060 / 5090). - Run under the Vulkan validation layer in CI and fail on any validation error (matches `make validate`). **Acceptance criteria.** - `ctest -L gpu` runs on a CI machine with only `mesa-vulkan-drivers` installed (set `VK_ICD_FILENAMES` to lavapipe; document the package names for the CI box). - At least 3 compute-shader tests (denoiser, ReSTIR reservoir, one weather map) pass on lavapipe and on the 1080 within their tolerances; each has a CPU reference that is the documented spec of the maths. - Changing a constant in the denoiser shader makes the test fail with a readable diff (max error, location). - Total `gpu` label time under 60 s on lavapipe; nothing from this ticket in the default build's 5 s budget. **Effort.** About 1 week for the headless runner, three shader tests and lavapipe CI; +3 days for the golden-image comparison once #1241 exists. Risk: lavapipe feature coverage (subgroup ops, bindless / descriptor indexing, `REQUIRE_FULL_SUBGROUPS` shaders): any shader that needs a capability lavapipe lacks gets the `gpu-real` label instead.
sky_line added this to the Slipstream project 2026-10-28 20:33:40 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
sky_line/Slipstream#1246
No description provided.