Skip to content

fix(agc): run wave32 compute programs on 32-wide subgroups - #316

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
adriadam10:fix/wave32-subgroup-size
Oct 4, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
adriadam10:fix/wave32-subgroup-size

Conversation

@adriadam10

Copy link
Copy Markdown
Collaborator

What

On devices whose default subgroup is 64 lanes (RADV; AMD on Windows reports 64 too), a wave32 compute program was compiled for a 64-wide host subgroup. With more than 32 threads per workgroup, two guest waves then share one host subgroup, so everything wave-wide in the second wave sees the first one: v_mbcnt lane ids, ballots/VCC, EXEC saved or changed by v_cmpx, v_readlane. On RADV, a 64-thread wave32 dispatch gives v_mbcnt_lo = 32..63 in the second wave instead of 0..31.

Fix, with VK_EXT_subgroup_size_control:

  • VulkanDevice enables subgroupSizeControl when the default subgroup is wider than 32, 32 lies in [minSubgroupSize, maxSubgroupSize] and compute is in requiredSubgroupSizeStages.
  • VulkanDevice::ComputeTarget(waveSize) returns Target() with subgroupSize = 32 for wave32 programs on such devices. Dispatch.cpp uses it, so the program is recompiled for a 32-wide host subgroup, the path NVIDIA already takes.
  • RecompileResult::hostSubgroupSize records the width the program was laid out for. It is stored in the disk cache and fills existing padding, so the struct size is unchanged. The compute pipeline adds VkPipelineShaderStageRequiredSubgroupSizeCreateInfoEXT{32} when it is 32 and the device's default is not.

Wave64 programs and devices whose default is already 32 (NVIDIA) are unchanged. The cache key already includes target.subgroupSize.

Not in this PR: graphics stages. RADV can require 32 for fragment/mesh/task, but not vertex. That needs per-stage widths in Pipeline.cpp and its own tests, so it is a follow-up.

Test agc_driver_wave32_subgroup: one 64-thread wave32 dispatch, hand-assembled (words from llvm-mc -mcpu=gfx1013). Each lane checks v_mbcnt_lo (lane in its wave), s_bcnt1 of a VCC compare (8 per wave), v_readlane_b32 lane 3 (its own wave's lane 3), and the EXEC save → v_cmpx → restore pattern. It is skipped when the device cannot run wave32 on 32-wide subgroups (lavapipe: 8).

Tested

Linux, full ctest (99) passes.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: ci: run the shader execution tests on lavapipe #251 (test harness); review only the last commit
  • AI-assisted: yes

adriadam10 and others added 2 commits October 1, 2026 22:45
MixPrecision and SdwaDestination dispatch hand-assembled RDNA through the
recompiler and compare every lane with a CPU model, but were never in ctest.
Move them to tests/execution/, which is globbed into one test per file with
the same target names, so new execution tests add a file and no CMake lines.
They exit 77 (skipped) when the host has no Vulkan device; the Linux runner
installs lavapipe and sets ANYPS5_REQUIRE_VULKAN=1 so they cannot skip there.

Co-Authored-By: Claude <noreply@anthropic.com>
On devices whose default subgroup is 64 lanes (RADV, AMD on Windows) a
wave32 compute program was compiled for a 64-wide host subgroup, so the two
guest waves of a 64-thread workgroup shared one subgroup: ballots, EXEC,
v_mbcnt and lane reads of the second wave saw the first one's lanes.

With VK_EXT_subgroup_size_control, when compute can require 32 lanes, the
dispatch recompiles wave32 programs for a 32-wide host subgroup
(VulkanDevice::ComputeTarget) and the pipeline requires that size. The
result records the width it was compiled for (hostSubgroupSize, also in the
disk cache).

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added build shader core/shader agc libSceAgc* ci .github labels Oct 2, 2026
This was referenced Oct 2, 2026
@oneandonlydean
oneandonlydean merged commit a75b2c9 into boykopovar:main Oct 4, 2026
3 checks passed
adriadam10 added a commit to adriadam10/AnyPS5 that referenced this pull request Oct 4, 2026
Since boykopovar#316 a wave32 dispatch recompiles for a 32-wide subgroup target,
so preparing for the device target built a source the dispatch never
looked up. PrepareSource is added to the shader cache's reviewed names.

Co-Authored-By: Claude <noreply@anthropic.com>
@adriadam10
adriadam10 deleted the fix/wave32-subgroup-size branch October 5, 2026 06:47
adriadam10 added a commit to adriadam10/AnyPS5 that referenced this pull request Oct 5, 2026
Since boykopovar#316 a wave32 dispatch recompiles for a 32-wide subgroup target,
so preparing for the device target built a source the dispatch never
looked up. PrepareSource is added to the shader cache's reviewed names.

Co-Authored-By: Claude <noreply@anthropic.com>
SP4C3B4R-8 pushed a commit to SP4C3B4R-8/AnyPS5 that referenced this pull request Oct 6, 2026
Takes the 13 gate-batch PRs that merged upstream (boykopovar#319 boykopovar#329 boykopovar#330 boykopovar#331 boykopovar#325
boykopovar#435 boykopovar#471 boykopovar#294 boykopovar#376 boykopovar#404 boykopovar#394 boykopovar#401 boykopovar#446) in place of astrobot's local
versions, as integ/gate-batch cab0eab had them: boykopovar#294 and boykopovar#376 in
astrobot's depth model (DepthTarget, state.depth), boykopovar#394 with the thick mip
tails, boykopovar#401 over astrobot's slab pool, boykopovar#471 keeping the settled single-lane
path. Also takes the rest of main since a818bee (boykopovar#532 boykopovar#340 boykopovar#486 boykopovar#484 boykopovar#316
boykopovar#243 boykopovar#334 boykopovar#434 boykopovar#437 boykopovar#470 boykopovar#373 boykopovar#452 boykopovar#455 and our upstreamed PRs, where
astrobot keeps its later revisions and env switches).

Resolution notes: relinker converter/matcher/tests are upstream's (boykopovar#432
decoder covers VEX RIP-relative followers); RecompileResult gains boykopovar#316's
hostSubgroupSize (set from request.target.subgroupSize); shader disk cache
format 9; boykopovar#452's scissor message; upstream's later live-stack and
memory-copy decode tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mgp3oZvembQziVNE2xARQK
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build ci .github shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants