[CrimsonMac]

What

Fixes #6666. On AMD RDNA 2 under Windows, a compute program with 64-bit LDS atomics hangs once its workgroup has more than one wave: agc_driver_lds_mixed_width_atomics runs for about 4 s and writes nothing (a TDR on an RX 6400, #5271).

The driver's ISA (VK_KHR_pipeline_executable_properties, Ryzen 5 7600 iGPU, 0x164e, driver 32.0.21042.62) shows why. LockedLdsUpdate releases the workgroup lock with OpAtomicStore (Release | WorkgroupMemory, Workgroup scope), and the driver compiles that release to s_waitcnt lgkmcnt(0), s_barrier, ds_write_b32. The 32-bit atomics inside the lock add a second s_barrier through their OpMemoryBarrier (AcquireRelease, Workgroup). The wave holding the lock waits at the barrier, and the other waves never reach it because they loop on the lock. This matches @veosett's capture on an RX 6900 XT in #6670.

ds_cmpst_rtn_b32 v1, v1, v1, v2 offset:16   ; acquire
s_waitcnt lgkmcnt(0)
buffer_gl0_inv
...
ds_read_b64 v[1:2], v3
v_add_co_u32 ...
ds_write_b64 v3, v[1:2]
s_waitcnt lgkmcnt(0)
s_barrier                                   ; emitted for the release
ds_write_b32 v3, v3 offset:16               ; lock = 0

On the same driver, a release on a read-modify-write atomic does not produce a barrier. The lock is now:

• Released with OpAtomicExchange with the same scope and semantics (Release | LDS memory, Workgroup). The driver compiles it to s_waitcnt lgkmcnt(0) + ds_write_b32, which still waits for every earlier LDS access before freeing the lock.
• Free of fences while it is held. The workgroup LDS OpMemoryBarrier after a 32-bit LDS atomic (Atomic32, SharedAtomicUpdate) and at the end of AtomicUpdateTyped's compare-exchange loop is not emitted between acquire and release. In a program that takes the lock, every LDS atomic runs under it, and the lock's acquire and release use the same scope and storage classes as those fences, so the next holder's acquire already sees everything the previous holder did. Fences outside the lock, and programs without 64-bit LDS atomics, are unchanged.

Release forms tried on the RDNA 2 iGPU (agc_driver_lds_mixed_width_atomics):

Release	Fences inside the lock	Result
OpAtomicStore Release (main)	kept / dropped	hangs / hangs
OpMemoryBarrier Release + relaxed OpAtomicStore	kept / dropped	hangs / hangs
OpAtomicExchange Release	kept	hangs
OpAtomicCompareExchange Release	kept	hangs
OpAtomicExchange Release (this PR)	dropped	passes
OpAtomicCompareExchange Release	dropped	passes

agc_spirv_lds_atomic_lock now checks that the lock is released by an exchange with Release semantics and that no OpMemoryBarrier sits between acquire and release, for ds_add_u64, ds_add_u32, ds_inc_u32 and a flat atomic into the LDS aperture. Against main's emitter it fails ("a ds_add_u64 is not serialised by one elected lane taking the workgroup lock"); with only the Atomic32 fence restored it fails with "ds_add_u32 emits a memory barrier while it holds the workgroup lock".

No title is known to hit this. The shader census in #4061 found no 64-bit LDS atomics, so this fixes the hang on RDNA 2 and GCN rather than a game.

#6670 changes the lock's initial store and adds a check to the same test file; the two changes are independent, and whichever merges second needs a small rebase of SpirvLdsAtomicLock.cpp.

Tested

Windows 11 (26100), Ryzen 5 7600, WinLibs GCC 15.2.0 r7, Release, main bf2053816 plus this commit.

	AMD Radeon iGPU (RDNA 2, 0x164e, 32.0.21042.62)	NVIDIA RTX 4070 SUPER (32.0.16.1742)
agc_driver_lds_mixed_width_atomics, main	fails, 4.3 s, lane 0 read 0xdeadbeefdeadbeef	passes
same, this PR, --repeat until-fail:20	20/20 pass	passes
agc_spirv_lds_atomic_lock	passes	passes
full ctest -j 8 --timeout 300	1375/1376, 17 skipped; fails agc_driver_layered_volume_vertex_no_gpl, as on main (#5804)	1374/1376, 14 skipped; fails agc_driver_f16_fma_denormal_modes and agc_driver_fma_mix_f16_denormal_modes (below)

• The two NVIDIA failures are the NaN bits of inf * 0 (0x7fff for 0xfe00, 0x7fffffff for 0xffc00000); the shaders use no LDS, and the f16 one fails the same way on unchanged main (reported on fix(shader): flush v_fma_f16 family denormals by the f16 denormal field #7034).
• guest_shader_link no longer hangs at exit after fix(tests): end Windows test processes through libc before DLL detach #6986; the earlier Driver::stop() hangs (Windows: agc_shader_helpers intermittently hangs at exit in Driver::stop() under ctest #6791) are gone from both full runs.
• Not run on lavapipe or Linux.
• python tools/check_conventions.py --base upstream/main and git diff --check pass.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: none
• GPUs tested: AMD Radeon(TM) Graphics (RDNA 2 iGPU, 0x164e), driver 32.0.21042.62, Windows 11; NVIDIA GeForce RTX 4070 SUPER, driver 32.0.16.1742, Windows 11
• AI-assisted: yes

[github-actions]

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 5021	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp
PR 6177	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
PR 6612	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
PR 6683	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
PR 6947	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
PR 6977	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
PR 7201	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
PR 7332	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp
PR 7392	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
PR 7484	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
PR 7642	core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp

[thuanquang]

RDNA 2 discrete run, since the PR lists only the iGPU. Windows 11 26200, AMD Radeon RX 6700 XT (gfx1031), driver 32.0.21045.5002, WinLibs GCC 15.2.0 r7, Release, ANYPS5_REQUIRE_VULKAN=1.

• main 20a84154f: agc_driver_lds_mixed_width_atomics fails after 2.97 s: lane 0 read 0xdeadbeefdeadbeef, expected 0x00001ffffffff000 after 4096 ds_add_u64 and 4096 ds_add_u32 on one slot (it fails in every full run on this PC).
• main + this PR (merges cleanly): agc_driver_lds_mixed_width_atomics and agc_spirv_lds_atomic_lock pass with --repeat until-fail:20, 0.26 s per run.
• Full ctest --parallel 4 --timeout 120 with this PR: 1379 tests; the LDS test passes; agc_driver_layered_volume_vertex_no_gpl fails as on main (AMD Windows: alpha-to-coverage still fails at the original executable path after #6436 #4768); four Python-driven tests time out, which happens on this PC only (slow Python startup).

AI-assisted: yes (Claude Code).

[CrimsonMac]

RDNA 2 discrete run, since the PR lists only the iGPU. Windows 11 26200, AMD Radeon RX 6700 XT (gfx1031), driver 32.0.21045.5002, WinLibs GCC 15.2.0 r7, Release, ANYPS5_REQUIRE_VULKAN=1.

* `main` `20a84154f`: `agc_driver_lds_mixed_width_atomics` fails after 2.97 s: `lane 0 read 0xdeadbeefdeadbeef, expected 0x00001ffffffff000 after 4096 ds_add_u64 and 4096 ds_add_u32 on one slot` (it fails in every full run on this PC).

* `main` + this PR (merges cleanly): `agc_driver_lds_mixed_width_atomics` and `agc_spirv_lds_atomic_lock` pass with `--repeat until-fail:20`, 0.26 s per run.

* Full `ctest --parallel 4 --timeout 120` with this PR: 1379 tests; the LDS test passes; `agc_driver_layered_volume_vertex_no_gpl` fails as on `main` ([AMD Windows: identical AGC executables invert GPL pass/fail results after relocation #4768](https://github.com/boykopovar/AnyPS5/issues/4768)); four Python-driven tests time out, which happens on this PC only (slow Python startup).

AI-assisted: yes (Claude Code).

Thank you for the test! It covers the gap in my testing and is valuable information.

[MarcZX10]

A new report, #7344, gives a second AMD RDNA 2 result: agc_driver_lds_lock_budget returns exactly half the expected sum in its device-backed LDS case on unchanged main. Your branch already reports repeated passes for agc_driver_lds_mixed_width_atomics, so this may share the lock/fence issue, but the report identifies a distinct lock-budget case. Could you or the reporter run agc_driver_lds_lock_budget on this head? That would tell us whether #6976 covers it or whether #7344 needs a separate fix.

[CrimsonMac]

I ran it on this head. #6976 does not cover #7344.

Windows 11, AMD Radeon(TM) Graphics iGPU (RDNA 2, 1002:164e), WinLibs GCC 15.2.0 r7, Release, ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, APS5_NO_PIPELINE_JOURNAL=1. This PR merged cleanly into main 0bb238b4a, because the lock-budget case isn't on the PR's base.

• main 0bb238b4a: agc_driver_lds_lock_budget fails 3/3 in the device-backed case (workgroup=0; device=8193). Lane 0 reads 0x00000ffffffff800, expected 0x00001ffffffff000, the same value as AGC: AMD device-backed LDS lock-budget case returns half of expected sum #7344.
• main + this PR: agc_driver_lds_lock_budget still fails every run with the same value. agc_driver_lds_mixed_width_atomics and agc_spirv_lds_atomic_lock pass with --repeat until-fail:20.
• Control, same merged build on an RTX 4070 SUPER: agc_driver_lds_lock_budget passes 20/20, with both the limit and limit-4 cases verified.

So #7344 reproduces on a second RDNA 2 device and needs a separate fix. I'll keep #6976 scoped to #6666. No open PR claims #7344 yet (#7351 lists it as an outstanding failure), so I'll take it next, since it reproduces here and I already have the ISA dump setup from #6976.

AI-assisted: yes (Claude Code).

[tezzgod74]

Fixes it on the iGPU from #6666, which is on a different driver build from the one in the description (32.0.21045.5002 vs 32.0.21042.62). Windows 11 Pro 10.0.26300, MinGW-w64 GCC 15.2.0 (WinLibs r7), Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON. A is main 20a8415, B is a663dfc merged into it (clean merge):

	A: main	B: main + this PR
agc_driver_lds_mixed_width_atomics, Ryzen 9 7950X iGPU (RDNA 2, 0x164e), 5 runs	fails 5/5, 4.4-5.4 s, 0xdeadbeefdeadbeef	passes 5/5, 0.3 s
same, RX 9060 XT (RDNA 4, 32.0.31041.1004), 5 runs	passes 5/5	passes 5/5
all agc_* tests (652), iGPU	2 failed	1 failed
all agc_* tests (652), RX 9060 XT	6 failed	the same 6

No test fails in B that passes in A on either device. The failures common to both builds are agc_driver_layered_volume_vertex_no_gpl (both devices) and, on the RX 9060 XT, agc_driver_division_output_modifiers, agc_driver_division_result_modifiers, agc_driver_f64_conversions, agc_driver_f64_transcendental and agc_driver_quad_pixel_mask.

A question on the dropped fences, not a reported failure: the OpMemoryBarrier after a 32-bit LDS atomic also kept the lane's later LDS reads from moving above that atomic. With only the lock's acquire and release left, a later read may move into the critical section. For data the lock protects that changes nothing. Is it also fine for a guest pattern that publishes through an LDS atomic and reads the published data outside the lock, or does nothing depend on that?

AI-assisted: yes

[CrimsonMac]

Thanks for the run, especially on the 21045 driver build.

On the fences: no guest pattern depends on them, because the lock already gives the same ordering. requirements.ldsLock is decided per shader, so once it's set, every LDS atomic in that shader (32-bit and 64-bit) goes through LockedLdsUpdate; locked and unlocked atomics never mix on the same LDS. A later read can move above the release, but not above the acquire CAS. In the publish pattern, if the reader's atomic observes the publisher's, the reader's CAS read the lock value written by the publisher's release exchange (or by a later RMW on the lock). So everything the publisher wrote before its atomic happens-before the reader's acquire, and the reader's later reads, inside or outside the lock, see it. The dropped OpMemoryBarrier added only an acquire after the lock's acquire and a release before the lock's release. A reader that polls a flag with a plain ds_read had no acquire before this change either; that pattern relies on s_barrier, which still emits OpControlBarrier with AcqRel on LDS.

AI-assisted: yes (Claude Code).

[jotitadev]

Apple GPU run, as a non-AMD check that the exchange release and the dropped in-lock fences still give correct results. On Apple, as on other GPUs, every 64-bit LDS atomic goes through the workgroup lock: MoltenVK reports shaderSharedInt64Atomics = false.

Machine: Apple M2 Max, macOS 27.2, x86-64 build under Rosetta. Head a663dfc. Method: macos-port @ b7eb6b6 + ImportWatched + PR diff, not main. The diff applied cleanly.

• Test only, emitter unchanged: agc_spirv_lds_atomic_lock fails with a ds_add_u64 is not serialised by one elected lane taking the workgroup lock. That is the message in the description, so the new check runs here.
• Full PR, all 13 lds tests pass (ctest -R lds, lds_atomics64 excluded) on MoltenVK 1.4.2. That includes agc_spirv_lds_atomic_lock, agc_driver_lds_mixed_width_atomics, agc_driver_lds_integer_atomics, agc_driver_lds_atomics and both pixel LDS tests.
• --repeat until-fail:20 on agc_driver_lds_mixed_width_atomics, agc_driver_lds_integer_atomics and agc_driver_lds_atomics: 20/20 on MoltenVK 1.4.2 and 20/20 on MoltenVK main + SPIRV-Cross #2704. No hang and no wrong sum. The mixed-width test uses a 256-thread workgroup, which is 8 subgroups of 32 on this GPU, so the multi-wave case behind #6666 runs here.
• Full ^agc suite with the PR (SDK driver, -j1 --timeout 30, flip/display/lds_atomics64 excluded): 13 failures out of 433, the same 13 as the port baseline. No new failures.

On this port baseline, agc_driver_lds_mixed_width_atomics already passes without the PR, so Apple does not show the RDNA 2 hang. What this run shows is that the new lock form does not break Apple.

GPUs tested: Apple M2 Max (MoltenVK 1.4.2 and MoltenVK main + SPIRV-Cross #2704)

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

[Amoenus]

@CrimsonMac Independent Windows validation after #7368 merged, on a different AMD iGPU (Ryzen 9 9950X3D, AMD Radeon(TM) Graphics 1002:13c0, driver 32.0.21036.18), plus NVIDIA RTX 5080 (10de:2c02, driver 610.62), Windows 11 Enterprise 26200, WinLibs GCC 15.2.0 r7 Release.

• Unmodified main 385ed71f2cfa8d67ff466d050d5e88e136d3ec28: agc_driver_lds_mixed_width_atomics and agc_driver_lds_lock_budget each fail once on AMD. The original device-backed limit readback passes; the remaining limit-4 case reads 0xdeadbeefdeadbeef, expected 0x00001ffffffff000, matching the remaining AGC: 64-bit LDS atomics under the workgroup lock never finish in workgroups over 64 threads on AMD RDNA 2 (Windows) #6666 coverage documented in fix(shader): use 64-bit buffer atomics for device-memory LDS #7368.
• Our preserved integration before this PR (bbff86ec6643e2420143ea5e2b04a7d509674a74, based on that main) reproduces both failures. Merging your exact head a663dfc9ab4a302eed28059674293dc46e152fc7, without adapting its code, produces tested integration 3b6498d74e. Only your four-file change separates those integration pins.
• With your change, agc_driver_lds_mixed_width_atomics, agc_driver_lds_lock_budget (both limit and limit-4), and agc_driver_lds_beyond_device_limit each pass 3/3 on both actual GPUs. The unrelated pipeline-header regression also passes 3/3 per GPU: 12 executions per GPU in this focused matrix. Device identity was checked per execution, and the Khronos validation layer actually loaded; zero VUIDs, validation errors or validation warnings.
• agc_spirv_lds_atomic_lock, agc_lds_lock_budget, and CPU graphics validation each pass 3/3 (9 CPU executions). Matching full Windows relinker/library build and conventions check pass. Each selected test finishes within 30 seconds.

This supports the combination of #7368 and #6976 for the remaining AMD cases. The misaligned-address fault printed by the lock-budget fixture is its expected negative control. No Linux result, title unblock, or performance improvement is claimed by this run. Your original authorship and commit history are retained in the fork integration.

AI-assisted: yes (Codex).

[MarcZX10]

Thanks for the expanded matrix. This adds an independent RDNA 2 discrete GPU, a second AMD iGPU, and an NVIDIA control, and it specifically checks the combined #7368/#6976 path against the two AMD failures. The result supports that tested composition on those Windows drivers; it does not add a Linux or title-run claim.

[adriadam10]

Rebased onto current main (only conflict: SpirvLdsAtomicLock.cpp; kept both #6670's lane-zero init check and this PR's release/fence checks). Checked the exchange release and the fence removal against the lock's acquire/release semantics, the updated agc_spirv_lds_atomic_lock, and the AMD/NVIDIA/Apple runs in the thread; AGC/AgcDriver ctest 827/827 after the rebase; merged.