Repository navigation
fix(recompiler): write a coherent dwordx2 as one 64-bit atomic store - #7603
Conversation
A glc or dlc buffer_store_dwordx2 is one 64-bit write on the GPU, so a
workgroup that polls a {flag, value} pair never sees the flag without
its value. It was emitted as two dword stores, and the coherent dwordx2
load was one 64-bit read only in programs with 64-bit buffer atomics,
the only ones that had the u64 storage-buffer view.
GTA V builds each BLAS with a single-pass decoupled look-back scan that
publishes its tiles' pairs with buffer_store_dwordx2 and has no 64-bit
atomics. A tile read an earlier tile's new flag with its old value 0,
the scan came out short, the BVH got overlapping nodes and the refit
kernel looped until the device was lost.
A program with a coherent raw two-dword buffer load or store now has
the u64 view whenever the device has shaderBufferInt64Atomics. Such a
store whose dwords are in bounds, adjacent and 8-byte aligned is one
device-scope OpAtomicStore. The load takes the same condition, which
now also checks the second dword's bounds and that the dwords are
adjacent. All other accesses keep the per-dword path.
LookBackScan also runs its kernel with the two buffer_atomic_swap_x2
replaced by glc buffer_store_dwordx2. Without this change that variant
fails its first whole scan.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
AMD run, since the description lists AMD as not tested. Radeon 860M (RDNA 3.5, driver 32.0.13058.8001), Windows 11 26300, WinLibs GCC 15.2.0 r7, Release,
AI-assisted |
|
Verified at head |
|
RDNA 4 data: Windows 11 Pro 26300, WinLibs GCC 15.2.0 r7, Release, every execution test built,
I also ran full CTest with this PR and the 14 other open AGC PRs I tested in the same batch merged together (#7647, #7641, #7630, #7627, #7600, #7591, #7579, #7535, #7533, #7474, #7462, #7050, #7048, #7650). Neither GPU showed a failure beyond AI-assisted: yes (Claude Code) |
|
RDNA 2 data, the PS5's own architecture, which this PR hasn't had yet: RX 6700 XT (gfx1031), AMD driver 32.0.21045.5002, Windows 11 26200, WinLibs GCC 15.2.0 r7, Release. The test.
The hardware claim ("a glc or dlc
So on RDNA 2 the 64-bit store and the 64-bit load are each single accesses, and the two controls show the check catches a split write or a split read when one exists. That supports emitting the pair as one 64-bit store and reading it as one, as this PR does. Desktop RDNA 2, not a PS5. |
|
First AMD-on-Linux (RADV) run of this PR. Merged onto AMD Radeon RX 7900 XT (RADV NAVI31, gfx1100), Mesa 25.2.8, Linux Mint 22.3 (Ubuntu 24.04 base), kernel 7.0.0-38, GCC 13.3, Release. Full suite ( GPUs tested: AMD Radeon RX 7900 XT (RADV NAVI31, gfx1100), Mesa 25.2.8, Linux Mint 22.3 / kernel 7.0.0-38 AI-assisted: yes |
|
Checked the diff: the coherent |
What
A glc or dlc
buffer_store_dwordx2is one 64-bit write on the GPU. It was emitted as two dword stores, flag word first. A workgroup that polls the pair can then read the new flag with the old value. #919 made the coherentbuffer_load_dwordx2one 64-bit read, but only in programs that already had the u64 storage-buffer view, which means programs with 64-bit buffer atomics. The store side was never covered.GTA V (PPSA04264) builds each BLAS with a single-pass decoupled look-back scan. Its tiles publish
{flag, value}with a glcbuffer_store_dwordx2and poll the earlier tiles with glcbuffer_load_dwordx2. The program has no 64-bit atomics, so both its stores and its loads were split. I dumped one build (45 tiles):Changes:
LoadBufferU32x2orStoreBufferU32x2now gets the u64 view (buffers_u64,Int64,Int64Atomics) when the device hasshaderBufferInt64Atomics. Before, only programs with 64-bit buffer atomics got it. On a device without that feature nothing changes.OpAtomicStore. Otherwise it keeps the per-dword path.coherentPair. Before, it only checked the first dword with an 8-byte access, which went wrong in two cases:Tested
Windows 11 Pro 26300, NVIDIA GeForce RTX 5090 (617.42, 32-lane subgroups), MinGW-w64 Release build of this branch on
main91ddd9e. After the rebase onto 81cb863, whose two new commits touch neither the recompiler nor the AGC driver,agc_driver_look_back_scanpasses again.agc_driver_look_back_scan_testsnow also runs its scan kernel with the twobuffer_atomic_swap_x2replaced by glcbuffer_store_dwordx2(0xe0745000). Without the recompiler change the new variant fails its first whole scan:buffer_store_dwordx2 whole scan 0: block 116's prefix is 57920, expected 58669. With it both variants pass, in 6 of 6 runs.ctest -j 24: 1488 of 1489 pass. The one failure,agc_driver_pixel_lds, is unrelated. It loses the device (Vulkan result -4) on this RTX 5090, and did so on plainmain55f82fe too.Linux and AMD not tested.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)