Skip to content

fix(recompiler): write a coherent dwordx2 as one 64-bit atomic store - #7603

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
HQasper:fix/coherent-dwordx2-store
Oct 20, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
HQasper:fix/coherent-dwordx2-store

Conversation

@HQasper

@HQasper HQasper commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

What

A glc or dlc buffer_store_dwordx2 is one 64-bit write on the GPU. It was emitted as two dword stores, flag word first. A workgroup that polls the pair can then read the new flag with the old value. #919 made the coherent buffer_load_dwordx2 one 64-bit read, but only in programs that already had the u64 storage-buffer view, which means programs with 64-bit buffer atomics. The store side was never covered.

GTA V (PPSA04264) builds each BLAS with a single-pass decoupled look-back scan. Its tiles publish {flag, value} with a glc buffer_store_dwordx2 and poll the earlier tiles with glc buffer_load_dwordx2. The program has no 64-bit atomics, so both its stores and its loads were split. I dumped one build (45 tiles):

  • Three tiles started too low. Tile 4's look-back came to 94 instead of 202, which is tiles 0 and 2 with tiles 1 and 3 counted as 0.
  • The scan total came out at 1822 instead of 1986.
  • The BVH built from those offsets has overlapping nodes, and the per-frame refit kernel loops forever on it. This is the device lost (or frozen queue) a few seconds into Story mode.

Changes:

  • The u64 view in more programs. A program with a coherent raw LoadBufferU32x2 or StoreBufferU32x2 now gets the u64 view (buffers_u64, Int64, Int64Atomics) when the device has shaderBufferInt64Atomics. Before, only programs with 64-bit buffer atomics got it. On a device without that feature nothing changes.
  • The store. When both dwords are in bounds, adjacent and 8-byte aligned, the pair is now written as one device-scope OpAtomicStore. Otherwise it keeps the per-dword path.
  • The load. It takes the same condition through a shared coherentPair. Before, it only checked the first dword with an 8-byte access, which went wrong in two cases:
    • In a structured buffer, a pair whose second dword lies past the record (offset 0, stride 4) was still read in one piece.
    • In a swizzled buffer, whose two dwords are not adjacent, the pair was read from adjacent memory.

Tested

Windows 11 Pro 26300, NVIDIA GeForce RTX 5090 (617.42, 32-lane subgroups), MinGW-w64 Release build of this branch on main 91ddd9e. After the rebase onto 81cb863, whose two new commits touch neither the recompiler nor the AGC driver, agc_driver_look_back_scan passes again.

  • agc_driver_look_back_scan_tests now also runs its scan kernel with the two buffer_atomic_swap_x2 replaced by glc buffer_store_dwordx2 (0xe0745000). Without the recompiler change the new variant fails its first whole scan: buffer_store_dwordx2 whole scan 0: block 116's prefix is 57920, expected 58669. With it both variants pass, in 6 of 6 runs.
  • Full ctest -j 24: 1488 of 1489 pass. The one failure, agc_driver_pixel_lds, is unrelated. It loses the device (Vulkan result -4) on this RTX 5090, and did so on plain main 55f82fe too.
  • GTA V with this change on top of feat(recompiler): run waterfall loops over per-lane T#s through image tables #6488: Story mode loads and the prologue plays for 5 minutes. Before, every run lost the device or froze about 16 s into Story.
  • I also replayed the game's own scan kernel locally over the dumped input, with all 45 tiles in one dispatch (not part of this PR, since it uses the game's code). The scan was wrong in 382 of 500 runs before and in 0 of 500 runs after.

Linux and AMD not tested.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: PPSA04264 Grand Theft Auto V (PPSA04264) #1203
  • GPUs tested: NVIDIA GeForce RTX 5090, driver 617.42, Windows 11 Pro 26300
  • AI-assisted: yes

A glc or dlc buffer_store_dwordx2 is one 64-bit write on the GPU, so a
workgroup that polls a {flag, value} pair never sees the flag without
its value. It was emitted as two dword stores, and the coherent dwordx2
load was one 64-bit read only in programs with 64-bit buffer atomics,
the only ones that had the u64 storage-buffer view.

GTA V builds each BLAS with a single-pass decoupled look-back scan that
publishes its tiles' pairs with buffer_store_dwordx2 and has no 64-bit
atomics. A tile read an earlier tile's new flag with its old value 0,
the scan came out short, the BVH got overlapping nodes and the refit
kernel looped until the device was lost.

A program with a coherent raw two-dword buffer load or store now has
the u64 view whenever the device has shaderBufferInt64Atomics. Such a
store whose dwords are in bounds, adjacent and 8-byte aligned is one
device-scope OpAtomicStore. The load takes the same condition, which
now also checks the second dword's bounds and that the dwords are
adjacent. All other accesses keep the per-dword path.

LookBackScan also runs its kernel with the two buffer_atomic_swap_x2
replaced by glc buffer_store_dwordx2. Without this change that variant
fails its first whole scan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 6177 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvRuntimeBuffer.cpp
PR 6235 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 6412 core/shader/recompiler/SpirvBackend/src/SpirvRuntimeBuffer.cpp
PR 6488 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 6947 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 6977 core/shader/recompiler/SpirvBackend/src/SpirvRuntimeBuffer.cpp
PR 7201 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 7332 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp
PR 7392 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 7484 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvRuntimeBuffer.cpp
PR 7642 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 7922 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 8004 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp

@LeandroLP23

Copy link
Copy Markdown
Contributor

AMD run, since the description lists AMD as not tested. Radeon 860M (RDNA 3.5, driver 32.0.13058.8001), Windows 11 26300, WinLibs GCC 15.2.0 r7, Release, BUILD_TESTING=ON. Build: main c3591adf9 with #7623, #7612, #7611, #7603, #7579, #7563 and #7541 merged together (each merge clean, built once).

agc_driver_look_back_scan and agc_driver_pixel_lds pass 3 runs of 3.

AI-assisted

@akkadbakkad89

Copy link
Copy Markdown
Contributor

Verified at head e5b4bd72d: agc_driver_look_back_scan passes on NVIDIA RTX 3070 (driver 617.42) and Intel UHD 630 (ANYPS5_GPU=UHD, igvk64) - second NVIDIA datapoint and first Intel. Windows 11, MinGW (WinLibs GCC 15.2.0), Release.

@savvaniss

Copy link
Copy Markdown
Contributor

RDNA 4 data: agc_driver_look_back_scan fails on main and passes with this PR on an AMD RX 9070 XT. The Intel UHD 770 skips it, since it has no shaderBufferInt64Atomics.

Windows 11 Pro 26300, WinLibs GCC 15.2.0 r7, Release, every execution test built, ANYPS5_NO_SHADER_CACHE=1, serial CTest. Intel UHD Graphics 770 (8086:a780), driver 32.0.101.7088. AMD Radeon RX 9070 XT (1002:7550), driver 32.0.31041.1004, with VkCache emptied first. This head e5b4bd72d merged into main f3059421f. "Without" means this PR's test files on main f3059421f, with none of its other changes.

Tests, ×3 UHD 770 RX 9070 XT
This head: agc_driver_look_back_scan skipped (no shaderBufferInt64Atomics) pass
Without skipped fails: buffer_store_dwordx2 whole scan 0: block 2's prefix is 0, expected 27

I also ran full CTest with this PR and the 14 other open AGC PRs I tested in the same batch merged together (#7647, #7641, #7630, #7627, #7600, #7591, #7579, #7535, #7533, #7474, #7462, #7050, #7048, #7650). Neither GPU showed a failure beyond main's own, except #7650's new test.

AI-assisted: yes (Claude Code)

@thuanquang

Copy link
Copy Markdown
Contributor

RDNA 2 data, the PS5's own architecture, which this PR hasn't had yet: RX 6700 XT (gfx1031), AMD driver 32.0.21045.5002, Windows 11 26200, WinLibs GCC 15.2.0 r7, Release. main 676269473; full builds in both columns.

The test. agc_driver_look_back_scan, 10 runs each:

  • main with only this PR's LookBackScan.cpp: fails 10 of 10, always in buffer_store_dwordx2 whole scan 0, e.g. block 6's prefix is 1605, expected 2399 and block 4's prefix is 794, expected 1589.
  • main + this PR (e5b4bd72d): passes 10 of 10. The 27 tests matching look_back|buffer_store|buffer_load|atomic|coherent|glc pass, except agc_driver_lds_mixed_width_atomics, which fails on main here too (fix(shader): release the LDS lock with an exchange and drop fences inside it #6976).

The hardware claim ("a glc or dlc buffer_store_dwordx2 is one 64-bit write"), measured natively through HIP. In one dispatch of 32 wave32 waves, 16 waves each store 20,000 pairs {i, i * 0x9e3779b9} with glc to 32 slots, and the other 16 waves read those slots 20,000 times with glc dlc and count pairs whose halves don't match. 10,240,000 reads per run, 3 runs per row:

writer / reader torn pairs per run
buffer_store_dwordx2 / buffer_load_dwordx2 0, 0, 0
global_store_dwordx2 / global_load_dwordx2 0, 0, 0
control: two global_store_dword / global_load_dwordx2 2,296,992; 3,584,736; 1,728,480
control: global_store_dwordx2 / two global_load_dword 76,256; 65,120; 30,176

So on RDNA 2 the 64-bit store and the 64-bit load are each single accesses, and the two controls show the check catches a split write or a split read when one exists. That supports emitting the pair as one 64-bit store and reading it as one, as this PR does. Desktop RDNA 2, not a PS5.

@DavidAmsellem

Copy link
Copy Markdown
Contributor

First AMD-on-Linux (RADV) run of this PR. Merged onto main 43a2abaf7 (clean merge), built in full and compared with the same main, on the same machine.

AMD Radeon RX 7900 XT (RADV NAVI31, gfx1100), Mesa 25.2.8, Linux Mint 22.3 (Ubuntu 24.04 base), kernel 7.0.0-38, GCC 13.3, Release.

Full suite (ANYPS5_REQUIRE_VULKAN=1 ctest -j8): 1636/1638 with this PR, against 1636/1638 on main. The failures are the same two on both, so this PR adds none: agc_driver_image_table_vouched_lookups (#7766, fixed by #7916) and agc_driver_guest_memory_lifetime (#7784). The PR adds no new ctest entry; every existing test it touches passed.

GPUs tested: AMD Radeon RX 7900 XT (RADV NAVI31, gfx1100), Mesa 25.2.8, Linux Mint 22.3 / kernel 7.0.0-38

AI-assisted: yes

@adriadam10
adriadam10 merged commit 7602dde into boykopovar:main Oct 20, 2026
10 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the diff: the coherent dwordx2 store is now one device-scope 64-bit OpAtomicStore when both dwords are in bounds, adjacent and 8-byte aligned, otherwise the per-dword path; load and store share coherentPair, which also fixes the structured and swizzled pair cases of the load. The new LookBackScan store variant fails without the change (reported on RDNA 2 by @thuanquang) and passes on the NVIDIA, AMD (RDNA 2/3.5/4, RADV) and Intel runs in the thread; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* shader core/shader unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants