Skip to content

perf(shader): declare the read-only descriptor tables NonWritable - #6748

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
milescoviello:perf/read-only-descriptor-tables
Oct 19, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
milescoviello:perf/read-only-descriptor-tables

Conversation

@milescoviello

@milescoviello milescoviello commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What

The recompiler declares three tables as plain storage buffers: the flattened SRT (flattened_srt), the BDA page table (bda_pagetable) and the image table map (table_map). The host fills them and the shader only loads from them (SpirvMemoryInstructions.cpp, SpirvBdaFault.cpp, SpirvImageEmitter.cpp). Nothing marks them read-only, though. A program that also stores, to buffers or through a device address, gives the driver no way to know those stores don't alias the tables. So every descriptor read stays a separate load after each store.

Crimson Desert (PPSA25286) has compute programs where this adds up. Its largest one, 1,835,706 SPIR-V words, reads flattened_srt 12,362 times. Creating its pipeline on NVIDIA 595.99 takes 200 s and 10.6 GiB of memory, and in the title that compile is where my 16 GiB cap ended every run.

The three variables are now decorated NonWritable. The decoration goes on the variables, not on the block's member, because the block type is shared with the written buffers.

Tested

Linux (Ubuntu 24.04, GCC 13.3, Release), RTX 4070 Ti, NVIDIA 595.99.

  • agc_shader_memory: the recompiled test program's flattened_srt must carry NonWritable, and so must bda_pagetable and table_map where a program has them. With main's SpirvEmitterHelpers.cpp it fails with the flattened SRT is not decorated NonWritable. It passes 3 of 3 with this change.
  • ctest (^agc_|recompiler|shader_cache|request_serializer|spirv, 609 tests, SPIR-V Tools on, so every recompiled module is validated): 608 pass. agc_driver_store_over_untracked fails the same way on main 5d9dcc4 here; test(agc): keep waiting for the neighbour copy case's result #6385 addresses it.
  • Pipeline creation of Crimson Desert's captured modules (APS5_DUMP_COMPUTE_PIPELINES), as the title compiled them and with only the three decorations added, timed with agc_compute_pipeline_replay and the NVIDIA disk cache off (__GL_SHADER_DISK_CACHE=0). Peak is the process's maximum RSS:
SPIR-V words as compiled (time, peak RSS) with NonWritable
111,936 2.3 s, 0.31 GiB 1.2 s, 0.27 GiB
113,420 30.4 s, 1.84 GiB 29.0 s, 1.82 GiB
126,401 10.4 s, 0.66 GiB 8.9 s, 0.65 GiB
137,705 1.1 s, 0.26 GiB 1.0 s, 0.25 GiB
152,689 3.5 s, 0.38 GiB 2.5 s, 0.32 GiB
157,909 19.4 s, 0.96 GiB 19.5 s, 0.91 GiB
181,837 47.0 s, 2.30 GiB 44.6 s, 2.27 GiB
199,155 227.0 s, 8.47 GiB 239.2 s, 8.49 GiB
368,151 115.9 s, 4.62 GiB 116.7 s, 4.75 GiB
592,790 85.9 s, 4.31 GiB 84.7 s, 4.25 GiB
856,754 77.7 s, 3.36 GiB 76.9 s, 3.32 GiB
867,093 57.0 s, 2.39 GiB 59.4 s, 2.25 GiB
1,378,664 34.7 s, 1.93 GiB 32.8 s, 1.57 GiB
1,835,706 200.1 s, 10.58 GiB 90.6 s, 4.34 GiB

The gain is concentrated where a program reads the tables most. The largest program drops from 200 s and 10.6 GiB to 91 s and 4.3 GiB. Others gain a little (112k words: 2.3 s to 1.2 s) or stay within run-to-run variation, and none gets worse beyond that. One run each.

Build Result
without this PR 4 of 4 runs end at 517 to 591 s, while the 1,835,706-word pipeline is being created
with this PR that pipeline is created in 87 s, and the run goes on to 643 s. It ends while the next large one, 1,916,980 words, is being created

The title still needs more memory than my cap allows at that point, so it doesn't present a frame yet.

@jotitadev cross-checked it on Ubuntu 26.04 (GCC 15.2, Release, SPIR-V Tools off), with this head merged onto main 143f130:

  • On an RTX 3070 (NVIDIA 595.99.02), the ShaderMemory.cpp check alone fails agc_shader_memory with the flattened SRT is not decorated NonWritable, and with the fix agc_shader_memory|^agc_driver_bda passes 3 of 3. ^agc|shader|recompiler gives 614 of 615; the one failure is agc_driver_store_over_untracked, which fails on main too.
  • On lavapipe (Mesa 26.0.3), 80 shader, recompiler, image and BDA tests pass, the same as main, so a second driver compiles and runs the decorated modules.

Validated on Windows 11 with an AMD Radeon 860M (RDNA 3.5, driver 32.0.13058.8001) by @LeandroLP23 (comment), in one build of main 6efb726 with this head and 13 other pull requests: agc_driver_store_over_untracked and agc_shader_memory pass 3 runs of 3. 713 of 716 agc_* tests pass; the 3 that fail fail the same way on main there.

Not checked on Intel or on a console.

Checklist

🤖 Generated with Claude Code

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linux cross-check on a second driver (lavapipe) plus another NVIDIA card. Head b4d65c4fd, merged onto main 143f13055.

Machine: Ubuntu 26.04, i7-8700, GCC 15.2, Release, SPIRV-Tools off. RTX 3070, NVIDIA 595.99.02 (Vulkan 1.4.329), and lavapipe, Mesa 26.0.3 (LLVM 21.1.8, Vulkan 1.4.335). All runs through ctest.

  • Red/green: the ShaderMemory.cpp change applied on its own over main fails agc_shader_memory with the flattened SRT is not decorated NonWritable. With the fix, agc_shader_memory|^agc_driver_bda passes 3/3, three runs, on the 3070.
  • 3070, ^agc|shader|recompiler (display/flip/lds_atomics64 excluded, -j1): 614/615. The one failure is agc_driver_store_over_untracked (the result did not reach a block the CPU left alone). It fails 10/10 on plain main 143f13055 here too, so it is not from this PR.
  • lavapipe, 80 shader/recompiler/image/BDA tests (^agc_shader|^agc_spirv|^agc_driver_bda|^agc_driver_image|^agc_driver_graphics$|recompiler|shader_cache|...): 80/80 with the PR, same as main. lavapipe compiles and runs the decorated modules without errors. Some cases skip themselves on the CPU device, the same on both sides: subgroup 8 per-wave image-table cases, EXEC-masked cases, and the BDA multi-dword write execution case.

Not run: pipeline-creation timing or memory, which is the point of this PR. No AMD/Intel, no Windows, no SPIR-V validation (Tools off in this build).

GPUs tested: NVIDIA GeForce RTX 3070 (595.99.02), lavapipe (Mesa 26.0.3), Ubuntu 26.04

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@milescoviello

Copy link
Copy Markdown
Contributor Author

Thanks @jotitadev, the lavapipe run is a useful second driver for the decorations, and the red/green on the 3070 matches mine. I added both to the description.

AI-assisted: yes

The flattened SRT, the BDA page table and the image table map are filled
by the host and only loaded by the shader, but they were declared as
writable storage buffers. A program that also stores through device
addresses then gives the driver no way to know those loads are not
aliased by its stores, so every descriptor read stays a separate load.
Crimson Desert's largest compute program reads the flattened SRT 12,362
times; with the decoration NVIDIA 595.99 creates its pipeline in 90 s
with a 4.3 GiB peak, against 200 s and 10.6 GiB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LeandroLP23

Copy link
Copy Markdown
Contributor

AMD on Windows: the tests this PR names pass on an RDNA 3.5 GPU.

Radeon 860M (RDNA 3.5, driver 32.0.13058.8001), Windows 11 26300, WinLibs GCC 15.2.0 r7, Release. main 6efb7261b with this PR (7e061035b) merged, together with #7296, #7446, #7377, #7445, #6708, #6785, #6706, #6613, #7449, #7391, #7489, #7484, #7514. They were built once, so the result is for the combination; each merge was clean.

  • agc_driver_store_over_untracked, agc_shader_memory: pass, 3 runs of 3.
  • Every agc_* test on the combined build: 713 of 716 pass. The 3 that fail, agc_driver_host_import_lifetime, agc_driver_quad_pixel_mask and agc_driver_layered_volume_vertex_no_gpl, fail the same way on unchanged main here, so none comes from these PRs.
  • agc_compute_pipeline_replay is not a ctest name here; I did not replay captures.

AI-assisted

@milescoviello

Copy link
Copy Markdown
Contributor Author

Thanks @LeandroLP23 for the AMD run on Windows. I added it to the description.

AI-assisted: yes (Claude Code)

@github-actions github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 18, 2026
@adriadam10 adriadam10 self-assigned this Oct 19, 2026
@adriadam10
adriadam10 merged commit a1dfc73 into boykopovar:main Oct 19, 2026
9 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Re-checked against current main: flattened_srt, bda_pagetable and table_map are still only loaded from (no stores or atomics, including in the FMASK, LDS-lock and ballot changes merged since), the decoration sits on the variables rather than the shared block member, and the agc_shader_memory check fails without the change. NVIDIA, lavapipe and AMD/Windows cross-checks noted; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* perf numbers missing perf pull request without a before and after summary line perf Performance change shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants