[adriadam10]

What

Phase B of #2242: on devices with graphics pipeline libraries (GPL), vertex and pixel shaders get their pipeline libraries built when they are registered, not at their first draw. A draw whose shaders were prebuilt only links (about 0.01 ms). Devices without GPL keep today's path unchanged; every test passes in both modes.

What each commit does:

1. Buffers: the shader reads the buffer V# words (stride, format, swizzle, validity, misalignment) at run time instead of specializing on them.
2. Heaps and images: descriptor heaps keep every element in place, so an inactive one gets a null or placeholder descriptor instead of being compacted out. Every image gets its ResourceMetadata, so direct images read mode, swizzle and mip range at run time the way indirect ones already did.
3. Compare and exports: the emulated depth-compare state goes into the image metadata flags and the export mappings into runtime state dwords. Texel-offset folding stays a specialization constant: as a runtime branch, the offset sample stays reachable with an unnormalized sampler, which the validation layer rejects (VUID-vkCmdDispatch-None-08611).
4. Fragment descriptor set: with GPL the pixel stage's bindings live in descriptor set 1 and the layout uses INDEPENDENT_SETS, so each stage's library depends only on its own set layout. The layout no longer enters the vertex library's key, which should also remove the repeated vertex-library compile Raghav measured when only the pipeline layout changed (not measured on NVIDIA).
5. Set layouts from the shaders: DescribeSetLayouts builds the set layouts from the compiled shaders alone, which registration needs. APS5_VERIFY_SET_LAYOUTS=1 compares them with the draw's layouts on every pipeline build; the whole suite passes with it.
6. Prebuild at registration: ResolveGraphicsStagesAbi (vertex stages) and the registered or static-ABI preparation of pixel shaders hand a registration result to PrebuildStageLibrary, which builds that stage's library with a stub partner and a fixed default state.
7. Vertex selectors: the vertex attribute component selectors are runtime state; only the numeric class stays in the module key.
8. Shader module lifetime: the pipeline library store no longer keeps every library's VkShaderModule until the device is torn down. A module can be destroyed once the pipeline is built (vkDestroyShaderModule), and the optimized link does not need it. This also changes main's draw path. Its cost shows with large modules: about 1.1 MB per library on RADV with Astro's Playroom shaders (3.17 GB → 1.98 GB peak for 1200 headers).
9. Flattened SRT placeholder for registration results, which have no runtime SRT yet.
10. Oversized shaders stay specialized: a module above registrationWordLimit (200000 SPIR-V words) is re-emitted specialized and compiles at its draw as today. CompiledShaderArtifact::registrationModule records the form and is stored in the disk cache. Prebuilds run on their own one-thread pool (APS5_PREBUILD_THREADS), separate from the registration pool, as Raghav suggested.
11. Only vertex and pixel shaders take the registration form, since they're the only stages prebuilt. Compute and the other stages stay specialized as on main. In registration form, the agc_driver_dword_pattern_fill compute shader hung CI's lavapipe in the build with ANYPS5_ENABLE_SPIRV_TOOLS=ON (not RADV, not without SPIRV-Tools). I didn't find the cause, so compute stays out until someone does (TechnicalDebt has the reproduction).
12. The prebuild pool is closed only if it was created. Before, closing it at shutdown created it, which hung agc_driver_wait_shutdown and agc_driver_flip_compute on Windows CI, where no prebuild ever runs.
13. The format word stays specialized for buffers accessed with the descriptor's format (buffer_*_format). As a runtime value, every such access compiled the decoder of every format: agc_driver_vertex_fetch went from 0.59 s to 9.96 s on CI's lavapipe with a cold driver cache (2 cores, SPIRV-Tools on, 88 % of the time in LLVM), past the 5 s fence on CI. With the word specialized it takes 1.05 s. This was most likely also behind the compute "hang" in item 11, which looked like a very long JIT rather than a loop. Shaders that still specialize something (these buffers, or texel-offset folding) get no prebuild, since their prebuilt module would never match a real draw.
14. The formatted buffer helper reads the V# words as specialization constants when they're known: always in the specialized form, and for the format word in registration form when the buffer is accessed with the descriptor's format. The first commit had made the helper take them as parameters in both forms, and since it's DontInline the driver couldn't fold the format switch anymore. That grew a compute shader in PPSA36082's loading frame from 51k to 236k words (22 → 794 ms) and kept that boot crashing (@Raghav314, Compiling Vulkan shader code at registration, not at the first draw (VK_EXT_shader_object) #2242). FormattedBufferOnce pins it: one formatted load gives 26.5k words (117.5k before; main 26.5k), and the vertex registration form stays under 1.5× its specialized form.
15. The buffer base misalignment stays a 1-bit specialization in registration form, as in the specialized form. Read at run time, it kept the unaligned compare-exchange store loops in every module: 6 loops in agc_driver_pixel_lds's pixel shader, which hung an RTX 4090 for Raghav (Xid 109) 3 times in 4 while main passes. Both forms of that shader now have no loops or atomics, and FormattedBufferOnce checks that registration form has no more compare-exchanges than the specialized form. Not yet confirmed on the 4090.
16. Docs: ARCHITECTURE and TechnicalDebt.

Why the word limit

Astro's Playroom registers 907 pixel shaders of 200k to 900k SPIR-V words (ubershaders). With runtime selectors the driver can no longer prune them: the largest one compiles as a fragment library in 7.1 s and 366 MB instead of 0.9 s and 132 MB, and prebuilding all of them took the process past 10 GB.

A paired sample of 60 pixel shaders (same shaders, main vs this branch, RADV Radeon 660M, Mesa 26.2.4, cache off) gave two clear groups:

SPIR-V words	n	compile, registration form vs specialized	optimized link
< 100k	17	×1.9	about ×2.5
100k–141k	11	×2.2	about ×3
> 280k	31	×5.2 (3.8–5.7)	about ×20–25

No shader falls between 141k and 280k, so the limit sits in that gap. Re-measured after items 14-15: below the limit ×1.84 (<100k) and ×2.24 (100-200k) compile with modules only ×1.10-1.14 the words; above it, in registration form with the limit off, 20 shaders still compiled ×5.9 (p10 ×5.2, p90 ×7.4) slower and ×20 on the optimized link, so the limit stays. The cost is the runtime values the driver can't fold, not module size. Below it, the extra compile time is paid once, on the background thread, and taken off the draw.

Tested

Linux (CachyOS), RADV Radeon 660M (Mesa 26.2.4), GCC 16, Release, rebased on main 8c4a295 (head b1ee794):

	result
^agc_|^shader, GPL on (with APS5_VERIFY_SET_LAYOUTS=1)	433/433
same with APS5_NO_GPL=1	NO425/425
^agc_ under VK_LAYER_KHRONOS_validation, GPL on and off	430/430 both; no VUID only with GPL
^agc_ on CI's lavapipe (Mesa 25.2.8)	430/430
CI's configuration locally (SPIRV-Tools on, CI's lavapipe, no /dev/udmabuf, --timeout 120 --parallel 4, clean build)	439/439

New or extended tests:

• BufferDescriptorRuntime: every V# case gives byte-identical SPIR-V and the same pipeline variant in registration form.
• TypedHeaps: present and null images give the same module.
• ZeroStrideVertexFetch: two swizzled cases (W Z 1 0), in and out of range, checked by value; all 7 cases give identical SPIR-V in registration form.
• VertexFetch: a mixed pipeline (vertex in registration form, pixel forced specialized with registrationWordLimit = 1).
• PreparedGraphics: the vertex stage alone and the pixel stage alone each build exactly their library without linking, the pair then finds both, and a second preparation of the same stages finds them again.
• ShaderMemory: serializer v17 round trip and cache key.

The new assertions were checked to fail when their expected value is mutated.

Astro's Playroom, every Shader header registered and resolved through the real AgcDriverRegisterShader/Resolve*Abi path (offline harness, not committed):

	peak memory	libraries prebuilt
main	4.21 GB	0
this branch, prebuild off	5.09-5.10 GB (3 runs)	0
this branch	3.96-5.45 GB (3 runs)	1937 (vertex/pixel shaders in registration form; the rest compile at their draw as today)

Not measured here: the effect on a real boot. I can't run titles on this machine. @Raghav314, a PPSA36082 cold start on this branch would be the deciding number. Your phase B simulation in #2242 suggested this boot survives once libraries are built at registration. Please don't merge before that run.

Checklist

• Based on current main; no other open PR implements the same functions (shares files with fix(agc): render all attachment layers with dynamic rendering #3763, perf(shader): convert f16 operands with UnpackHalf2x16 where f16 denormals are kept #3988, feat(agc): capture and replay compute pipeline compilation #3717, fix(agc): store D16 depth with stencil as D24 or D32 where the device lacks it #3846, fix(agc): name the shader address and stage in preparation failures #3946, fix(agc): read a merged program's vertex tables past its hidden user words #3738, fix(shader): decode a GPU-selected formatted load from a format table #3656; none implements these functions)
• One topic per PR; follow-ups go in a new PR (one topic, phase B; the commits only make sense together, since each alone makes the first draw slower)
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• AI-assisted: yes (Claude). All numbers come from runs on my machine.

[Raghav314]

I tested this on PPSA36082 cold starts on my machine (RTX 4090, Linux) and posted the details in the discussion: #2242 (reply in thread)

In short:

• Graphics compiles leave the loading frame as intended (267 ms on main, 0 here).
• dbb39bd makes the formatted buffer helper take the V# words as parameters in both forms. Since the helper is DontInline, a compute shader in that frame goes from 51,382 to 236,457 SPIR-V words, and its pipeline create from 22 to 757 ms. The loading frame ends up slower than on main and the boot still crashes.
• With the helper reading them as specialization constants again outside registration modules (a ~10-line change, in the discussion), the cold boot survives 3/3 and the loading frame drops to about 780 ms (1041 on main).
• agc_driver_pixel_lds hung the GPU on this PR's head (Xid 109, device lost) on a freshly booted GPU, with and without that change. Main passed it there. Small numbers, details in the discussion.

LLM assistance used.

[adriadam10]

Changes since opening, after the first two CI rounds (the body is updated):

• Windows: agc_driver_wait_shutdown and agc_driver_flip_compute hung at exit, because closing the prebuild pool at shutdown created it, and with no Vulkan device on Windows CI it never existed before. The pool is now closed only if it exists.
• Linux, lavapipe: agc_driver_dword_pattern_fill and then agc_driver_vertex_fetch hit the 5 s fence. Cause: in registration form the buffer V# format word was read at run time, so every buffer_*_format access compiled every format's decoder. On CI's lavapipe with a cold driver cache that's 9.96 s instead of 0.59 s for vertex_fetch (2 cores, SPIRV-Tools on, 88 % in LLVM). The format word now stays specialized for those buffers (1.05 s), and shaders that still specialize something get no prebuild. Compute shaders stay specialized as on main, since only vertex and pixel shaders are prebuilt.
• History: commit 1 now carries the formatted-helper SPIR-V validity fix that had landed in commit 4, so every commit passes spirv-val.

Third round: full build green on Linux (926/926) and Windows, plus all PR checks. Astro's Playroom coverage: 1937 libraries prebuilt, 5.41 GB peak against 5.07 GB with prebuild off.

Still waiting on a PPSA36082 cold start before merging.

AI-assisted: yes (Claude).

[Raghav314]

Reran PPSA36082 cold starts on 14b52c1 on my machine (RTX 4090, Linux), details in the discussion: #2242 (reply in thread)

Short version: the formatted buffer helper still makes the loading frame's compute shader 236,457 words with a 794 ms pipeline create, and the stock boot still crashes loading GUI.pkg (3/3). With the helper change from my earlier comment on top, it boots 3/3 with frame 1 at about 790 ms, and the Linux test job passes 921/923 (same two failures as main). agc_driver_pixel_lds hung the GPU again on 14b52c1 as pushed (Xid 109); main passes it here.

LLM assistance used.

[Raghav314]

Same formatted buffer helper, registration form this time. On Demon's Souls (PPSA01341), dumped on my machine and compiled offline, the helper still switches over every format when the format is specialized at the call site, because it takes word 3 as a parameter. That makes the vertex shaders' registration form 3.3x their specialized form, with 39 of the 58 largest over registrationWordLimit. Reading word 3 as the specialization constant inside the helper in that case brings it to 1.1x and none over the limit. Details and the diff in the discussion: #2242 (reply in thread)

LLM assistance used.

[Amoenus]

Independent Windows / RTX 5080 validation of exact 14b52c1

I built the exact PR head 14b52c1ac38d2f9c254299189e0bccba26bed752 and its actual merge-base bca7a29d259299aaa8972c94d12694232bb3bb08 separately. No additional failing test was observed on the PR in this configuration. This is matching-base validation, not a test merged into today's main.

Windows 11 Enterprise 26200, Ryzen 9950X3D, RTX 5080 / NVIDIA 610.62 (32.0.16.1062), WinLibs GCC 15.2 r7, Release, testing ON, bundled dependencies, SPIRV-Tools OFF. Full builds and system-library targets pass; source checkouts remain clean. Vulkan required, ANYPS5_GPU="RTX 5080", ANYPS5_NO_SHADER_CACHE=1; the PR additionally uses APS5_VERIFY_SET_LAYOUTS=1.

Serial ctest -R "^(agc_|shader)" --parallel 1 --timeout 30:

Revision	GPL	Passed	Failed	Explicit skips
bca7a29	on	420	2	0
14b52c1	on	421	2	0
bca7a29	off (APS5_NO_GPL=1)	412	2	8
14b52c1	off	413	2	8

The same two failures on both revisions, in both modes:

• agc_write_tracking: an unwatched range reads as not stored over.
• agc_driver_prepared_shaders_registered_workgroup_sync: expected preparation failure (also noted in the PR's matching-base results).

Each failure reproduces 5/5 isolated attempts on each revision. The eight GPL-off skips are the pipeline_optimized_readback D32/D16/color32/color64 cases and their fast variants; the same skips occur on the base. The earlier selection of 16 relevant graphics/registration/cache groups passes five repeats each on both revisions. Maximum passed-test duration across the full runs is 10.674 s on this machine, not a lavapipe/CI timing guarantee.

For the NVIDIA LDS concern in #2242: agc_driver_pixel_lds passes five isolated repeats on each revision, as well as both full-suite modes on each. No timeout or VK_ERROR_DEVICE_LOST was logged. These are Windows / RTX5080 measurements on an already running GPU, not fresh-boot/Linux RTX4090 controls; they do not disprove the reported Xid109 problem there.

Limits: NVIDIA driver disk-cache coldness is not established. This is behavior validation, not a registration/first-draw speedup measurement. No PR-head title cold boot, PPSA36082/Hades II, PPSA01341, or Astro shader replay was performed; PPSA99902 needs additional startup/graphics prerequisites on this base. No game binaries/shaders/images were uploaded. AI-assisted independent build/test/report: Codex.

[adriadam10]

@Amoenus thanks, that's a useful control, especially agc_driver_pixel_lds on another NVIDIA generation.

Since 14b52c1, three changes are coming in the next push (rebased onto current main; it goes up once the local suites, validation layer and CI-like run pass on the rebased tree):

• Raghav's two fixes to the formatted buffer helper (Compiling Vulkan shader code at registration, not at the first draw (VK_EXT_shader_object) #2242). In the specialized form the helper takes the V# words as specialization constants again; that was a regression from the first commit and it inflated compute shaders (51k → 236k words in PPSA36082's loading frame). In registration form the format word is a constant inside the helper when it's known. FormattedBufferOnce now pins both: one formatted load gives 26.5k words, same as main, against 117.5k before.
• The buffer base misalignment stays a 1-bit specialization in registration form too. In pixel_lds's pixel shader that removes the 6 unaligned compare-exchange loops that registration form had and the specialized form doesn't (0 loops, 0 atomics in both forms now). It's the only structural difference I found in that shader, so it's my best guess for Raghav's Xid 109, but I can't confirm that without his 4090.

AI-assisted: yes (Claude).

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
#757	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp L1342-1343, L1421-1423, L1435-1437, L3463-3466, L3468-3469, L3474-3485 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp L586-587 core/shader/recompiler/Optimization/include/Optimization/DescriptorBindingBuilder.hpp L16 core/shader/recompiler/Optimization/src/BindingAllocator.cpp L105 core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp L296-297, L330-342, L407-418, L429 core/shader/recompiler/ShaderDiskCache.cpp L67-70 core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp L49-58, L60-62, L1656, L1679-1693 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvDescriptors.cpp L57-82
#758	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp L1342-1343, L1421-1423, L1435-1437, L3463-3466, L3468-3469, L3474-3485 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp L586-587 core/shader/recompiler/Optimization/include/Optimization/DescriptorBindingBuilder.hpp L16 core/shader/recompiler/Optimization/src/BindingAllocator.cpp L105 core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp L296-297, L330-342, L407-418, L429 core/shader/recompiler/ShaderDiskCache.cpp L67-70 core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp L49-58, L60-62, L1656, L1679-1693 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvDescriptors.cpp L57-82
#1366	core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp L430-434
#3656	core/libs/prx/libSceAgcDriver/tests/FormattedBufferOnce.cpp L101-102
#4048	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp L2939 core/shader/recompiler/CacheKey.hpp L243 core/shader/recompiler/Recompiler.cpp L420 core/shader/recompiler/Recompiler.hpp L232 core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp L23 core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp L93 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp L252
#4339	core/shader/recompiler/Recompiler.cpp L420-421, L425
#4363	core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp L138-139