Skip to content

perf(recompiler): keep runtime-specialized shader modules in the shader disk cache - #6924

Open
moong15 wants to merge 6 commits into
boykopovar:mainfrom
moong15:pr/specialized-module-disk-cache
Open

moong15 wants to merge 6 commits into
boykopovar:mainfrom
moong15:pr/specialized-module-disk-cache

Conversation

@moong15

@moong15 moong15 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What

Runtime-specialized shader modules and their binding plans were rebuilt on every launch, even with a warm shader disk cache. They are built on the queue worker when a draw or dispatch first needs them, so the rebuild lands on the critical path while a title loads.

They are now stored in the shader disk cache next to their base artifact, keyed by the base artifact's digest plus the exact specialization inputs, and loaded on a warm start. A missing, differently keyed or damaged record is rebuilt. The disk cache format goes to 41. APS5_NO_SPECIALIZED_DISK_CACHE=1 turns it off. The [shader-disk-cache] line reports specialized module and binding plan hits, misses and writes.

The specialized records are bounded. APS5_SPECIALIZED_DISK_CACHE_ENTRIES (default 65536) and APS5_SPECIALIZED_DISK_CACHE_MAX_MIB (default 2048) cap them together, and the least recently used specialized record is deleted when either limit is exceeded, after each specialized write and once at start. Base artifacts and resource plans are never evicted. 0 keeps none, the same as APS5_NO_SPECIALIZED_DISK_CACHE=1. Both switches are neutral to the cache key and documented in docs/user/USAGE.md; the defaults are guesses, recorded in docs/dev/debt/shader.md. The start-up [shader-disk-cache] line prints the specialized entry count and size on disk, and the 10 s line reports evictions.

A variant compiled without going through findOrCompileVariant (useCache=false: recompileOnce, and RecompileImpl without a cache or capture source) has no disk key. CompiledVariant::artifactIdentity is now empty for such a variant instead of zero-filled, and specializedDiskCache returns false unless the identity has the full artifact-identity length. Those compiles neither read nor write specialized entries, so two of them with different targets can no longer share one zero-keyed module or binding plan.

Tested

Windows 11 Pro 10.0.26300, MinGW Release build, AMD Radeon RX 9070 XT, AMD proprietary driver 26.8.1 (LLPC), 3840x2160. Stellar Blade (PPSA13197). Builds: main 95d901b and this branch rebased on it (e822824); the branch also merges cleanly with main dbac9fa. Both builds staged as whole library sets, swapped between runs, the embedded commit checked in each summary line. Captures as the Performance report form describes: warm shader and pipeline caches (each cache key ran the scene first), same scene and settings, one session, three runs per build back to back, no APS5_PROFILE_* switch. Deviations from the form: APS5_TRACE_EVENTS=4194304 is set because the default 262144 events per thread fill in about 11 s on the busiest queue worker of this title; stdout and stderr were written to files.

Loading: first 90 s from the first present (APS5_TRACE_SECONDS=0:90): main menu, save load and story start. Specialized time is the sum of Shader.DrawStage.materialize, Driver.Draw.shader_compile_and_link and Driver.Dispatch.materialize slices in each trace.

Run Build Presents p50 / p90 / p99 ms Longest frame ms Specialized time s
1 main 1042 31.4 / 145.7 / 577.1 16606 20.0
2 main 1063 31.5 / 143.9 / 600.2 15864 19.8
3 main 1108 31.4 / 133.2 / 562.2 15963 19.6
4 this PR 1054 31.4 / 135.9 / 681.1 15414 21.2
5 this PR 1002 31.5 / 152.2 / 642.0 13974 8.4
6 this PR 1028 31.5 / 150.5 / 705.9 13662 7.6

With the specialized records read from disk (runs 5 and 6), specialized time falls from 19.6-20.0 s to 7.6-8.4 s and the longest loading stall from 15.9-16.6 s to 13.7-14.0 s. Run 4, the first launch after the build switch, read 1041 specialized records but spent 21.2 s, more than main, with 5.4 s in Shader.PrepareArtifact; it was the first read of that cache directory in the session. Presents and frame percentiles do not improve in this window: the menu runs at its 32 fps cap and the loading frames are dominated by Driver.Packet.memory_wait in both builds. Each run still writes 66-91 new specialized records, so the directory keeps growing (17132 to 19627 entries over the three runs).

Steady play: 60-210 s from the first present (APS5_TRACE_SECONDS=60:150), same machine and session, to check for a regression: main 1417 / 1364 / 1778 presents (9.44 / 9.09 / 11.85 fps), this PR 1755 / 1778 / 1388 (11.70 / 11.85 / 9.25 fps). The spread between runs of one build is larger than the difference between builds; no regression.

Summary lines (runs 3 and 6):

{"commit":"95d901b9371f","dirty":false,"env":{"APS5_CONTINUE_AFTER_FLIP_FAILURE":"1","APS5_FLAT_3D_COLOR_TARGET":"1","APS5_HOST_IMPORT_MIB":"16384","APS5_PAD_INPUT_FILE":"<path>","APS5_PREBUILD_THREADS":"16","APS5_TRACE_EVENTS":"4194304","APS5_TRACE_SECONDS":"0:90"},"gpu":"AMD Radeon RX 9070 XT","driver":"AMD proprietary driver 26.8.1 (LLPC)","os":"Windows 10.0.26300","resolution":"3840x2160","window":{"spec":"seconds 0:90","first_present":0,"ms":90015.371},"presents":1108,"frame_ms":{"p50":31.386,"p90":133.159,"p99":562.207,"max":15963.420},"gpu_busy_ms_per_present":25.991,"worker_busy_pct":{"queue worker 0x0":87.342,"queue worker 0x20":65.636},"top_phases_ms_per_frame":{"Driver.Packet":123.855,"Driver.Packet.memory_wait":49.825,"Graphics.Draw":31.030,"Graphics.Draw.draw_and_resource_release":26.082,"VideoOut.Flip":24.079,"Driver.Present":23.980,"VideoOut.Flip.present":23.974,"Driver.Present.acquire_image":23.243},"trace_dropped":false,"trace":"aps5-trace-95d901b9371f-6884.json.gz"}
{"commit":"e822824b83ce","dirty":false,"env":{"APS5_CONTINUE_AFTER_FLIP_FAILURE":"1","APS5_FLAT_3D_COLOR_TARGET":"1","APS5_HOST_IMPORT_MIB":"16384","APS5_PAD_INPUT_FILE":"<path>","APS5_PREBUILD_THREADS":"16","APS5_TRACE_EVENTS":"4194304","APS5_TRACE_SECONDS":"0:90"},"gpu":"AMD Radeon RX 9070 XT","driver":"AMD proprietary driver 26.8.1 (LLPC)","os":"Windows 10.0.26300","resolution":"3840x2160","window":{"spec":"seconds 0:90","first_present":0,"ms":91383.375},"presents":1028,"frame_ms":{"p50":31.543,"p90":150.496,"p99":705.882,"max":13661.595},"gpu_busy_ms_per_present":26.949,"worker_busy_pct":{"queue worker 0x0":87.391,"queue worker 0x20":68.124},"top_phases_ms_per_frame":{"Driver.Packet":137.629,"Driver.Packet.memory_wait":56.181,"Graphics.Draw":39.495,"Graphics.Draw.draw_and_resource_release":33.229,"Driver.Packet.draw":21.546,"Driver.Draw":21.484,"VideoOut.Flip":20.939,"Driver.Present":20.838},"trace_dropped":false,"trace":"aps5-trace-e822824b83ce-16792.json.gz"}

Traces, trace_digest.py output and summary lines of all twelve captures (load-1 to load-6 and play-1 to play-3 per build): https://github.com/moong15/AnyPS5/releases/tag/pr6924-evidence

Tests, all on the same machine and GPU, ctest -V -j1 with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, on main and on this branch: agc_shader_disk_cache, agc_float16_conversion, agc_denormal_flush_elimination, agc_shader_control_flow, agc_wave_control_decode, agc_driver_f16_denormal_flush, agc_driver_f32_denormal_flush: 7/7 pass on both. Neither log reports was not found but was requested by env var VK_INSTANCE_LAYERS; neither log has a VUID- line.

  • agc_shader_disk_cache now also runs a child process (--uncached-specialized) that compiles twice with useCache=false for a device with and one without RoundingModeRTE against one cache directory, and requires no specialized or binding-plan hit, miss or write and no stored entry. With the identity change reverted it fails: "Recompile(request) with useCache=false for a device without RoundingModeRTE was served the module compiled for a device with it (specialized hits 1, misses 1, writes 1; binding plan hits 1, misses 1, writes 1 ...)".
  • The review's sequences, with a real cache directory and without ANYPS5_NO_SHADER_CACHE: agc_float16_conversion_tests into an empty directory, and agc_shader_control_flow_tests, agc_wave_control_decode_tests, agc_denormal_flush_elimination_tests into one directory, each with the specialized cache on and with APS5_NO_SPECIALIZED_DISK_CACHE=1: all pass, and no specialized-* file is written.
  • agc_shader_disk_cache also covers store, load, damaged and truncated records, APS5_NO_SPECIALIZED_DISK_CACHE=1, the entry cap (an evicted record misses and is rebuilt, a recently used one survives), the byte cap, a malformed and a zero cap value, a base artifact that changes under the same specialization inputs, and the DescriptorBinding round trip including imageAtomic64.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: AMD Radeon RX 9070 XT, AMD proprietary driver 26.8.1 (LLPC), Windows 11 Pro 10.0.26300
  • AI-assisted: yes

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1896 docs/dev/ENVIRONMENT.md
PR 3786 docs/user/USAGE.md
PR 4472 docs/dev/ENVIRONMENT.md
PR 4847 docs/user/USAGE.md
PR 5021 core/shader/recompiler/Recompiler.cpp
docs/dev/debt/shader.md
docs/user/USAGE.md
PR 5315 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 5371 docs/dev/debt/shader.md
PR 5609 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 5925 core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/user/USAGE.md
PR 6171 docs/user/USAGE.md
PR 6185 docs/dev/ENVIRONMENT.md
PR 6186 docs/dev/ENVIRONMENT.md
PR 6235 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6280 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 6412 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 6488 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 6501 docs/dev/ENVIRONMENT.md
PR 6612 core/shader/recompiler/Recompiler.cpp
docs/dev/debt/shader.md
PR 6858 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 6959 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 6970 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 7085 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 7201 core/shader/recompiler/Recompiler.cpp
PR 7223 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 7392 docs/dev/debt/shader.md
PR 7395 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 7410 core/shader/recompiler/Recompiler.cpp
docs/dev/debt/shader.md
PR 7449 docs/dev/ENVIRONMENT.md
PR 7484 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 7514 docs/dev/ENVIRONMENT.md
PR 7522 core/shader/recompiler/Recompiler.cpp
PR 7604 docs/dev/ENVIRONMENT.md
PR 7622 core/shader/recompiler/ShaderDiskCache.cpp
PR 7642 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 7643 docs/dev/ENVIRONMENT.md
PR 7684 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 7689 core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 7694 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 7922 docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 8004 core/shader/recompiler/Recompiler.cpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 8077 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 8160 docs/dev/ENVIRONMENT.md
PR 8161 docs/dev/ENVIRONMENT.md
PR 8246 docs/dev/ENVIRONMENT.md
PR 8458 docs/dev/debt/shader.md
PR 8542 docs/dev/ENVIRONMENT.md
PR 8887 docs/dev/ENVIRONMENT.md
PR 8896 docs/dev/ENVIRONMENT.md
PR 8939 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 8963 docs/dev/ENVIRONMENT.md
PR 9012 docs/dev/ENVIRONMENT.md
PR 9020 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 9098 docs/dev/ENVIRONMENT.md
PR 9129 docs/dev/ENVIRONMENT.md
PR 9176 docs/dev/ENVIRONMENT.md
PR 9183 core/shader/recompiler/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 9193 core/shader/recompiler/Recompiler.cpp
docs/dev/debt/shader.md
PR 9243 docs/dev/ENVIRONMENT.md
PR 9248 docs/dev/ENVIRONMENT.md
PR 9287 docs/dev/ENVIRONMENT.md
PR 9289 docs/dev/ENVIRONMENT.md
PR 9335 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 9345 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 9346 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 9356 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/ENVIRONMENT.md
PR 9387 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 9457 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 9508 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 9626 docs/dev/ENVIRONMENT.md
PR 9633 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 9671 docs/dev/debt/shader.md
PR 9676 docs/dev/ENVIRONMENT.md
PR 9712 docs/dev/ENVIRONMENT.md
PR 9723 docs/dev/ENVIRONMENT.md
PR 9743 docs/dev/debt/shader.md
PR 9748 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 9749 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 9790 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 9852 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/ENVIRONMENT.md
PR 9875 docs/dev/ENVIRONMENT.md
PR 9879 docs/dev/ENVIRONMENT.md
PR 9889 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 9897 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 9898 docs/dev/ENVIRONMENT.md
PR 9969 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 10000 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 10043 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 10049 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 10069 docs/dev/ENVIRONMENT.md
PR 10193 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 10194 docs/dev/ENVIRONMENT.md
PR 10233 docs/dev/debt/shader.md
PR 10351 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 10359 core/shader/recompiler/Recompiler.cpp
PR 10382 docs/dev/ENVIRONMENT.md
PR 10388 docs/dev/ENVIRONMENT.md
PR 10442 docs/dev/ENVIRONMENT.md
PR 10443 docs/dev/ENVIRONMENT.md
PR 10472 docs/dev/ENVIRONMENT.md
PR 10479 docs/user/USAGE.md
PR 10487 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 10489 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/debt/shader.md
PR 10509 docs/user/USAGE.md
PR 10547 core/shader/recompiler/Recompiler.cpp
docs/dev/ENVIRONMENT.md
PR 10556 docs/dev/debt/shader.md
PR 10577 docs/user/USAGE.md
PR 10652 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
docs/dev/debt/shader.md
PR 10668 docs/dev/ENVIRONMENT.md
PR 10678 docs/dev/ENVIRONMENT.md
PR 10712 docs/user/USAGE.md
PR 10720 core/shader/recompiler/ShaderDiskCache.cpp
docs/dev/ENVIRONMENT.md
PR 10750 docs/dev/ENVIRONMENT.md
PR 10757 core/shader/recompiler/Recompiler.cpp
docs/dev/ENVIRONMENT.md
PR 10792 docs/user/USAGE.md
PR 10803 docs/dev/debt/shader.md
PR 10809 docs/user/USAGE.md
PR 10831 docs/dev/debt/shader.md
PR 10846 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 10858 docs/dev/debt/shader.md
PR 10867 docs/dev/debt/shader.md
PR 10871 docs/user/USAGE.md
PR 10894 docs/dev/ENVIRONMENT.md
PR 10941 core/shader/recompiler/Recompiler.cpp
docs/dev/debt/shader.md
PR 10970 docs/user/USAGE.md
PR 10976 core/shader/recompiler/Recompiler.cpp
PR 10983 docs/user/USAGE.md
PR 10997 docs/user/USAGE.md
PR 11084 docs/user/USAGE.md
PR 11107 docs/user/USAGE.md
PR 11112 docs/dev/debt/shader.md
PR 11130 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 11137 docs/user/USAGE.md
PR 11199 docs/dev/debt/shader.md
PR 11207 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 11225 docs/user/USAGE.md
PR 11226 docs/user/USAGE.md
PR 11254 docs/dev/debt/shader.md
PR 11255 docs/dev/debt/shader.md
PR 11256 docs/dev/debt/shader.md
PR 11265 core/shader/recompiler/Recompiler.cpp
PR 11297 core/shader/recompiler/Recompiler.cpp
PR 11305 core/shader/recompiler/Recompiler.cpp
PR 11307 docs/dev/ENVIRONMENT.md
PR 11351 core/shader/recompiler/Recompiler.cpp
docs/dev/ENVIRONMENT.md
PR 11363 docs/dev/ENVIRONMENT.md
PR 11365 docs/dev/ENVIRONMENT.md
PR 11434 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 11435 docs/dev/ENVIRONMENT.md
PR 11436 docs/dev/ENVIRONMENT.md
PR 11451 docs/dev/ENVIRONMENT.md
PR 11456 docs/dev/ENVIRONMENT.md
PR 11457 docs/user/USAGE.md
PR 11483 core/shader/recompiler/Recompiler.cpp
docs/dev/debt/shader.md
PR 11489 docs/user/USAGE.md
PR 11507 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 11510 docs/dev/debt/shader.md
PR 11520 docs/dev/ENVIRONMENT.md
PR 11551 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp

@CnRJay CnRJay left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is sound (rebuilding specialized modules and binding plans on every launch lands on the queue worker during loading), and the key test covers each input. The evidence does not yet show the gain, and the cache has no bound.

  • The two summary lines the description attaches point the other way in the window they cover: cache on 438 presents, p90 563 ms, p99 2145 ms, Driver.Packet 343 ms per frame; cache off 783 presents, p90 42 ms, p99 1368 ms, Driver.Packet 186 ms per frame. The description explains that the windows cover different parts of loading, which is the problem: they can't be compared. The "119 s against 138 s to the menu" is one pair read off the 10 s intervals, and the earlier runs it cites (83-112 s on, 105-159 s off) overlap. The cache-on trace also lost 344,091 worker events after 35 s (trace_dropped: true). Please post a pair that covers the same stretch (start the window at the same title milestone, e.g. the first menu present, and run to a fixed time after it), a second pair to show the spread, and traces that are whole (APS5_TRACE_EVENTS).
  • The key includes the captured descriptors and the specialization constants, so a title whose descriptors vary writes a new entry for every distinct combination. Nothing bounds the number or size of these entries or evicts them. Post the entry count and size on disk after the Stellar Blade runs, and add a cap with eviction before this lands, otherwise the shader cache grows with play time.
  • Disk cache format 37 is also taken by the rebased #5656 (SrtUserDataUse) and by JJenkx's refresh of it, and #6947 uses 38. Whoever lands second has to renumber; say which this PR expects to follow.
  • The test covers store, load, damaged and truncated records. It does not cover a base artifact that changes under the same specialization inputs (the key is the base digest, so it should miss): add that case.

@moong15
moong15 force-pushed the pr/specialized-module-disk-cache branch from 6b77e05 to 80a6016 Compare October 17, 2026 02:16
@moong15

moong15 commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

Bound on the entries. Added. Specialized module and binding-plan entries are capped by APS5_SPECIALIZED_DISK_CACHE_ENTRIES (default 65536) and APS5_SPECIALIZED_DISK_CACHE_MAX_MIB (default 2048), read like APS5_PIPELINE_STORE_ENTRIES and APS5_PIPELINE_JOURNAL_MAX_MIB and neutral to the cache key. When either limit is exceeded the least recently used specialized entry is deleted (after each specialized write and once at start). Base artifacts and resource plans are never evicted, since only files with the specialized- prefix are candidates. 0 turns the feature off. Both defaults are guesses and are recorded in docs/dev/debt/shader.md. Start-up now prints the entry count and size on disk, and the 10 s [shader-disk-cache] line reports evictions:

[shader-disk-cache] 2044 specialized entries, 189.4 MiB on disk, of a limit of 65536 entries and 2048 MiB

Tests: --specialized-cap-entries (the cap is enforced, an evicted entry misses and is rebuilt, a recently used one survives), --specialized-cap-bytes, --specialized-cap-invalid, --specialized-cap-off.

A base artifact that changes under the same specialization inputs. Added --load-specialized-artifact-changed. It changes one code word of a stored fixture's base artifact and requires one module miss and write, one plan miss and write, and the other 15 fixtures still hitting. The entry damage test also requires KeyMismatch when an entry is decoded under another base digest with the same inputs.

Format number. It stays 37 for now. If #5656 lands first, this PR moves to 38, or to 39 if #6947 also lands. The only literal is ShaderDiskCache.hpp:17.

Tests: agc_shader_disk_cache passes; full ctest 1346/1351, and the 5 failures also fail on main or depend on the environment.

@Sam-mon7

Copy link
Copy Markdown
Contributor

From reading the diff: the disk load and store wrap buildSpecializedModule inside specializeModule, and the binding plan build inside preparedBindingPlan. On main, neither function sits inside #if ANYPS5_ENABLE_SPIRV_TOOLS, and the diff adds no tools check, so this should also cover tools-off builds (the CMake default).

That is the case raised in the Demon's Souls discussion #1283: 351 specialized modules rebuilt on every launch, about 60 s on Linux NVIDIA with the tools off, where the persistent cache in another branch only covered the tools-on path. A cold and a warm tools-off launch of that title would confirm it. I haven't measured it yet.

AI-assisted: yes

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 17, 2026
@moong15
moong15 force-pushed the pr/specialized-module-disk-cache branch from d77d30a to 19054f1 Compare October 17, 2026 14:22
@moong15

moong15 commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main 47a266c. Two conflicts, both from code main added next to this PR's: runningWriters next to buildSpecializedKeyHeader in ShaderDiskCache.cpp, and removeCacheDirectory next to the new test helpers in the test file. I kept both sides of each. agc_shader_disk_cache (all modes, including the cap and changed-artifact cases) and the shutdown thread tests main added pass on the rebased head, and check_conventions.py passes.

The disk cache format stays at 37. #5656 and #6947 are still open, so the renumbering plan from my earlier comment still applies.

@Sam-mon7, correct. Neither specializeModule nor preparedBindingPlan sits inside an ANYPS5_ENABLE_SPIRV_TOOLS block, and the cache adds no tools check. The Stellar Blade numbers in the description also came from a tools-off build (ANYPS5_ENABLE_SPIRV_TOOLS=OFF, the CMake default), so tools-off is the case that was measured. I haven't run Demon's Souls.

@CnRJay, the matched pairs you asked for are still to come: two pairs of cache on and off, each starting at the first menu present and running a fixed time after it, with whole traces. I'll post them in a separate comment. The code is ready for review in the meantime.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 17, 2026
@simonbronk

Copy link
Copy Markdown
Contributor

Tested on Onimusha's menu: no measurable change in this scene.

Setup: Onimusha: Way of the Sword (PPSA27836), boot to the language menu with no input, APS5_TRACE_SECONDS=60:30, warm cache. Windows 11 26200, RTX 5070 (617.42), 3840x2160. Base main 84b0d2f + #6557 #6727 #6762 (5c521efea), 6 runs (3 before and 3 after the PR runs). The PR is merged on top of it (clean merge), 3 runs.

  • Base frame p50 (ms): 84.5, 82.6, 81.8, 81.5, 77.9, 83.5 (mean 82.0); p90 84.3-93.6.
  • With this PR, p50: 80.7, 75.8, 80.8. Within the base's run-to-run range.
  • The PR changes the disk cache format, so one 180 s warm-up run rebuilt the cache before the 3 measured runs. I can't confirm that rebuild finished inside the warm-up, and this menu scene doesn't measure the in-game compile spikes the PR targets.

This scene may simply not exercise the path this PR speeds up. Summary lines and traces available on request. AI-assisted: yes.

@moong15

moong15 commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main 09127ea. A correction to my earlier comments about the format number: neither #5656 nor #6947 landed first. #6095 took 37 on main, so this PR now moves the disk cache format to 38 (ShaderDiskCache.hpp:17, still the only literal). #6947 also uses 38, so whichever of the two lands second renumbers.

The only conflict was that line. agc_shader_disk_cache and agc_shader_disk_cache_large_file pass on the rebased head, and check_conventions.py passes.

@simonbronk, thanks for the Onimusha numbers. A language menu right after boot probably compiles few specialized modules, so no change there is expected. The gain shows up when a title loads its scenes on a second launch.

@CnRJay, the matched pairs follow in a separate comment.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@moong15

moong15 commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

@CnRJay, matched runs of this branch at 3483ed2 (rebased onto main 09127ea, format 38), Stellar Blade, same scene: boot, start the story, idle in the opening cutscene. All four used the same warm cache directory (07a088e75f6aaf46, warmed by three earlier runs), APS5_TRACE_SECONDS=0:240, APS5_TRACE_EVENTS=4194304, APS5_PREBUILD_THREADS=16. Order ABBA: on, off, off, on; off is APS5_NO_SPECIALIZED_DISK_CACHE=1.

The first 60 s after the first present are a loading stretch whose frame count swings in both settings (1204, 640, 710 and 663 presents), so the comparison below is 60–240 s from the first present, where all four runs show the same scenes in the same order. Numbers are from the present events and slices in the traces.

Run Specialized cache Presents, 60–240 s Frame p50 / p90 / p99 ms, 60–240 s Driver.Draw.shader_compile_and_link, 60–200 s Specialized hits / misses Binding plan hits / misses
1 on 1670 85.9 / 180.8 / 393.9 8.28 s 1307 / 106 2016 / 2373
2 off 1562 87.1 / 186.3 / 511.1 8.96 s n/a n/a
3 off 1494 92.7 / 196.1 / 498.1 9.22 s n/a n/a
4 on 1686 86.3 / 179.8 / 414.0 9.20 s 1376 / 107 2156 / 2429
  • On: 1678 presents on average against 1528 off (+10%), p99 404 ms against 505 ms (−20%), p90 180 ms against 191 ms. The two on runs agree within 16 presents, and both are above both off runs.
  • Compile and link time in 60–200 s is about the same in both settings (8.3–9.2 s), so the gain shows up as fewer long frames rather than less compile time on the draw path. The 60–200 s range stops before queue worker 0x0's slice buffer filled (at 205–212 s in all four runs); the summary totals stay complete (totals_full: false).
  • Specialized modules hit about 93% of the time on a warm cache.
  • Binding plans miss about 2400 times in each on run, and each on run adds about 2600 files to the directory (11492 → 14082 specialized entries after run 1, +2639 after run 4, about 40 MiB each time). The off runs add none. With the same scene and a warm cache, those plans look like they get a new key every run, so they never hit and the directory grows every session until eviction.
  • No FATAL in runs 1–4. One earlier off attempt crashed 26 s in on the game's FAsyncLoadingThread, with the same call chain as an intermittent load crash I have seen with both settings. It wrote no trace and was repeated as run 2.

The four traces and their tools/trace_digest.py summaries are available on request.

Windows 11, AMD Radeon RX 9070 XT, AMD 26.8.1 (LLPC), 3840×2160.

@moong15
moong15 force-pushed the pr/specialized-module-disk-cache branch from 3483ed2 to 2a816b6 Compare October 18, 2026 18:26
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@moong15
moong15 force-pushed the pr/specialized-module-disk-cache branch from 2a816b6 to 98c84f6 Compare October 18, 2026 18:39
@github-actions github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line and removed needs rebase Conflicts with main, rebase needed labels Oct 18, 2026
@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed needs rebase Conflicts with main, rebase needed labels Oct 19, 2026
moong15 and others added 6 commits October 20, 2026 07:34
…er disk cache

Specialized modules and their binding plans were rebuilt on every launch
even with a warm disk cache. They are now stored next to their base
artifact, keyed by its digest plus the exact specialization inputs, and
loaded on a warm start; a key mismatch or corrupt record rebuilds. The disk
cache format goes to 37. APS5_NO_SPECIALIZED_DISK_CACHE=1 turns it off.
The [shader-disk-cache] line reports specialized module and binding plan
hits, misses and writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…viction

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ment reference

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…der's DescriptorBinding assert

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… specialized disk cache

The specialized module and binding plan entries are keyed by the base
artifact's identity, which only findOrCompileVariant fills. Variants
compiled with useCache=false (recompileOnce, and RecompileImpl without a
cached source) kept an all-zero identity, so every such compile read and
wrote the same entries whatever the shader or target. The identity is now
empty unless the variant has a disk key, and specializedDiskCache refuses
an empty one. The new --uncached-specialized mode compiles one shader
twice without the cache, with and without RoundingModeRTE, and requires
that the second compile gets its own module and that nothing is stored.

After the rebase onto main, the DescriptorBinding encoder assert counts
24 fields and the sameBinding round trip compares imageAtomic64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@moong15
moong15 force-pushed the pr/specialized-module-disk-cache branch from e822824 to d36ab7d Compare October 20, 2026 02:09
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 20, 2026
@eliasbruno124-dev

eliasbruno124-dev commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

I tested d36ab7d on GTA III, PPSA03527/01.003.000, and found repeated specialization misses that may be worth fixing here.

After warming the cache, a separate diagnostic launch reported 57 specialized-module hits and 9 misses/writes, with no load or write failures. All nine newly written records already had an older record with the same base-artifact identity and every other specialized-key field, except planIdentity. Each new payload was byte-identical to its older alternatives. Across the retained starts there are 92 module records but only 66 groups when that field is excluded; all nine duplicate groups have identical payloads.

buildSpecializedModule consumes the artifact, vertex classes, constants and target. specializeModule also shares results in memory by artifact variant and constants. The disk key additionally includes the binding-plan identity, so a different descriptor snapshot can rebuild the same module on the next process. Could the specialized-module key omit that extra identity while retaining it for the separately cached binding plan? A cross-process test should vary the descriptor snapshot without changing specialization inputs, require a module hit, and still check the resulting bindings against an uncached compile. At the time of this initial report I had not changed the runtime key. No competing PR has been published.

The existing cache does work. The same eight focused test cases pass on both sides. The candidate also passes its cache round trips, changed-base-artifact, corruption, bounds and uncached different-target regressions. The real Khronos validation layer was loaded for the GPU case, with no new errors or skips. Host: RTX 3060 Laptop, NVIDIA 617.42, Windows 10.0.26300, i5-11400H, 32 GiB RAM.

The two alternating pairs of complete 20-second automatic-startup traces do not establish an overall speedup. Baseline maximum frame intervals were 4263/4314 ms; with this PR, 4065/4745 ms. All four traces had no dropped events and 61 verified loaded PRXs. Both sides used main 7526640 plus the same deferred-settle correction from #8527; only this PR differed. These are opening-startup observations, not gameplay or a claim that the initial long interval is a shader stall.

Update after a local regression and original-title check: changing only the guarded-image descriptor snapshot in a second process reproduces 16 module hits and 1 unnecessary miss/write on the unchanged implementation, although the result matches an uncached compile. Removing planIdentity only from the specialized-module key gives 17 hits and zero misses/writes; the changed binding plan still misses and is rebuilt, with output matching the uncached control. Artifact/target identity, module words, vertex classes, constants and registration defaults remain in the key. The new encoding uses a new format version.

The same original GTA III startup, after warming the new cache normally, now reports 66 specialized hits, zero misses and zero new module records, versus 57 hits/9 misses before. All 66 module payloads match payloads already produced by the previous build. Both runs verified 61 loaded PRXs and unchanged original files. The corrected build passes the same eight native cases with the real Khronos layer, no validation errors or skips, plus the new cross-process regression. This proves removal of redundant specialization, not a loading-time or FPS gain. The focused local patch and failing/passing regression are ready for coordination here.

Prepared with AI assistance.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 20, 2026
BiladeiroSupremo added a commit to BiladeiroSupremo/AnyPS5Wolverine that referenced this pull request Oct 22, 2026
…t in the background

The specialized entries under <entries>/specialized grew without a bound and
were compressed and written on the thread that built them (the queue worker or
a specialization worker).

- A background writer compresses and writes each new entry.
- An in-memory LRU index (rebuilt from file write times at first use; a hit
  stamps its file) evicts the least recently used entries past
  APS5_SPECIALIZED_DISK_CACHE_ENTRIES (65536) or
  APS5_SPECIALIZED_DISK_CACHE_MAX_MIB (2048), the defaults of upstream boykopovar#6924.
- Temp files older than 10 minutes, left by a write the process exit cut off,
  are removed at the first scan.
- One stderr line per 10 s with activity: hits, misses, writes, evictions and
  the entries held against the caps.
- ShaderDiskCache::Flush also waits for the specialized writer.
  agc_shader_disk_cache_tests hung in its final remove_all: libstdc++ 15.2
  kept stat-ing a .tmp. entry the writer had renamed after the listing.

Checked: agc_shader_disk_cache_tests and the sin/cos tests pass;
agc_shader_replay stores one entry, loads it in a second process with
identical SPIR-V, keeps 1 file with
APS5_SPECIALIZED_DISK_CACHE_ENTRIES=1 and two shaders, and removes a
1-hour-old temp file while keeping a fresh one. Not measured in game yet.
alex73630 added a commit to alex73630/AnyPS5 that referenced this pull request Oct 24, 2026
Integrate PR boykopovar#6924 at d36ab7d and preserve the local image-read and sampler-LOD metadata in cached binding plans. Keep uncached variants outside the persistent cache and bound specialized entries independently of base shader artifacts.

Validation: the same 10 focused shader and pipeline tests pass on origin/main d01070d and this integration with required Vulkan and the Khronos validation layer, with no VUID errors. The GT7 warm launch loads specialized modules across processes; matched cache-disabled timing remains under test.

Refs boykopovar#6924

Co-authored-by: umang <sheoranumang@gmail.com>
alex73630 added a commit to alex73630/AnyPS5 that referenced this pull request Oct 24, 2026
Integrate PR boykopovar#6924 at d36ab7d and preserve the local image-read and sampler-LOD metadata in cached binding plans. Keep uncached variants outside the persistent cache and bound specialized entries independently of base shader artifacts.

Validation: the same 10 focused shader and pipeline tests pass on origin/main d01070d and this integration with required Vulkan and the Khronos validation layer, with no VUID errors. The GT7 warm launch loads specialized modules across processes; matched cache-disabled timing remains under test.

Refs boykopovar#6924

Co-authored-by: umang <sheoranumang@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs needs rebase Conflicts with main, rebase needed perf Performance change shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants