Skip to content

feat(agc): trace draw-key cache misses - #5364

Merged
adriadam10 merged 3 commits into
boykopovar:mainfrom
MarcZX10:marc/trace-draw-key-misses
Oct 15, 2026
Merged

adriadam10 merged 3 commits into
boykopovar:mainfrom
MarcZX10:marc/trace-draw-key-misses

Conversation

@MarcZX10

@MarcZX10 MarcZX10 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Adds per-group attribution for draw register-key cache misses: stable registers, user-data registers, auxiliary registers, device, and shader set, plus misses where none of those components changed.

This extends the existing APS5_TRACE_DRAW_CACHE report, which already distinguishes new-register misses, user-word/pointer-only misses, evictions, and keys seen but never inserted. It reuses traceDrawCache() and its 10-second report; there is no separate environment variable or stderr logging path. Shader-set identity is included so distinct snapshots are tracked separately.

Tested

  • Linux: built agc_driver_register_key_tests and libSceAgcDriver.prx successfully with Ninja Release.
  • APS5_VERIFY_REGISTER_KEYS=1 ctest --test-dir /tmp/anyps5-open-diagnostics-main-build -R '^agc_driver_register_key$' --output-on-failure — the test executable cannot enter its test body on this host: Xeon E5-1620 v2 lacks LZCNT, BMI1, BMI2, MOVBE, AVX2, and FMA.
  • python3 tools/check_conventions.py --base upstream/main — passed.
  • git diff --check upstream/main...HEAD — passed.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical-debt comments
  • Unimplemented paths throw or are documented; not applicable
  • No notes, investigation files or images added
  • No third-party code added
  • Generic behaviour, not title-specific
  • Depends on: none
  • Unblocks: none
  • GPUs tested: n/a (diagnostic only)
  • AI-assisted: yes

@fma-rc

fma-rc commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

I ran this head merged into main 0aa4cb1 on Astro's Playroom (Linux, RTX 5080) with APS5_TRACE_DRAW_KEY_MISSES=1. About 145,000 draw-key lookups miss per 10 s there, and the whole run printed four lines:

[draw-key-miss] shader-set=cbf29ce484222325 changed=first-seen
[draw-key-miss] shader-set=cbf29ce484222325 changed=stable-registers,user-data-registers,device
[draw-key-miss] shader-set=cbf29ce484222325 changed=stable-registers,user-data-registers,device
[draw-key-miss] shader-set=cbf29ce484222325 changed=stable-registers

cbf29ce484222325 is the FNV offset basis. In drawRegisterKey, (*components)[4] is set to it and the shader identities are mixed into (*components)[3] only, so every draw has the same shader-set id and the four-lines-per-set cap stops the trace after four misses. It also means device reports a change whenever the shaders differ. agc_driver_register_key passes here.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

mixShaderIdentity mixes into (*components)[3], the device slot, so (*components)[4] stays at the FNV basis for every draw (DrawRegisterKey.cpp:86-95). Every draw then has the same shader-set id, device changes whenever the shaders do, and the four-line cap ends the trace after four misses, as fma-rc's run shows. Please mix the shader identity into [4] and add a check in tests/RegisterKey.cpp that two shader sets get different ids.

@MarcZX10

Copy link
Copy Markdown
Contributor Author

Fixed the component mix-up: shader snapshot identities and offsets now feed the shader-set key in components[4], while components[3] remains the device serial. Added a regression comparing two registries with different shader snapshots. Linux Release build of agc_driver_register_key_tests succeeded; focused CTest passed 1/1, and conventions passed.

@adriadam10 adriadam10 self-assigned this Oct 14, 2026
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the new head: components[4] now carries the shader-set key and components[3] the device serial. agc_driver_register_key_tests builds and passes with APS5_VERIFY_REGISTER_KEYS=1, and the new shader-set check fails when the snapshot mix in DrawShaderSetKey is removed. Default behaviour is unchanged with the variable unset. Merging once the full Build is green.

@parl0v

parl0v commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Ran this in a title: PPSA25344 (Sword of the Sea), title screen, main f0854887d + this PR at 2dd5ecdf4 (after the components[4] fix), APS5_TRACE_DRAW_KEY_MISSES=1, RTX 3060, Windows 10, 300 s. It builds and runs, and frame rate matched main (3.70 FPS both).

It printed 217 [draw-key-miss] lines from 57 shader sets (the per-set cap of 4 is hit by 52 of them):

changed= lines
stable-registers,user-data-registers 81
user-data-registers 76
first-seen 57
no-component-change 1
truncated (changed=st, a line cut by interleaved output) 1

Among the sampled repeat misses, user-data registers changed in all but one, and stable registers changed along with them in about half. One note from using it: there's no per-10 s total, so the lines show which components change but not how many draws miss in all. A count line like #5408's [keymiss] summary would make it comparable across runs.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 14, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now conflicts with main in DrawRegisterKey.hpp/.cpp, Driver.hpp and Draw.cpp, and it is not a mechanical rebase: #58b6ddf72 added APS5_TRACE_DRAW_CACHE (traceDrawCache(), absentUserWords / absentNewRegisters / absentEvicted counters) and changed RegisterStateKey/drawRegisterKey to return a shapeKey through the same parameter slot this PR uses for RegisterStateKeyParts / components.

Could you rebase onto current main and:

  • reuse the existing traceDrawCache() gate and shape key instead of adding a second env var and a second miss path in Driver::draw, or say why a separate trace is needed;
  • keep only what main does not already report (the per-group breakdown: stable / user-data / auxiliary / device / shader-set), and say in the PR body what it adds over the absentUserWords / absentNewRegisters split;
  • rerun agc_driver_register_key_tests with APS5_VERIFY_REGISTER_KEYS=1 on the rebased head.

The components[4] fix and the new shader-set test looked right on the previous head; they only need to survive the rebase.

@MarcZX10
MarcZX10 force-pushed the marc/trace-draw-key-misses branch from 2dd5ecd to fdba064 Compare October 14, 2026 23:58
@MarcZX10

Copy link
Copy Markdown
Contributor Author

Rebased onto current main and integrated the requested breakdown with traceDrawCache() and its existing 10-second report. The existing absentNewRegisters / absentUserWords / eviction accounting remains intact; the report now adds miss counts for stable registers, user-data registers, auxiliary registers, device, first-seen shader sets, and misses where no component changed. Shader-set identity remains in component slot 4, with a regression test for distinct snapshots.

Validation on the rebased head: the focused agc_driver_register_key_tests target builds successfully and conventions pass. CTest starts the binary but cannot enter the test body on this Xeon E5-1620 v2 because the host lacks LZCNT, BMI1, BMI2, MOVBE, AVX2, and FMA. I recorded that limitation in the PR description.

@fma-rc

fma-rc commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

@MarcZX100 the force-push at 23:58 UTC left this branch with a single commit, fdba064 fix(libkernel): log resolved paths for failed opens, which only touches core/libs/prx/libkernel/File/src/Open.cpp and core/libs/tests/GuestFilesystem.cpp. The draw-key work and the traceDrawCache() breakdown you describe aren't in the diff any more (no traceDrawCache or register_key lines), so it looks like another branch went to marc/trace-draw-key-misses. Once the right commit is back I'll build it and run it on Astro's Playroom, where most draw-cache misses carry the same key as the set's last one.

@MarcZX10
MarcZX10 force-pushed the marc/trace-draw-key-misses branch from fdba064 to d1afc59 Compare October 15, 2026 00:08
@MarcZX10

Copy link
Copy Markdown
Contributor Author

You were right: I accidentally pushed the unrelated diagnostics commit to this branch. I restored the intended rebased trace work; the current head is d1afc595e, and GitHub now shows the expected seven AGC draw-cache files. Sorry for the confusion.

I also corrected the PR description: this version uses the existing APS5_TRACE_DRAW_CACHE report and does not add a second trace switch or stderr path. The focused target builds and conventions pass. The CTest executable is blocked before its test body on this Xeon E5-1620 v2 due to missing LZCNT, BMI1, BMI2, MOVBE, AVX2, and FMA; I have updated the validation text to say that accurately.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 15, 2026
@MarcZX10
MarcZX10 force-pushed the marc/trace-draw-key-misses branch from d1afc59 to 4f142b1 Compare October 15, 2026 00:47
@MarcZX10

Copy link
Copy Markdown
Contributor Author

Rebased onto current main and pushed as 4f142b189. The update reuses traceDrawCache() / APS5_TRACE_DRAW_CACHE and the existing 10-second [draw-cache] report. It adds the stable-register, user-data, auxiliary-register, device, and shader-set attribution (plus an unchanged-component counter) beyond the existing new-register vs. user-word/pointer-only classification. The testShaderSetKey regression remains. Local conventions and git diff --check upstream/main...HEAD pass. I could not execute the focused test body on my Xeon E5-1620 v2 due to unsupported required CPU features; the previously reported supported-host run of the component test remains relevant.

@fma-rc

fma-rc commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

I built 4f142b1 on main 8cd937e and ran agc_driver_register_key, agc_driver_draw_cache_reuse, agc_driver_draw_cache_reuse_off and agc_driver_draw_cache_reuse_verify: all 4 pass on an RTX 5080, on RADV (Radeon 610M) and on lavapipe.

I also ran Astro's Playroom (PPSA01325) into its heavy gameplay scene with APS5_TRACE_DRAW_CACHE=1 for 264 s with no throw. Three consecutive 10 s reports:

lookups hits no entry stable registers user data auxiliary unchanged
241848 11032 230226 4608 230226 1415 0
241706 9042 232404 4354 232404 1659 0
238983 8592 230249 4154 230249 1599 0

So every register-key miss there has changed user data, and only about 2% also changed stable registers. That is what the data-only path work in PR 5656 and PR 5609 goes after, and this report makes it easy to see.

@MarcZX10

Copy link
Copy Markdown
Contributor Author

I believe the requested updates are now present in head 4f142b189: it builds on current main, reuses traceDrawCache() / APS5_TRACE_DRAW_CACHE and the existing report, and adds the per-component stable-register, user-data, auxiliary-register, device and shader-set attribution beyond the existing register/user-data split. The component-slot collision is fixed and the distinct shader-set regression remains.

For validation, fma-rc reports agc_driver_register_key, agc_driver_draw_cache_reuse, agc_driver_draw_cache_reuse_off and agc_driver_draw_cache_reuse_verify passing on RTX 5080, RADV Radeon 610M and lavapipe, plus a 264-second Astro's Playroom trace without throws. The hosted eligibility, conventions and report checks pass; hardware/path-filtered jobs skip. I still cannot enter the local test body on this Xeon because of the host CPU guard. Could you re-review the current head against your requested changes?

@adriadam10
adriadam10 force-pushed the marc/trace-draw-key-misses branch from 4f142b1 to 63c46c6 Compare October 15, 2026 11:08

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked the rebased head: it now reuses traceDrawCache() and the existing 10 s [draw-cache] report, components[4] carries the shader-set key and components[3] the device serial. agc_driver_register_key and the four agc_driver_draw_cache_reuse* tests pass with APS5_VERIFY_REGISTER_KEYS=1, and the new test does not build without the change. Default behaviour is unchanged with the variable unset. Confirmed by @fma-rc on RTX 5080, RADV and lavapipe.

Added on top as a small commit: dropped the unused identity temporary in drawRegisterKey (DrawRegisterKey.cpp) and a stray blank line. Merging once the full Build is green.

@adriadam10
adriadam10 merged commit 6ea62f9 into boykopovar:main Oct 15, 2026
8 checks passed
adriadam10 pushed a commit that referenced this pull request Oct 15, 2026
Since #5364, Draw/DrawRegisterKey.hpp includes Shaders/ShaderRegistry.hpp,
which reaches ShaderMemory.hpp and its "Recompiler.hpp". tests/Pm4.cpp
includes DrawRegisterKey.hpp, but agc_driver_pm4_tests only links
libSceAgcDriver, libc and libkernel, so the recompiler's include
directories are not on its path and Pm4.cpp no longer compiles. The test
targets that already reach ShaderMemory.hpp link shader_recompiler_agc_driver;
this one now does too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants