Repository navigation
perf(agc): grow the resource cache while the keys it evicts come back - #7189
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XBXM4pgkMHYEtqN1EQni2P
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
AMD data point, since the description lists NVIDIA: head
Not checked: other tests, titles, timings. AI-assisted: yes (Claude Code). |
|
Thanks for this. Review and full CI pass on c652880, and our games gate shows no regressions, but the cache growth needs one change before we merge. In Astro Bot the cache grows from 1024 to 8192 entries, and peak VRAM goes from 13.2-13.7 GB on main to 14.6 GB with this PR (about +1.0-1.4 GB). The bound only ever grows, not even Could you let the bound come back down? For example, reset it to the base size on If you can also post the miss/return counters from GTA V before and after (the 47k -> 120 per 10 s numbers), with the VRAM figure from the same run, that would help us weigh the trade-off. |
|
Thanks for the Astro Bot numbers, that's a fair point. I'll let the bound come back down: reset to the base size on AI-assisted (Claude Code). |
Halve the grown bound (not below 1024 entries) once a window of the bound's uses times a shrink window was served by at most a quarter of it, and reset the bound on Clear(). The shrink window starts at 8 and doubles (up to 1024) each time growth undoes the last shrink, so a set of keys that comes back over a longer span stops the bound from oscillating. The entries over the bound leave at the next insert, with the other evictions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XBXM4pgkMHYEtqN1EQni2P
|
Done in bea4be8: the bound now comes back down.
Test (
Run on c652880, the test fails at step 2 ("64 sets served 40960 dispatches, but the bound stayed at 2048 entries"). Without the doubling, it fails at step 5. GTA V, open world. Build: main + #6488 + #6848, same save, one run each, last 12 windows of 10 s. VRAM is the whole GPU's
The base misses are lower than the 47k in the PR body because the respawn point changes the scene, and misses vary a lot between runs. The committer figures carry the same noise. With the backoff, GTA settles after two shrinks, at 8192 entries (it used to reach about 9.7k at the 16384 cap). In this title the larger cache costs no measurable VRAM: the peak is within run-to-run noise of main. AI-assisted (Claude Code). |
|
Thanks for adding the shrink. The bound now comes back down as asked, and the review of bea4be8 found no problems in the cache code. One thing blocks CI: the new Could you make the test cheaper rather than raising the timeout? One option is a test-only way to set a smaller base bound and window, so the grow, shrink and regrow steps take a few thousand dispatches instead of ~100k. It should still fail without the doubling and without the |
The test now sets a base bound of 32 entries and a first shrink window of 2 (ResourceCache::SetBase, applied at the next Clear()) and turns dispatch recipes off, so every dispatch goes through the cache: grow, shrink, regrow, backoff and the Clear() reset take about 1.2k dispatches instead of about 100k. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XBXM4pgkMHYEtqN1EQni2P
|
Thanks. f27ad68 makes the test short without touching the timeout.
Every step runs in about 1.2k dispatches in total (it was about 100k): grow, shrink, trim at the next insert, regrow, the backoff, then Each control still fails at its own check:
AI-assisted (Claude Code). |
The graphics pipeline store kept a fixed bound of 1024 entries, and the journal replayed only its first 1024 records, which the replay then skipped once the store was full. A title with more pipelines rebuilt the rest at the draw under the GPU mutex, about 200 ms each. In GTA V's open world that is 80 to 150 rebuilds in about six minutes of play; with this change, 42 (pipelines no earlier run recorded). - The journal replays up to 16384 records (APS5_PIPELINE_JOURNAL_ENTRIES still sets it), and a replayed pipeline that finds the store full doubles the bound instead of being skipped, up to 16384. - During play, the store keeps the hashes of the pipelines it evicted, as the resource cache does (boykopovar#7189). Every bound/32 evictions, if more than half came back, the bound doubles, up to 16384. - The bound never shrinks during a run: a pipeline is cheap to keep and costly to rebuild. A device that clears its pipelines resets it, and APS5_PIPELINE_STORE_ENTRIES still fixes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017EjjFU1FjMR11oWDYs8UMo
The graphics pipeline store kept a fixed bound of 1024 entries, and the journal replayed only its first 1024 records, which the replay then skipped once the store was full. A title with more pipelines rebuilt the rest at the draw under the GPU mutex, about 200 ms each. In GTA V's open world that is 80 to 150 rebuilds in about six minutes of play; with this change, 42 (pipelines no earlier run recorded). - The journal replays up to 16384 records (APS5_PIPELINE_JOURNAL_ENTRIES still sets it), and a replayed pipeline that finds the store full doubles the bound instead of being skipped, up to 16384. - During play, the store keeps the hashes of the pipelines it evicted, as the resource cache does (boykopovar#7189). Every bound/32 evictions, if more than half came back, the bound doubles, up to 16384. - The bound never shrinks during a run: a pipeline is cheap to keep and costly to rebuild. A device that clears its pipelines resets it, and APS5_PIPELINE_STORE_ENTRIES still fixes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017EjjFU1FjMR11oWDYs8UMo
What
The resource cache (
ResourceCache, the builtShaderResourcesthat a repeated dispatch or draw binds again) has a fixed bound of 1024 entries. A title that binds more distinct resource sets per frame than that evicts each set before the next frame needs it again, so every frame rebuilds every set.The bound now grows when that happens:
[rescache] capacity A -> B entriesline is printed each time.The bound also comes back down. Once grown, the cache counts the distinct keys over a window of
bound × shrink windowuses. When at most a quarter of the bound served that window, the bound halves, down to 1024 at least, and the extra entries leave at the next insert along with the other evictions. The shrink window starts at 8 and doubles, up to 1024, each time growth undoes the last shrink, so the bound stops swinging when keys come back over a longer span.Clear()resets the bound to 1024.APS5_RESOURCE_CACHE_ENTRIESstill sets a fixed bound, with no growth. The table-element budget (APS5_RESOURCE_CACHE_TABLE_ELEMENTS) is unchanged, and so are the evictions it makes.In GTA V (PPSA04264) in the open world, about 2,400 draws per frame bind about 9,700 distinct sets. On
main, the draw path missed the cache 47,236 times per 10 s; the misses repeat keys the cache had evicted ([rescache] miss churn: "same key" 98,736). The draw committer spent about 10 % of its samples building these sets again (ShaderResources::ShaderResources).Test:
agc_driver_resource_cache_growth(tests/execution/ResourceCacheGrowth.cpp). It sets a base bound of 32 entries and a first shrink window of 2 (ResourceCache::SetBase, so the test stays near 1.2k dispatches) and turns dispatch recipes off (APS5_NO_DISPATCH_RECIPE), so every dispatch goes through the cache. It runs 48 compute dispatches, each with its own buffer, four times in turn, and checks that each dispatch stored its value and that the cache then holds all 48 sets. It then checks that two hot sets bring the bound back down, that the next insert trims the cache to it, that the large set grows it again, that the same hot pass then leaves it there (the backoff), and thatClear()resets it to the base. Without the growth it fails: "the resource cache kept only 32 entries (initial bound 32), so every pass rebuilt every set". Without the shrink, the backoff doubling or theClear()reset, it fails at the matching check.Related: #6972 (index the resource cache by a four-lane key hash) rewrites the same class's private members for a different purpose, so the two conflict textually. Whichever merges second needs a short rebase; the growth only needs the evicted keys' hashes, which #6972 already stores in each entry. #6492 (drop entries that hold released sampled textures) adds a member next to the same lines; only those lines need merging by hand.
Tested
Windows 11 Pro (26200), RTX 4090 (NVIDIA 616.92), i9-13900K, MinGW GCC 15.2.0:
agc_driver_resource_cache_growth: passes in about 0.5 s; with the growth, the shrink, the backoff doubling or theClear()reset removed, it fails (messages above).ctest -R "agc_driver|agc_masked|agc_shader"on this branch (f27ad68): 524/524 pass, 2 skipped. The GTA build below uses the same change on a base that also carries feat(recompiler): run waterfall loops over per-lane T#s through image tables #6488 and perf(agc): clear single-key DCC surfaces on a verified uniform color #6848, which GTA needs to render.APS5_PROFILE_DRAW=1, last 12 windows of 10 s:[rescache] drawsmisses / 10 sThe bound grew 1024 -> 2048 -> 4096 -> 8192 -> 16384, and the cache settled at 9,708 entries. Process RAM was the same in both runs (25 GB).
APS5_TRACE_SECONDS=240:60, same scene and session, no otherAPS5_TRACE_*/PROFILE_*variables:Traces: base, this PR. Both commits are the base above plus two local bench hooks that do nothing unless their variable is set. Read these numbers with care: the frame rate in this scene depends on where the save respawns the player, both runs were still compiling some shaders (
Driver.Draw.shader_compile_and_link106 and 55 ms per frame), and queue worker 0's trace buffer filled after about 4 s in both runs (trace_dropped). The committer figures above are the steadier comparison.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code
https://claude.ai/code/session_01XBXM4pgkMHYEtqN1EQni2P