Repository navigation
Conversation
11 tasks done
github-actions
Bot
added
perf
|
This pull request is labeled |
11 tasks done
Contributor
|
@Raghav314 thanks for the clear notes on what was dropped and what #9508 keeps. We'll measure #9508 on Resident Evil Village (PPSA01556) on our RTX 3070 Laptop GPU, whose driver (617.42) exposes |
github-actions
Bot
removed
the
perf numbers missing
EagleBearerYS
added a commit
to EagleBearerYS/AnyPS5
that referenced
this pull request
Oct 22, 2026
Contributor
|
One checklist detail appears inconsistent with the repository rule for |
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
MarcZX10
suggested changes
Oct 22, 2026
MarcZX10
left a comment
MarcZX10
left a comment
Contributor
There was a problem hiding this comment.
The updated body now satisfies the performance-report format with same-scene summary lines, trace attachments and digests; I withdraw that earlier concern. I am requesting changes for a separate correctness gap in the non-heap fallback. In core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp, shader.heap is nullable and heapStage is set when any stage is in GPU mode. In Complete()/build(), the if (heapStage) RefreshHeapStage(shaders); call applies to the entire shader span. RefreshHeapStage (Execution/src/Driver/Draw/HeapStages.cpp) requires every shader stage to be GPU mode and to have a heap-backed descriptor plan. But this PR’s stated fallback explicitly allows a stage whose module cannot map onto the heap, or a mixed draw, to use the existing per-draw path. A graphics pipeline with one heap-eligible stage and one ineligible stage therefore appears to take the documented mixed/per-draw fallback and then hit a Require while refreshing all stages. Please either make refresh handle the mixed set by skipping non-heap stages (and keep their per-draw resources valid), or route the entire draw through the per-draw path before heapStage is marked. Add a regression with mixed eligible/ineligible stages on an extension-capable device, or a test of the selection/refresh helper. Hosted relinker matrix and ASan, eligibility/conventions/intake/report checks pass; product Linux/Windows jobs are skipped. The Windows runtime and title measurements in the body are author-reported. I did not run local GPU tests.
Contributor
Author
|
@MarcZX10 thanks for looking. LLM assistance used. |
MarcZX10
suggested changes
Oct 22, 2026
MarcZX10
left a comment
MarcZX10
left a comment
Contributor
There was a problem hiding this comment.
The author has added the missing mixed-stage regression and clarified the path it exercises. I reviewed that update and withdraw my earlier mixed-stage correctness concern: the test now covers the heap-eligible vertex plus an unmappable fragment, verifies image output, checks one retry to per-draw, and checks the repeated pair remains per-draw.
I am keeping the review at changes requested for the new hosted Windows relinker result. That run completed 232/233 tests and failed only conventions_function_bodies_shard_11 by timeout. Since the other 232 passed and the failure is a timeout, it may be a runner-load issue, but the check is currently red; please rerun it or provide a green Windows matrix result before merge. The other hosted relinker jobs (Ubuntu, macOS arm64/x86_64 and ASAN), conventions, eligibility, labels and report checks pass. Linux/Windows product jobs are skipped. The Hades II trace evidence and Vulkan validation comparison are author-reported; I did not run local GPU tests.
Contributor
|
I reviewed the new mixed-stage fallback regression and withdraw my earlier correctness concern; it now covers the eligible/unmappable stage pair and per-draw retry. The latest Windows relinker run is still red: 232/233 tests passed, with |
EagleBearerYS
added a commit
to EagleBearerYS/AnyPS5
that referenced
this pull request
Oct 22, 2026
Rebuilds boykopovar#7642's device columns inside boykopovar#9508's GPU-mode stages. The recompiler takes SpirvTarget::deviceImageTables (part of the recompile cache key). ResourceTracker plans a V# table column on the device table only in a GPU-mode stage that reads its table map by address (gpuResolvedImages and descriptorMemory, vertex, pixel or compute, Int64 and buffer device addresses), the reverse of boykopovar#7642's gate: plain column, sampled reads, gathers, queries and loads only, a direct S# that is not forced unnormalized, no depth compare, packed or by-element access. ImageResource::deviceTable marks such an image, and the result's deviceImageTables tells the driver. A device column needs no table snapshot, capture memory or map records: the snapshot and ResolveImageTables skip it, so its map part stays empty, and a stage whose only tables are device tables still gets its table map, which carries the trailer. Each device-table image's interface is registered when a result is materialized, and a direct S# paired with one gets its sampler class as specialization constant DeviceSamplerClassBase plus the sampler index. The access calls device_image_lookup with the T# words, the interface id, the sampler class and the pc. It takes the bucket and miss ring chunk addresses from the four trailer words before HeaderElementBase and reads both through buffer device addresses; a map without a table (zero addresses) misses and records nothing. A hit samples the heap table array of the mode's kind at the bucket's slot, which aliases the GPU heap's first sampled binding, so the device table now takes its slots in that kind. A miss or a faulted bucket reads zeros and appends one record per key and batch to the ring; the driver resolves it for the next batch, and a program the table cannot serve is recompiled on boykopovar#9508's CPU map. writeHeapTableMap prepares the device table for the open batch and writes its trailer for a stage that reads it. The shader disk cache encodes the new fields; its format moves to 48. agc_shader_device_image_tables compiles a compute and a pixel program reading a 2,000-entry V# table at a user-data index with a direct S#: in a GPU-mode stage the column reads the device table, the capture reads no record, the map has no record for it, the module calls the lookup keyed by the registered interface and the S#'s class, reads all four trailer words, declares no set 2 or 3, aliases the first sampled heap binding and passes spirv-val; with the option off, or on the per-draw path, the column stays on the CPU map, as do a storage write, a depth compare, a sampler table and an unnormalized S#. Based on fable-void's d35b01a and 81119ee (wip/new-gpu-path).
Raghav314
force-pushed
the
perf/agc-vk-descriptor-heap
branch
from
October 22, 2026 01:57
fb07b25 to
d52c109
Compare
Contributor
|
The new performance evidence in the PR still does not meet the repository report rule for a |
MarcZX10
suggested changes
Oct 22, 2026
MarcZX10
left a comment
MarcZX10
left a comment
Contributor
There was a problem hiding this comment.
The new mixed-stage fallback test resolves my previous code-path concern, and the Linux test report is useful. I am requesting changes for the repository-required evidence only: this perf PR needs compliant same-scene/same-machine warm-cache summary lines, attached base/change traces and tools/trace_digest.py output, and the AGC main/change focused validation-layer comparison. Please add those artifacts or remove the performance claim/perf labeling. I did not run local tests; hosted Linux/Windows product jobs were skipped.
This was referenced Oct 22, 2026
Raghav314
force-pushed
the
perf/agc-vk-descriptor-heap
branch
from
October 22, 2026 02:44
d52c109 to
23b2a3f
Compare
EagleBearerYS
added a commit
to EagleBearerYS/AnyPS5
that referenced
this pull request
Oct 22, 2026
Rebuilds boykopovar#7642's device columns inside boykopovar#9508's GPU-mode stages. The recompiler takes SpirvTarget::deviceImageTables (part of the recompile cache key). ResourceTracker plans a V# table column on the device table only in a GPU-mode stage that reads its table map by address (gpuResolvedImages and descriptorMemory, vertex, pixel or compute, Int64 and buffer device addresses), the reverse of boykopovar#7642's gate: plain column, sampled reads, gathers, queries and loads only, a direct S# that is not forced unnormalized, no depth compare, packed or by-element access. ImageResource::deviceTable marks such an image, and the result's deviceImageTables tells the driver. A device column needs no table snapshot, capture memory or map records: the snapshot and ResolveImageTables skip it, so its map part stays empty, and a stage whose only tables are device tables still gets its table map, which carries the trailer. Each device-table image's interface is registered when a result is materialized, and a direct S# paired with one gets its sampler class as specialization constant DeviceSamplerClassBase plus the sampler index. The access calls device_image_lookup with the T# words, the interface id, the sampler class and the pc. It takes the bucket and miss ring chunk addresses from the four trailer words before HeaderElementBase and reads both through buffer device addresses; a map without a table (zero addresses) misses and records nothing. A hit samples the heap table array of the mode's kind at the bucket's slot, which aliases the GPU heap's first sampled binding, so the device table now takes its slots in that kind. A miss or a faulted bucket reads zeros and appends one record per key and batch to the ring; the driver resolves it for the next batch, and a program the table cannot serve is recompiled on boykopovar#9508's CPU map. writeHeapTableMap prepares the device table for the open batch and writes its trailer for a stage that reads it. The shader disk cache encodes the new fields; its format moves to 48. agc_shader_device_image_tables compiles a compute and a pixel program reading a 2,000-entry V# table at a user-data index with a direct S#: in a GPU-mode stage the column reads the device table, the capture reads no record, the map has no record for it, the module calls the lookup keyed by the registered interface and the S#'s class, reads all four trailer words, declares no set 2 or 3, aliases the first sampled heap binding and passes spirv-val; with the option off, or on the per-draw path, the column stays on the CPU map, as do a storage write, a depth compare, a sampler table and an unnormalized S#. Based on fable-void's d35b01a and 81119ee (wip/new-gpu-path).
Contributor
Author
|
@MarcZX10 thanks, updated the description. Your points on warm caches,
What changed:
LLM assistance used. |
12 tasks done
MarcZX10
suggested changes
Oct 22, 2026
MarcZX10
left a comment
MarcZX10
left a comment
Contributor
There was a problem hiding this comment.
Correction to my 02:04 review: I rechecked the current body against the performance form and the author's 02:54 response. I withdraw my claim that the required summary lines, warm-cache capture details, trace digests, or AGC validation-layer comparison are missing. The body now reports the same-machine comparison and 54 matching base tests / 134 change tests; the layer is found in both logs with zero VUID errors. Hosted relinker jobs also now pass, resolving the earlier Windows timeout.
Two items still need correction before merge:
- The Performance report links appear swapped: the main bullet names
aps5-trace-de49f41349a9-624652.json.gzbut links to the568369d27a2e-625697heap capture; the heap bullet names568369d27a2e-625697but links to thede49f41349a9-624652main capture. Please correct the attachments/links so each scenario points to its own trace. The baseline trace also reportstrace_dropped: true; please keep that limitation explicit when describing its trace detail. - The source and all new validation evidence are still based on main
a6dcea9279(plus the declared #6946/#6300 overlays), while current upstream main isc61d20f54801efc3798555829f6fedcff71ad7ff. Please update the branch and rerun conventions/overlap and the focused validation comparison on the updated tree. For the performance comparison, retain the #6300 integration needed to reach the same Hades II scene and identify the exact current-main integration base in the report.
The mixed-stage fallback regression and the latest hosted relinker matrix resolve my earlier code-path and timeout concerns. I did not run local GPU tests.
12 tasks done
Contributor
|
@Raghav314, an overlap note from #9503 (mine), which the reviewer asked me to coordinate with you. Both PRs change the same
The two changes don't contradict each other. Whichever lands second keeps both: your wrapped call, placed inside the lambda after #9503 is a one-line fix that has already been validated on two AMD drivers, so I'd suggest it goes first. Your rebase would then be just moving the wrapped call into the lambda. If you'd rather land first, I'll rebase #9503 onto yours instead. Either order works for me; tell me which. |
Contributor
|
@Raghav314 a merge-order note for two small drafts next to this PR:
Neither side needs to wait, and the rebase is ours either way: if this PR merges first, #9563 and #9564 rebase onto it, and if they merge first, I'll put the resolved merge on a branch of my fork for you to take. Does that work for you? |
buildRingWords cleared the stored ShaderData words, so a ring hit fell back to the template's words past the user-data prefix; it now copies the current user words over the stored ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rotating the two-generation memo destroyed the whole previous generation (up to 16384 keys of captures and words) on the draw thread, a 16-20 ms heap_direct stall every few seconds in MC D2. Each insert now frees two old entries, so the old generation is empty before the next rotation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…PIPELINE_STATS Per-call time of the draw stage's preparation, heap stage call, walk memo, walk, materialize and finish on the queue worker, and of the resource template find and build, slot mapping, stage data and buffer address writes on the committer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cost line Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…its key Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An indirect dispatch with USE_THREAD_DIMENSIONS stores its thread limit in the ShaderData words at record time, which heap stages keep in a ring span, so these dispatches were kept off the heap. The limit now goes into the dispatch's ring span: the stage takes a fresh span before its data is written and drops it afterwards, so a kept span that other dispatches read never changes. The ring takes transfer writes for the copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ces too small for its slots The sampler heap held 256 samplers, an eighth of what D3D12 allows, so scenes with many samplers swept it early. It now holds 2048, which fits NVIDIA's sampler heap beside its embedded sampler range. A device whose resource or sampler heap cannot hold the slots no longer enables the heap and takes the per-draw path, instead of failing at the first heap use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es the per-draw path Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and buffer back in classic descriptor mode Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tor heap agc_driver_heap_draw_sample_heap, _nogpl_heap, _full_heap and _full_heap_pipelined, and agc_driver_pipeline_journal_gpu_heap set APS5_GPU_HEAP and then require the heap to take the draw's stages, to refuse a capped heap or to put a compute stage in GPU mode. On a device where DescriptorHeapEnabled() stays false (no VK_EXT_descriptor_heap, or one without maintenance5, room for the heap's slots or enough push data) none of that can happen, and they failed instead of skipping. They now exit 77 with a "skipped, the device lacks VK_EXT_descriptor_heap, ..." line there, and run as before on a device with the heap. (cherry picked from commit 0a01340)
A run with APS5_GPU_HEAP recorded its pipelines in the same journal as a run without it, so each mode replayed the other's records: those failed at the pipeline layout check, and they took places in the replay limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the descriptor heap on, buffers are reached through buffer addresses, which have no Coherent buffer array and no 64-bit view. GLC loads and stores became plain 32-bit accesses, so a compute shader that polls a queue slot another workgroup writes never saw the write and the GPU hung (RoboCop: Rogue City at the splash, Xid 109). GLC dword accesses through a buffer address are now device-scope atomic loads and stores, and GLC two-dword accesses use one 64-bit atomic as the storage-buffer path does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r cache invalidation A program with GLC or DLC accesses that invalidates the vector caches reads the whole guest buffer array as Coherent on the storage-buffer path, so its loads without GLC see other work-groups' writes after the invalidation. Through buffer addresses those loads and stores stayed plain. They are now device-scope atomic loads and stores in such programs, as GLC accesses already are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… on Windows MinGW declares no setenv, so the heap draw sample did not build on Windows; it now sets its environment with _putenv_s there, as the other tests that set a variable before opening the device do. Those tests and agc_storage_refresh_proofs also exited with 0xc0000135 on Windows without libc's directory on PATH, which they now get like the rest of the driver's tests.
…bgroups
The atomics kernel built its table key with v_readfirstlane_b32 of 5 plus the lane index, which is 5 for the whole wave64 only when the host runs the wave as one subgroup. lavapipe runs it as 8-lane subgroups and each takes its own first lane, so lanes 8 to 63 read records 13 and up ("dispatch 0 texel 8 returned 13059, expected 5059"). The key now adds the lane index shifted right by 6, which is 0 in every lane of the wave and still a value the planner can't evaluate.
DeviceBuffer returns its allocation to the pool without a device address. With the GPU heap's forced addresses, a texture's DeviceBuffer and a device-local Buffer with SHADER_DEVICE_ADDRESS share one pool key, so a Buffer of the same size took that allocation and kept the address 0. Its first DeviceAddress() threw "buffer has no device address" from a compute dispatch's BDA table. Query the address when a reused allocation has none.
A pixel stage whose LDS lives in the pixel function LDS buffer used its own descriptor set, which heap pipelines cannot bind, so the stage was refused and the whole draw fell back to per-draw descriptor sets. The resource heap now keeps one storage buffer descriptor after the image slots. The pool buffer gets a device address when the device has the heap, and its descriptor is written into that slot the first time a heap draw prepares it. The pixel LDS set maps onto the slot with a constant offset, so heap pipelines no longer bind the set. Tested: agc_driver_heap_draw_sample* draws a texture through LDS at dword 4352 (past the 4096-dword private storage, so it uses the pool) with and without the heap. With the mapping disabled the heap run refuses "the heap did not take the fragment stage". Full ctest -j4 -E agc_driver_bda_device: only the three sample_order tests fail, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No behaviour change. Only code this branch adds is touched; functions that exist on main keep their shape. - BindlessHeap: the texture, storage and view slot requests share one take-or-sweep, claim and free path; image and sampler descriptors are written directly instead of through a VkWriteDescriptorSet that was unpacked again; heapBindings returns one result struct instead of five out parameters, and the heap mappings are built by two helpers. - MaterializeHeapStage: the memo lookup, the walk's reads, the settle check (written twice before), the guards and the memo insert are separate functions. - ShaderResources::writeHeapStage: slot key, slot currency, slot taking, generations, verification and slot word patching are separate members; the table map and slot code share one placeholder view helper. - Statistics lines, the compute heap pipeline call and the heap draw sample's shader and message selection no longer sit in one line each. - The files this branch adds, and its heap functions in ShaderResources, are wrapped at 140 columns with clang-format (4-space indent, attached braces, short ifs and loops kept on one line, includes not sorted, string literals not split). Tested: full ctest -j4 -E agc_driver_bda_device passes as before (only the three sample_order tests fail); agc_driver_heap_draw_sample with and without APS5_GPU_HEAP under VK_LAYER_KHRONOS_validation reports no VUID. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…wn program A draw-resource-cache hit reuses a build whose stages may belong to another program with the same bindings. writeHeapTableMap found its table by the build's program pointer: a different program skipped the map refresh, and a new program at a freed one's address matched and read the freed program's image table binding (Resident Evil Village PPSA01556 crashed on its atomic flags). The table is now found by its stage, and the image table binding comes from this draw's program, the last ImageTable or ImageStorageTable binding as the build picks it. The build's program and binding pointers are cleared once the build finishes, so no table keeps a pointer into a program it does not own. The test builds a heap set, destroys its program and refreshes the set for an equal-content program, once at another address and once at the destroyed program's address with its freed binding memory overwritten. Without the fix the first leaves the table map unset and the second crashes. Co-authored-by: Yunus Shaikh <75296019+EagleBearerYS@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agc_driver_heap_compute_sample_heap and its pipelined variant could not load libc on Windows (0xc0000135); the draw heap variants already prepend libc's directory to PATH. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t run The sample took its expected mode from DescriptorHeapEnabled(), so with APS5_GPU_HEAP set on a device without the heap it passed by testing the plain path. It now takes the request from APS5_GPU_HEAP and exits 77 with a reason when the heap is requested but unavailable, as the draw sample does; on a capable device the heap assertions stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows a module thread_local's destructor runs after emulated TLS has already freed the object (prx_thread_local_destructors). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raghav314
force-pushed
the
perf/agc-vk-descriptor-heap
branch
from
October 24, 2026 15:53
ba0cfa6 to
b175572
Compare
Contributor
Author
|
@oneandonlydean Moved the heap's swept-storage thread_local into HostThreadLocal (b175572). All five registrations came from that one object; the PR's other new thread_locals are trivially destructible. I also rebased onto main 3fcb5e4. Main's 2181bfc removed |
12 tasks done
12 tasks done
Collaborator
|
@Raghav314 Thanks for the quick thread_local fix in b175572; our delta review passes it on the source side, and the Windows run comes next. One request on size before we go further: at +7,215/-779 across 122 files, this is a lot to review, gate and land as one change, and every fix round means re-reviewing the whole thing. Could you split it into smaller PRs that land in order? Roughly:
Exactly where the lines fall is your call; what matters is that each piece builds, passes CI and can be reviewed on its own. Smaller pieces will merge a lot faster than one large PR, and the review work already done carries over. |
Contributor
Author
|
@oneandonlydean Thanks. I'd prefer to keep this as one PR. The heap is opt-in behind APS5_GPU_HEAP and off by default, so it changes nothing unless the switch is set. The pieces you describe aren't independent, though. The recompiler's GPU mode and the driver's heap stages are one contract:
Neither side can be checked without the other. A recompiler-only PR would be codegen with no consumer, and a driver-only PR would be heap plumbing with no stage that uses it. The tests that matter are end to end for the same reason: the heap draw and compute samples, address passes, table stage reuse, and RoboCop's hang case. Those tests are what found the real bugs:
Split into a stack, the early PRs would land without the tests that exercise them, and each later piece would reopen the earlier ones whenever a fix crossed the boundary, as these fixes did. The source review is already done on this head, so another full round over a reshuffled stack would cost more than landing it as is. If you still want it split after the Windows run, I'll do it. |
12 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
An opt-in GPU descriptor heap,
APS5_GPU_HEAP=1, built onVK_EXT_descriptor_heap. Stages that can use it stop rebuilding and binding descriptor sets on the CPU for every draw and dispatch. It builds on the draw pipeline (#6946, merged) and replaces #6947, which is closed.APS5_HEAP_SLOT_CAP) and one sampler heap (2048 slots, D3D12's limit). Per-draw shader data, the flattened SRT and the image table map go into a ring in device memory, and the shaders reach them through pushed addresses. A heap draw binds no descriptor set.gpuResolvedImages). Its image metadata names the binding element, and the driver writes the resolved heap slots into the stage's shader data. Guest buffers are read by device address (addressedBuffers), not through storage buffer descriptors. A stage whose module doesn't map onto the heap, or a draw whose stages come out mixed, is rebuilt on the per-draw path.VK_EXT_descriptor_heap. Disk cache format 48 (main took 46 and 47 meanwhile).APS5_VERIFY_GPU_HEAP=1checks heap slots against the bound textures, rebuilds 1 in N GPU-mode stages on the per-draw path and compares them, and fails when a reused stage data span would differ ([heap-slots],[gpu-heap]).PPSA36082 (Hades II)
Title and scene: PPSA36082 (Hades II), the hub (Crossroads), player standing still,
APS5_TRACE_SECONDS=120:10. Caches warm: each setting was run once into the hub and the title restarted before its capture. No otherAPS5_TRACE_*orAPS5_PROFILE_*switch, log not redirected (the title ran in a terminal). All three withAPS5_PIPELINED_DRAWS=32 APS5_PIPELINE_DISPATCH=1and nothing else changed.Current
mainstops Hades II while the hub loads (native multisample depth metadata fill transition is not implemented), so every build here has #6300 merged locally. Themainbuild is a6dcea9 + #6300 (local merge commit de49f41349); this PR's is 23b2a3f + #6300 (local merge commit 568369d27a), the PR as of the descriptor heap rebind fix onmaina6dcea9. Since then the PR's commits changed only by test-only commits and rebases onto newermain; the numbers were not captured again on the current head.maina6dcea9 + #6300APS5_DRAW_PIPELINE=1APS5_DRAW_PIPELINE=1 APS5_GPU_HEAP=1The heap is +14% over the draw pipeline alone and +44% over
main.Summary lines
maina6dcea9 + #6300{"commit":"de49f41349a9","dirty":false,"env":{"APS5_PIPELINED_DRAWS":"32","APS5_PIPELINE_DISPATCH":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"120:10"},"gpu":"NVIDIA GeForce RTX 4090","driver":"NVIDIA 615.78.08","os":"Linux 7.2.10-1-cachyos","cpu":"AMD Ryzen 7 7800X3D 8-Core Processor","cpu_threads":16,"memory_mib":31743,"title_id":"PPSA36082","title":"Hades II","content_version":"01.006.000","resolution":"1920x1080","window":{"spec":"seconds 120:10","first_present":8859,"ms":10001.243},"presents":706,"frame_ms":{"p50":13.955,"p90":15.665,"p99":18.680,"max":25.064},"gpu_busy_ms_per_present":2.224,"worker_busy_pct":{"queue worker 0x0":97.766},"top_phases_ms_per_frame":{"Driver.Packet":13.844,"Driver.Packet.draw":11.959,"Driver.Draw":11.850,"Driver.Draw.shader_compile_and_link":8.284,"Graphics.Draw":8.166,"Graphics.Draw.draw_and_resource_release":6.859,"Shader.DrawStage":5.206,"Shader.DrawStage.capture_resources":3.196},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":6.816,"events_lost":132221,"totals_full":false}},"trace":"aps5-trace-de49f41349a9-624652.json.gz"}this PR + #6300,
APS5_DRAW_PIPELINE=1{"commit":"568369d27a2e","dirty":false,"env":{"APS5_DRAW_PIPELINE":"1","APS5_PIPELINED_DRAWS":"32","APS5_PIPELINE_DISPATCH":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"120:10"},"gpu":"NVIDIA GeForce RTX 4090","driver":"NVIDIA 615.78.08","os":"Linux 7.2.10-1-cachyos","cpu":"AMD Ryzen 7 7800X3D 8-Core Processor","cpu_threads":16,"memory_mib":31743,"title_id":"PPSA36082","title":"Hades II","content_version":"01.006.000","resolution":"1920x1080","window":{"spec":"seconds 120:10","first_present":10851,"ms":10010.638},"presents":886,"frame_ms":{"p50":11.172,"p90":12.083,"p99":13.652,"max":41.180},"gpu_busy_ms_per_present":2.340,"worker_busy_pct":{"queue worker 0x0":96.015},"top_phases_ms_per_frame":{"Driver.Packet":10.838,"Driver.Packet.draw":8.115,"Graphics.Draw":8.087,"Driver.Draw":8.030,"Driver.DrawFinish":7.151,"Graphics.Draw.draw_and_resource_release":6.678,"Driver.Draw.prepare":4.793,"Driver.DrawFinish.shader_compile_and_link":4.199},"trace_dropped":false,"trace":"aps5-trace-568369d27a2e-626697.json.gz"}this PR + #6300,
APS5_DRAW_PIPELINE=1 APS5_GPU_HEAP=1{"commit":"568369d27a2e","dirty":false,"env":{"APS5_DRAW_PIPELINE":"1","APS5_GPU_HEAP":"1","APS5_PIPELINED_DRAWS":"32","APS5_PIPELINE_DISPATCH":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"120:10"},"gpu":"NVIDIA GeForce RTX 4090","driver":"NVIDIA 615.78.08","os":"Linux 7.2.10-1-cachyos","cpu":"AMD Ryzen 7 7800X3D 8-Core Processor","cpu_threads":16,"memory_mib":31743,"title_id":"PPSA36082","title":"Hades II","content_version":"01.006.000","resolution":"1920x1080","window":{"spec":"seconds 120:10","first_present":12138,"ms":10004.892},"presents":1014,"frame_ms":{"p50":9.743,"p90":10.718,"p99":12.061,"max":17.616},"gpu_busy_ms_per_present":2.080,"worker_busy_pct":{"queue worker 0x0":95.834},"top_phases_ms_per_frame":{"Driver.Packet":9.445,"Driver.Packet.draw":6.921,"Driver.Draw":6.847,"Graphics.Draw":6.376,"Driver.DrawFinish":6.362,"Graphics.Draw.draw_and_resource_release":5.216,"Driver.Draw.prepare":3.983,"Driver.DrawFinish.shader_compile_and_link":3.903},"trace_dropped":false,"trace":"aps5-trace-568369d27a2e-625697.json.gz"}Traces:
maina6dcea9 + feat(agc): import cleared native multisample D32S8 targets and whole HTILE fills #6300:aps5-trace-de49f41349a9-624652.json.gzaps5-trace-de49f41349a9-624652.json.gzthis PR + feat(agc): import cleared native multisample D32S8 targets and whole HTILE fills #6300,
APS5_DRAW_PIPELINE=1:aps5-trace-568369d27a2e-626697.json.gzaps5-trace-568369d27a2e-626697.json.gzthis PR + feat(agc): import cleared native multisample D32S8 targets and whole HTILE fills #6300,
APS5_DRAW_PIPELINE=1 APS5_GPU_HEAP=1:aps5-trace-568369d27a2e-625697.json.gzaps5-trace-568369d27a2e-625697.json.gzDigests (
python3 tools/trace_digest.py <trace>)`main` a6dcea9 + #6300
seconds 120:10 from present 8859: 706 presents, frame p50 14.0 / p90 15.7 / p99 18.7 / max 25.1 ms, GPU busy 2.2 ms per present
NVIDIA GeForce RTX 4090, NVIDIA 615.78.08, Linux 7.2.10-1-cachyos, 1920x1080, commit de49f41349a9, dropped: yes
{"commit":"de49f41349a9","dirty":false,"env":{"APS5_PIPELINED_DRAWS":"32","APS5_PIPELINE_DISPATCH":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"120:10"},"gpu":"NVIDIA GeForce RTX 4090","driver":"NVIDIA 615.78.08","os":"Linux 7.2.10-1-cachyos","cpu":"AMD Ryzen 7 7800X3D 8-Core Processor","cpu_threads":16,"memory_mib":31743,"title_id":"PPSA36082","title":"Hades II","content_version":"01.006.000","resolution":"1920x1080","window":{"spec":"seconds 120:10","first_present":8859,"ms":10001.243},"presents":706,"frame_ms":{"p50":13.955,"p90":15.665,"p99":18.68,"max":25.064},"gpu_busy_ms_per_present":2.224,"worker_busy_pct":{"queue worker 0x0":97.766},"top_phases_ms_per_frame":{"Driver.Packet":13.844,"Driver.Packet.draw":11.959,"Driver.Draw":11.85,"Driver.Draw.shader_compile_and_link":8.284,"Graphics.Draw":8.166,"Graphics.Draw.draw_and_resource_release":6.859,"Shader.DrawStage":5.206,"Shader.DrawStage.capture_resources":3.196},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":6.816,"events_lost":132221,"totals_full":false}},"trace":"aps5-trace-de49f41349a9-624652.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 3.6, Driver.Packet.draw 3.5, Driver.Draw 3.4, Graphics.Draw 2.5, batch 2.2, Graphics.Draw.draw_and_resource_release 1.6, Driver.Draw.shader_compile_and_link 1.4, Recorder.Sync 0.3, VideoOut.Flip 0.3, VideoOut.Flip.present 0.3
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
this PR + #6300, `APS5_DRAW_PIPELINE=1`
seconds 120:10 from present 10851: 886 presents, frame p50 11.2 / p90 12.1 / p99 13.7 / max 41.2 ms, GPU busy 2.3 ms per present
NVIDIA GeForce RTX 4090, NVIDIA 615.78.08, Linux 7.2.10-1-cachyos, 1920x1080, commit 568369d27a2e, dropped: no
{"commit":"568369d27a2e","dirty":false,"env":{"APS5_DRAW_PIPELINE":"1","APS5_PIPELINED_DRAWS":"32","APS5_PIPELINE_DISPATCH":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"120:10"},"gpu":"NVIDIA GeForce RTX 4090","driver":"NVIDIA 615.78.08","os":"Linux 7.2.10-1-cachyos","cpu":"AMD Ryzen 7 7800X3D 8-Core Processor","cpu_threads":16,"memory_mib":31743,"title_id":"PPSA36082","title":"Hades II","content_version":"01.006.000","resolution":"1920x1080","window":{"spec":"seconds 120:10","first_present":10851,"ms":10010.638},"presents":886,"frame_ms":{"p50":11.172,"p90":12.083,"p99":13.652,"max":41.18},"gpu_busy_ms_per_present":2.34,"worker_busy_pct":{"queue worker 0x0":96.015},"top_phases_ms_per_frame":{"Driver.Packet":10.838,"Driver.Packet.draw":8.115,"Graphics.Draw":8.087,"Driver.Draw":8.03,"Driver.DrawFinish":7.151,"Graphics.Draw.draw_and_resource_release":6.678,"Driver.Draw.prepare":4.793,"Driver.DrawFinish.shader_compile_and_link":4.199},"trace_dropped":false,"trace":"aps5-trace-568369d27a2e-626697.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Graphics.Draw 2.6, Driver.Packet 2.4, batch 2.3, Driver.Packet.draw 1.9, Driver.Draw 1.8, Graphics.Draw.draw_and_resource_release 1.8, Driver.DrawFinish 1.7, Driver.DrawFinish.draw_enqueued 1.0, Driver.Draw.prepare 0.9, Driver.Draw.draw_handed_off 0.5
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
this PR + #6300, `APS5_DRAW_PIPELINE=1 APS5_GPU_HEAP=1`
seconds 120:10 from present 12138: 1014 presents, frame p50 9.7 / p90 10.7 / p99 12.1 / max 17.6 ms, GPU busy 2.1 ms per present
NVIDIA GeForce RTX 4090, NVIDIA 615.78.08, Linux 7.2.10-1-cachyos, 1920x1080, commit 568369d27a2e, dropped: no
{"commit":"568369d27a2e","dirty":false,"env":{"APS5_DRAW_PIPELINE":"1","APS5_GPU_HEAP":"1","APS5_PIPELINED_DRAWS":"32","APS5_PIPELINE_DISPATCH":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"120:10"},"gpu":"NVIDIA GeForce RTX 4090","driver":"NVIDIA 615.78.08","os":"Linux 7.2.10-1-cachyos","cpu":"AMD Ryzen 7 7800X3D 8-Core Processor","cpu_threads":16,"memory_mib":31743,"title_id":"PPSA36082","title":"Hades II","content_version":"01.006.000","resolution":"1920x1080","window":{"spec":"seconds 120:10","first_present":12138,"ms":10004.892},"presents":1014,"frame_ms":{"p50":9.743,"p90":10.718,"p99":12.061,"max":17.616},"gpu_busy_ms_per_present":2.08,"worker_busy_pct":{"queue worker 0x0":95.834},"top_phases_ms_per_frame":{"Driver.Packet":9.445,"Driver.Packet.draw":6.921,"Driver.Draw":6.847,"Graphics.Draw":6.376,"Driver.DrawFinish":6.362,"Graphics.Draw.draw_and_resource_release":5.216,"Driver.Draw.prepare":3.983,"Driver.DrawFinish.shader_compile_and_link":3.903},"trace_dropped":false,"trace":"aps5-trace-568369d27a2e-625697.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): batch 2.1, Driver.Packet 2.0, Driver.Packet.draw 1.6, Driver.Draw 1.5, Driver.DrawFinish 1.4, Graphics.Draw 1.2, Graphics.Draw.draw_and_resource_release 1.0, Driver.DrawFinish.draw_enqueued 0.8, Driver.Draw.prepare 0.7, Driver.Draw.draw_handed_off 0.6
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
Where this can take more frame rate
The heap moves descriptor work off the CPU, but Hades II is still CPU-bound: the queue worker is 96–98% busy in all three captures and the GPU is busy about 2.1 ms of a 9.7 ms frame. Follow-ups that build on the heap:
Tested
Linux (CachyOS), x86-64, GCC, 7800X3D / RTX 4090, NVIDIA 615.78.08, on
mainc9b2660.agc_driver_sample_order,agc_driver_sample_order_grid_referenceandagc_driver_sample_order_recipe_depth_locations, which fail the same way without this PR.agc_host_import_sparseandagc_driver_sparse_dma_buf_importskip on this machine (its udmabuf size limit is above 1024 MiB).*_gpu_heaptests run the draw pipeline PR's switched tests and the sampling, table, atomics, compute, mesh relocation and thread-dimension dispatch tests withAPS5_GPU_HEAP=1 APS5_VERIFY_GPU_HEAP=1.mainvs this PR. Same machine and GPU,VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation ANYPS5_NO_SHADER_CACHE=1 ANYPS5_REQUIRE_VULKAN=1 ctest -V -j4:mainc9b2660main(the base tests of the*_gpu_heapset)*_gpu_heap,*heap*)was not found but was requested by env var VK_INSTANCE_LAYERSVUID-errorsThe first run of this comparison found
VUID-vkCmdDispatch-None-11308in the heap compute tests: avkCmdPushConstantscall puts the command buffer back in classic descriptor mode, which unbinds the heaps, and the heap's per-command-buffer bind record was not reset. Every push constant call now goes throughCmdPushConstants, which resets it (last commit). It also found test processes aborting at exit under the layer on #6946; that is fixed on #6946.Known limits
VK_EXT_descriptor_heapand heaps that hold 4096 image slots per kind and 2048 samplers; other devices run the existing path.Testing other titles
Please post:
APS5_TRACE_SECONDS=70:10, captured as the Performance report form describes: warm caches, no other trace or profile switch, log not redirected) withAPS5_DRAW_PIPELINE=1and withAPS5_GPU_HEAP=1 APS5_DRAW_PIPELINE=1, both withAPS5_PIPELINED_DRAWS=32 APS5_PIPELINE_DISPATCH=1and nothing else changed;APS5_VERIFY_GPU_HEAP=1, the last[heap-slots]and[gpu-heap]lines (a slot holding another texture or a differing stage is a bug).Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)