Skip to content

fix(shader): keep mesh register windows out of local memory on NVIDIA - #12233

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
beardsciences:fix/nvidia-mesh-window-buffer
Oct 30, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
beardsciences:fix/nvidia-mesh-window-buffer

Conversation

@beardsciences

@beardsciences beardsciences commented Oct 28, 2026 •

Copy link
Copy Markdown

What

On NVIDIA's proprietary driver, function arrays that the driver keeps in local memory return wrong values in subgroups 1–3 of a mesh workgroup when they are read under a data-dependent branch. Subgroup 0 is correct, and the same reads without a branch are correct. A wave64 geometry program that uses relative VGPR addressing (v_movrels/v_movreld) keeps its registers in the vector window, which is two such arrays per invocation, so waves 1–3 of a 256-thread NGG group lose register writes.

  • Measured with standalone GLSL mesh shaders (128 invocations, two arrays of 64 or more dwords): subgroups 1–3 fail every run. Uniform or per-invocation indices, with or without barriers, and function or module scope all fail. A branchless check of the same arrays passes, as do arrays of 48 dwords.
  • This broke the clipping geometry program of Sony's PS5 PSP emulator in MediEvil: Resurrection (PPSA07550). Replaying a captured menu draw on the host, the empty waves lost v3 = v22 << 2 in the setup block. They never cleared their candidate primitive slots, read 0 there, and wrote 76 degenerate triangles over wave 0's 6, so the 3D menu drew spikes.

On that driver a mesh program's window now lives in a storage buffer:

  • A 64 MiB device-local pool at descriptor set 3 holds 256 workgroup slots of 65536 dwords. Invocation 0 claims a slot through a per-slot lock (ticket counter plus compare-exchange), and releases it after the final barrier. The pool exists only on NVIDIA's proprietary driver. It is created before the first draw that uses it records anything, and its header is zeroed in a batch of its own that is submitted and waited for; the pool is published only after that wait, and a failure releases it so the next draw creates it again.
  • Direct register accesses keep one function variable per register:
    • A direct store also writes the buffer when a buffer read can see it (backward liveness).
    • Relative reads load the buffer; relative writes store only to it.
    • A direct read that may follow a relative write loads the buffer and refreshes the variable (forward analysis), so relative writes need no switch over the window.
  • A buffer entry is zeroed only when a buffer read can see it before a write.
  • APS5_NO_MESH_WINDOW_BUFFER=1 keeps the arrays. Other drivers are unchanged.

Cost, measured in PPSA07550's 3D main menu with #7893 (needed to reach it), RTX 3060 Ti, warm caches, APS5_TRACE_SECONDS=260:60 and no other trace or profile switch, two alternating runs per build. main is a59fd393ec6d (3a2b919 + #7893 76bea98), which draws the corrupt scene; the change is a6abcf9de1ec (this PR + the same #7893 head):

presents in 60 s frame p50 / p90 / p99 ms GPU busy per present
main + #7893, run 1 1654 38.0 / 43.3 / 52.2 24.2 ms
change + #7893, run 1 1280 36.4 / 77.9 / 110.7 42.9 ms
main + #7893, run 2 1646 38.0 / 43.2 / 53.7 24.4 ms
change + #7893, run 2 1284 36.2 / 76.2 / 116.0 42.9 ms

The window buffer costs about 18–19 ms of GPU time per present in this menu. All four summaries report trace_dropped (the event buffers of the queue worker and the draw committer filled after 9–38 s); totals_full is false for every thread, so the summary numbers are complete. Traces and tools/trace_digest.py output attached.

Only mesh programs that keep a vector window for relative register addressing (v_movrels/v_movreld) take this path, and only on NVIDIA's proprietary driver; every other shader, and every other driver, is compiled as on main. The faster main frames above render the corrupt scene, so the cost buys correct output rather than slowing down working output.

Summary lines
{"commit":"a59fd393ec6d","dirty":false,"env":{"APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_SECONDS":"260:60"},"gpu":"NVIDIA GeForce RTX 3060 Ti","driver":"NVIDIA 621.01","os":"Windows 10.0.26200","cpu":"AMD Ryzen 5 5600X 6-Core Processor","cpu_threads":12,"memory_mib":32674,"title_id":"PPSA07550","title":"MediEvil: Resurrection","content_version":"01.000.003","resolution":"3840x2160","window":{"spec":"seconds 260:60","first_present":6193,"ms":60012.311},"presents":1654,"frame_ms":{"p50":38.023,"p90":43.264,"p99":52.195,"max":69.873},"gpu_busy_ms_per_present":24.174,"worker_busy_pct":{"queue worker 0x0":47.762},"top_phases_ms_per_frame":{"VideoOut.Flip":18.221,"Driver.Present":18.085,"VideoOut.Flip.present":18.073,"Driver.Present.render_fence_wait":17.454,"Vulkan.Retire":17.453,"Vulkan.Retire.wait":17.447,"Driver.Packet":17.321,"Graphics.Draw":15.833},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":9.625,"events_lost":940448,"totals_full":false},"draw committer":{"events_full_after_s":24.553,"events_lost":339616,"totals_full":false}},"trace":"aps5-trace-a59fd393ec6d-23000.json.gz"}
{"commit":"a6abcf9de1ec","dirty":false,"env":{"APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_SECONDS":"260:60"},"gpu":"NVIDIA GeForce RTX 3060 Ti","driver":"NVIDIA 621.01","os":"Windows 10.0.26200","cpu":"AMD Ryzen 5 5600X 6-Core Processor","cpu_threads":12,"memory_mib":32674,"title_id":"PPSA07550","title":"MediEvil: Resurrection","content_version":"01.000.003","resolution":"3840x2160","window":{"spec":"seconds 260:60","first_present":5638,"ms":60026.885},"presents":1280,"frame_ms":{"p50":36.386,"p90":77.920,"p99":110.653,"max":183.849},"gpu_busy_ms_per_present":42.881,"worker_busy_pct":{"queue worker 0x0":92.287},"top_phases_ms_per_frame":{"Driver.Packet":43.262,"Graphics.Draw":41.560,"Driver.Packet.draw":41.526,"Driver.Draw":41.448,"Graphics.Draw.draw_and_resource_release":40.413,"VideoOut.Flip":36.666,"Driver.Present":36.499,"VideoOut.Flip.present":36.484},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":16.317,"events_lost":669945,"totals_full":false},"draw committer":{"events_full_after_s":33.279,"events_lost":205045,"totals_full":false}},"trace":"aps5-trace-a6abcf9de1ec-5384.json.gz"}
{"commit":"a59fd393ec6d","dirty":false,"env":{"APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_SECONDS":"260:60"},"gpu":"NVIDIA GeForce RTX 3060 Ti","driver":"NVIDIA 621.01","os":"Windows 10.0.26200","cpu":"AMD Ryzen 5 5600X 6-Core Processor","cpu_threads":12,"memory_mib":32674,"title_id":"PPSA07550","title":"MediEvil: Resurrection","content_version":"01.000.003","resolution":"3840x2160","window":{"spec":"seconds 260:60","first_present":6228,"ms":60025.286},"presents":1646,"frame_ms":{"p50":37.986,"p90":43.180,"p99":53.654,"max":71.361},"gpu_busy_ms_per_present":24.428,"worker_busy_pct":{"queue worker 0x0":47.833},"top_phases_ms_per_frame":{"VideoOut.Flip":18.659,"Driver.Present":18.379,"VideoOut.Flip.present":18.375,"Driver.Present.render_fence_wait":17.833,"Vulkan.Retire":17.832,"Vulkan.Retire.wait":17.825,"Driver.Packet":17.435,"Graphics.Draw":16.070},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":10.260,"events_lost":920764,"totals_full":false},"draw committer":{"events_full_after_s":25.193,"events_lost":334873,"totals_full":false}},"trace":"aps5-trace-a59fd393ec6d-2260.json.gz"}
{"commit":"a6abcf9de1ec","dirty":false,"env":{"APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_SECONDS":"260:60"},"gpu":"NVIDIA GeForce RTX 3060 Ti","driver":"NVIDIA 621.01","os":"Windows 10.0.26200","cpu":"AMD Ryzen 5 5600X 6-Core Processor","cpu_threads":12,"memory_mib":32674,"title_id":"PPSA07550","title":"MediEvil: Resurrection","content_version":"01.000.003","resolution":"3840x2160","window":{"spec":"seconds 260:60","first_present":5891,"ms":60022.217},"presents":1284,"frame_ms":{"p50":36.193,"p90":76.179,"p99":115.956,"max":183.309},"gpu_busy_ms_per_present":42.876,"worker_busy_pct":{"queue worker 0x0":90.375},"top_phases_ms_per_frame":{"Driver.Packet":42.230,"Driver.Packet.draw":40.516,"Graphics.Draw":40.486,"Driver.Draw":40.439,"Graphics.Draw.draw_and_resource_release":39.372,"VideoOut.Flip":36.761,"Driver.Present":36.625,"VideoOut.Flip.present":36.608},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":19.564,"events_lost":665710,"totals_full":false},"draw committer":{"events_full_after_s":37.669,"events_lost":200843,"totals_full":false}},"trace":"aps5-trace-a6abcf9de1ec-37332.json.gz"}

Overlap: the overlap comment lists #6416, #6751, #6959, #7392, #8246, #10547, #10846, #11557, #11951, #12548 and #12662. None of them implements this or needs it, and this needs none of them: each adds its own flag, field, environment row or debt line next to this PR's (SpirvTarget, CompiledShaderArtifact, the cache key, SpirvEmitterState, the draw paths in Draw.cpp, ENVIRONMENT.md, debt/shader.md), so whichever merges second keeps both lines. #6959 also changes EmitMeshEntryPoint, but in the layer output, not around the window claim and release.

Tested

Windows 11, NVIDIA GeForce RTX 3060 Ti, driver 621.01 (32.0.16.2101). main is 3a2b919, the change is this PR on it (7f0e19d).

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: NVIDIA GeForce RTX 3060 Ti, driver 621.01 (32.0.16.2101), Windows 11
  • AI-assisted: yes (Claude Code)

@github-actions github-actions Bot added the unblocks title Names a title ID on the Unblocks line label Oct 28, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found one initialization lifetime issue in the host pool setup that should be resolved before merge. PrepareMeshWindow records the header fill into a recorder command buffer and immediately sets pool.cleared = true. Recorder batches can be discarded if vkEndCommandBuffer or vkQueueSubmit fails; after that discard, later draws skip the fill even though no submitted command initialized the ticket counter and per-slot locks. The first mesh draw could then spin claiming a slot or reuse stale lock state. Please tie the initialized state to a successfully submitted initialization batch, or use an initialization path whose completion is guaranteed before later mesh draws can use the pool. Also include a focused check for the failed/discarded-first-batch case if feasible. I reviewed the shader indexing, liveness/initialization analysis, cache serialization, descriptor layout and release ordering statically. I could not reproduce the proprietary NVIDIA driver failure or run the new GPU regression test on this host, so I am not approving the driver-specific behavior without the author’s hardware evidence and this lifecycle case being addressed.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@beardsciences thanks, this reads well. I checked the slot lock (claim and release are well-structured, because the release sits in the mesh entry point after the guest call, so early returns still reach it), the pool size (header included) and the gating (NVIDIA proprietary + mesh stage + non-empty window + fits a slot, so other drivers are unchanged).

One thing blocks CI: conventions fails with "Fill in the pull request template: add the '### What' heading of the template". The description starts straight with the text, so adding a ### What heading above the first paragraph should be enough.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 6416 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp L326
PR 6751 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp L326
PR 6959 core/shader/recompiler/Recompiler.cpp L896
PR 7392 core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp L326
PR 9508 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp L2435, L3368-3369
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp L529-530
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp L45-46, L51
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp L332
PR 10547 core/shader/recompiler/CacheKey.hpp L357
core/shader/recompiler/Recompiler.hpp L353
PR 10846 core/shader/recompiler/Recompiler.hpp L353
PR 11351 core/shader/recompiler/CacheKey.hpp L357
core/shader/recompiler/Recompiler.cpp L679
core/shader/recompiler/Recompiler.hpp L353
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp L46
PR 11557 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp L2351-2352
PR 11951 core/shader/recompiler/Recompiler.cpp L896-897
PR 12548 docs/dev/debt/shader.md L198
PR 12662 core/shader/recompiler/CacheKey.hpp L357
core/shader/recompiler/Recompiler.hpp L353
PR 12764 core/shader/recompiler/Recompiler.cpp L718, L1126
core/shader/recompiler/Recompiler.hpp L757
core/shader/recompiler/ShaderDiskCache.cpp L543, L607
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp L51
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp L332
PR 12944 core/shader/recompiler/Recompiler.cpp L718, L1126
core/shader/recompiler/Recompiler.hpp L757
core/shader/recompiler/ShaderDiskCache.cpp L543, L607
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp L51
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp L332
PR 12980 core/shader/recompiler/CacheKey.hpp L357
core/shader/recompiler/Recompiler.hpp L353

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1368 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 1395 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 2434 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4040 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4472 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 5021 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
docs/dev/debt/shader.md
PR 5371 docs/dev/debt/shader.md
PR 5609 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 5925 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Draw.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6185 docs/dev/ENVIRONMENT.md
PR 6186 docs/dev/ENVIRONMENT.md
PR 6188 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Draw.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/shader/recompiler/Recompiler.hpp
PR 6235 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp
docs/dev/debt/shader.md
PR 6280 core/shader/recompiler/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6412 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6488 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
docs/dev/debt/shader.md
PR 6804 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6924 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 6960 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6970 core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6977 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7256 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 7275 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
PR 7369 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 7449 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 7457 docs/dev/ENVIRONMENT.md
PR 7464 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7514 docs/dev/ENVIRONMENT.md
PR 7604 docs/dev/ENVIRONMENT.md
PR 7642 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp
docs/dev/ENVIRONMENT.md
PR 7684 core/shader/recompiler/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 7689 core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 7922 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 8004 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 8014 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 8223 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 8246 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
docs/dev/ENVIRONMENT.md
PR 8484 docs/dev/ENVIRONMENT.md
PR 8542 docs/dev/ENVIRONMENT.md
PR 9012 docs/dev/ENVIRONMENT.md
PR 9345 docs/dev/ENVIRONMENT.md
PR 9356 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Draw.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp
core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp
docs/dev/ENVIRONMENT.md
PR 9387 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Draw.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp
core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp
docs/dev/ENVIRONMENT.md
docs/dev/debt/shader.md
PR 9432 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9489 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
PR 9712 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
docs/dev/ENVIRONMENT.md
PR 9748 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp
docs/dev/debt/shader.md
PR 9889 core/shader/recompiler/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 9891 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 9893 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9895 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9897 core/libs/prx/libSceAgcDriver/Graphics/include/Draw.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 9898 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 9915 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 10026 docs/dev/ENVIRONMENT.md
PR 10043 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 10049 core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 10069 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 10193 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp
docs/dev/ENVIRONMENT.md
PR 10233 docs/dev/debt/shader.md
PR 10442 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
docs/dev/ENVIRONMENT.md
PR 10472 docs/dev/ENVIRONMENT.md
PR 10497 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 10652 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
docs/dev/debt/shader.md
PR 10757 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
docs/dev/ENVIRONMENT.md
PR 10894 docs/dev/ENVIRONMENT.md
PR 11452 core/libs/prx/libSceAgcDriver/Graphics/include/Draw.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 11551 core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp
PR 11554 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 11948 docs/dev/ENVIRONMENT.md
PR 12120 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 12246 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/shader/recompiler/CacheKey.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 12324 core/shader/recompiler/Recompiler.cpp
docs/dev/ENVIRONMENT.md
PR 12569 docs/dev/ENVIRONMENT.md
PR 12619 core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 12702 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 12745 docs/dev/ENVIRONMENT.md
PR 12828 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/Recompiler.hpp
PR 12863 docs/dev/ENVIRONMENT.md
PR 12921 docs/dev/ENVIRONMENT.md

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@beardsciences thanks, the window buffer path reads correctly to me: the slot claim and release, the bound that falls back to the arrays when a window doesn't fit a slot, and the stale-load and live-store planning all check out, and the cache key and disk cache carry the new flag.

One change before merge: the new entry in docs/dev/debt/shader.md names the game next to PPSA07550. Debt docs can carry the title ID but not game names, so please keep PPSA07550 and drop the name (PSPHD can stay).

@github-actions github-actions Bot added shader core/shader docs agc libSceAgc* labels Oct 28, 2026
@beardsciences

beardsciences commented Oct 28, 2026 •

Copy link
Copy Markdown
Author

@MarcZX10 thanks, you're right: cleared was set when the fill was recorded, so a discarded first batch left the ticket counter and slot locks uninitialized for every later draw.

Fixed in 076cec9. The header's initialization no longer goes into a draw's command buffer:

  • PrepareMeshWindow now runs before the draw records anything (before StartDrawPass in recordDraw, before the batch is taken in recordPreparedDraw) and no longer takes a command buffer.
  • On first use it creates the pool and zeroes the header in a CommandBatch of its own, with a transfer-to-all-commands barrier, then SubmitAndWait. The pool counts as ready (its descriptor set is published) only after that wait returns, so no later draw can bind it uninitialized.
  • If anything in that path throws (including the submit or the wait), the pool is released, and the next draw that needs it creates and zeroes it again.
  • After that the header needs no reset: every workgroup releases its slot after the final barrier.

Two other approaches I tried and dropped:

  • Resetting the header in every draw that uses the pool, behind a barrier on earlier mesh work (as the pixel function LDS counter does) hung MediEvil's menu at its first window draw in 2 of 2 runs: presents stopped while the guest kept flipping. I did not find why.
  • Zeroing the header through a host mapping needs host-visible memory. Here the device-local host-visible heap is 214 MiB with no budget left, so the pool landed in host memory and the menu dropped from about 21 to 11 fps.

I did not add a test for the discarded-batch case: no recorded initialization is left to discard, and I did not find a way in the test harness to make vkEndCommandBuffer or vkQueueSubmit fail on purpose.

Rerun on 076cec9 (RTX 3060 Ti, 621.01, Windows 11, current main cccdaa2):

@beardsciences

Copy link
Copy Markdown
Author

@oneandonlydean done in 7779011: the debt entry now names only PPSA07550. Thanks.

@fogydes

fogydes commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

verdict: should-fix

  1. [read-only] The body makes compile-time and frame-rate claims but does not include the paired aps5-summary.jsonl lines and tools/trace_digest.py summaries required by AGENTS.md for speed claims.
  2. [read-only] The tested section reports the focused CTest results, but does not document the required Vulkan validation environment or provide logs confirming the validation layer loaded and that the change adds no VUID errors.
  3. [verified] I could not reproduce the required main/change GPU test pair here: the available build-full tree failed during CMake regeneration, and the focused test executables are absent. The NVIDIA GPU is available, but the results in the body remain author-reported.
  4. [verified] The PR is 180 commits behind main. The merge tree against current main is clean, but pairwise merge-tree checks conflict with open non-draft PRs fix(shader): keep faulting invocations in their wave and mask their later writes #6416, test(shader): skip faulting wave test on narrow subgroups #6751, and fix(shader): mask writes after barrier-safe BDA faults #7392 in SpirvEmitter.cpp; fix(agc): render layered mesh targets within device limits #6959 and fix(shader): leave the workgroup memory set out of a compute module's bindings #11951 in Recompiler.cpp; fix(agc): preserve CPU writes after resident linear target draws #8246 in Draw.cpp; and fix(shader): fence the LDS lock's critical section on MoltenVK #12188 in VulkanDevice.cpp, CacheKey.hpp, Recompiler.cpp, and SpirvEmitter.hpp. Please rebase and reconcile these shared-file changes.
  5. [read-only] The PR title differs from its latest commit subject: docs(shader): name the mesh window title by its ID only.
  6. [verified] tools/check_conventions.py --base cccdaa2 and git diff --check pass. The PR body’s shader/GPU test results are author-reported.

Freshness envelope: head 7779011; merge-base cccdaa2; behind main: 180; git merge-tree --write-tree origin/main refs/remotes/pr-head/12233: exit 0, clean.

@MarcZX10

Copy link
Copy Markdown
Contributor

I rechecked head 77790116601ca240390208a280ad51481acbb5fa. The pool initialization race from my earlier review appears addressed: header initialization now runs in a separate submitted-and-waited batch, and pool readiness is published afterward. The report still does not satisfy the current review requirements:

  1. The branch is diverged from current main (316615d042c087942e3fa11f258669abd92c3f27) by 276 commits behind and 3 ahead. Please rebase/update it to current main, resolve the newly exposed overlap, and rerun the focused tests.
  2. The new mesh shader path needs the required paired validation run for this shader change: agc_driver_mesh_vector_window_waves on current main and the updated head with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation; provide logs confirming the layer loaded and compare new VUID- findings. The currently reported RTX 3060 Ti runs and APS5_NO_MESH_WINDOW_BUFFER negative control are useful, but do not document that required pair or validation result.
  3. The description makes frame-rate and compilation-time comparisons but has no paired current-main/change aps5-summary.jsonl lines, attached traces and trace_digest.py output. Please provide those for the speed claim or remove the unsupported speed comparison.
  4. The checklist says Unblocks: PPSA07550 (no Game bring-up discussion yet). CONTRIBUTING requires the title ID and its Game bring-up discussion, or none; please add the discussion link or change the line to none.

Conventions passes, but only conventions/labels are reported for this head; I found no current focused GPU validation run. I am keeping changes requested pending the above evidence and rebase.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rechecked head 77790116601ca240390208a280ad51481acbb5fa. The pool initialization race from my earlier review appears addressed: header initialization now runs in a separate submitted-and-waited batch, and pool readiness is published afterward. The report still does not satisfy the current review requirements:

  1. The branch is diverged from current main (316615d042c087942e3fa11f258669abd92c3f27) by 276 commits behind and 3 ahead. Please rebase/update it to current main, resolve the newly exposed overlap, and rerun the focused tests.
  2. The new mesh shader path needs the required paired validation run for this shader change: agc_driver_mesh_vector_window_waves on current main and the updated head with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation; provide logs confirming the layer loaded and compare new VUID- findings. The currently reported RTX 3060 Ti runs and APS5_NO_MESH_WINDOW_BUFFER negative control are useful, but do not document that required pair or validation result.
  3. The description makes frame-rate and compilation-time comparisons but has no paired current-main/change aps5-summary.jsonl lines, attached traces and trace_digest.py output. Please provide those for the speed claim or remove the unsupported speed comparison.
  4. The checklist says Unblocks: PPSA07550 (no Game bring-up discussion yet). CONTRIBUTING requires the title ID and its Game bring-up discussion, or none; please add the discussion link or change the line to none.

Conventions passes, but only conventions/labels are reported for this head; I found no current focused GPU validation run. I am keeping changes requested pending the above evidence and rebase.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 28, 2026
@beardsciences
beardsciences force-pushed the fix/nvidia-mesh-window-buffer branch from 7779011 to 1f9b309 Compare October 28, 2026 16:12
@github-actions github-actions Bot removed the unblocks title Names a title ID on the Unblocks line label Oct 28, 2026
@beardsciences

beardsciences commented Oct 28, 2026 •

Copy link
Copy Markdown
Author

@MarcZX10 @fogydes thanks. Updated:

  1. Rebased onto main 750cf6c and squashed into one commit, 1f9b309, whose subject is the PR title. fix(shader): fence the LDS lock's critical section on MoltenVK #12188 had added ldsLockFences on the same lines as meshWindowBuffer in VulkanDevice.cpp, CacheKey.hpp, Recompiler.cpp and SpirvEmitter.hpp; both flags are kept.
  2. Validation pair on 750cf6c and 1f9b309, ctest --test-dir build -R "mesh|vector_window|relative|window_lane|ngg|provoking" -V with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. Logs attached: neither has the layer-not-found line, the layer reports the same 10 WARNING-VkImageSubresourceRange-layerCount-compatibility messages in both, and there is no VUID- error. main passes 26 of 26 and the change 27 of 27; the extra one is agc_driver_mesh_vector_window_waves, which does not exist on main. On the change it also fails with APS5_NO_MESH_WINDOW_BUFFER=1 (0 64 64 64) and passes without it, 2 of 2 each.
  3. Speed: the body now shows the four summary lines (two alternating warm runs each of main + fix(shader): hand a geometry program strip and fan triangles in the order of its provoking vertex mode #7893 and the change + fix(shader): hand a geometry program strip and fan triangles in the order of its provoking vertex mode #7893, same menu, APS5_TRACE_SECONDS=260:60) with traces and trace_digest.py output attached: GPU busy per present 25.0–25.8 ms against 38.2–41.7 ms. I removed the untraced compile-time and fps comparisons.
  4. Unblocks: is now none.

Validation logs:
validation-main-750cf6cf48.txt
validation-pr-1f9b309c58.txt

Traces (main + #7893 is 4ee67efbf774, the change + #7893 is 561c654a23d7):
aps5-trace-4ee67efbf774-10424.json.gz
aps5-trace-4ee67efbf774-5776.json.gz
aps5-trace-561c654a23d7-31416.json.gz
aps5-trace-561c654a23d7-8952.json.gz

trace_digest.py output:
digest-tr-main-1.txt
digest-tr-main-2.txt
digest-tr-pr-1.txt
digest-tr-pr-2.txt

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 28, 2026
@adriadam10 adriadam10 self-assigned this Oct 28, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewing exact head 1f9b309c584049a6c6bec2836dc1a969cae4b62c against current main 319621ba2e578bd053b0baba2877fe70520e0737.

What I ran

  • git diff --check against current main: passed.
  • git merge-tree --write-tree against current main: clean merge tree (e7369b6c37aa7841c323a1ae0362c44c9aa9b0e5).
  • Inspected the PR body, current diff, review history, and hosted checks. Conventions, labels, report, eligibility, and Windows passed; Linux and specialized jobs were skipped.
  • I did not build the project or run CTest/GPU tests locally. The validation and trace evidence below is author-reported, not independently reproduced.

Results

Check Result Evidence
Patch whitespace Pass Local git diff --check against current main
Mergeability Pass Local merge-tree against current main
Hosted Windows Pass GitHub Actions for this head
Hosted conventions, labels, report, eligibility Pass GitHub Actions
Hosted Linux / specialized jobs Skipped GitHub Actions
Focused GPU validation Stale baseline Author posted a same-machine RTX 3060 Ti pair for 750cf6cf48 and this PR head, with Vulkan validation enabled and logs attached. Current main is 319621ba2e578bd053b0baba2877fe70520e0737, so that report does not cover the current base.
Performance comparison Stale baseline The paired traces and trace_digest.py outputs compare 750cf6cf48 plus #7893 with this PR plus #7893. They document a substantial GPU-time regression, but do not compare current main with the change.

Findings

Blocking

  • The branch and its validation evidence are behind current main. The PR's merge base is 750cf6cf487f948017295b1f3717af6f886f0f85; current main is 319621ba2e578bd053b0baba2877fe70520e0737 (282 commits ahead of the PR branch). The merge-tree is clean, but the PR template requires a current-main basis, and this branch has not been integrated with the latest base. Please update/rebase it onto current main, resolve any newly exposed overlaps, and rerun the focused checks on the updated head.
  • The required shader/AGC validation pair must cover current main and the updated head. AGENTS.md requires the same focused tests on both builds, on the same machine and GPU, with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation; confirm the layer loaded and compare all VUID- lines. The posted pair uses 750cf6cf48, not current main. Keep the PR Draft until this updated pair is available, or post the pair and update the report.
  • The performance claim also needs a current-main pair. The attached summary lines, traces, and digests use 750cf6cf48 as “main”; they do not establish the comparison against current main. Please rerun the same scene and machine against the updated base and head and attach the matching summary lines, traces, and digest output, or remove the stale comparison from the PR description.

The previous pool-initialization concern appears addressed by publishing the descriptor set only after the separately submitted initialization batch completes. I did not independently execute the GPU regression test on this head.

Sources

Not verified

  • I did not run a build, CTest, or GPU test locally.
  • I did not independently inspect the attached GPU logs or trace files.
  • No current-main validation or performance comparison is included in the evidence I checked.

To get this to a good state

Rebase/update onto current main 319621ba2e578bd053b0baba2877fe70520e0737, then post fresh same-machine GPU validation logs and trace comparison for that base and the new head. The validation logs should confirm actual layer loading and show no new VUID- errors; the performance evidence should include the paired summary lines, traces, and trace_digest.py output. Keep the PR Draft until the required GPU pair is available.

@MarcZX10

Copy link
Copy Markdown
Contributor

Correction to the Sources section of my formal review: the author’s current-base GPU and trace evidence is in this comment. The earlier link in my review used the wrong comment ID; the review findings are unchanged.

@beardsciences
beardsciences marked this pull request as draft October 29, 2026 01:43
@beardsciences
beardsciences force-pushed the fix/nvidia-mesh-window-buffer branch from 1f9b309 to 7f0e19d Compare October 29, 2026 01:43
@beardsciences

beardsciences commented Oct 29, 2026 •

Copy link
Copy Markdown
Author

@MarcZX10 thanks. Updated on current main:

  1. Rebased onto main 3a2b919 (clean, no conflicts); the PR is one commit, 7f0e19d, with the same subject as the title.
  2. Validation pair on 3a2b919 and 7f0e19d, same machine and GPU (RTX 3060 Ti, 621.01): ctest --test-dir build -R "mesh|vector_window|relative|window_lane|ngg|provoking" -V with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. Logs attached. Neither contains was not found but was requested by env var VK_INSTANCE_LAYERS; both report the same 10 WARNING-VkImageSubresourceRange-layerCount-compatibility messages and no VUID- error. main passes 26 of 26, the change 27 of 27 (the extra one is agc_driver_mesh_vector_window_waves). On the change that test fails with APS5_NO_MESH_WINDOW_BUFFER=1 (wrong threads per wave: 0 64 64 64) and passes without it, 2 of 2 each.
  3. Traces redone against the new base: main a59fd393ec6d (3a2b919 + fix(shader): hand a geometry program strip and fan triangles in the order of its provoking vertex mode #7893 76bea98) and the change a6abcf9de1ec (7f0e19d + the same fix(shader): hand a geometry program strip and fan triangles in the order of its provoking vertex mode #7893 head), same menu, warm caches, APS5_TRACE_SECONDS=260:60, alternating runs. GPU busy per present 24.9 / 25.0 ms against 41.9 / 42.0 ms. The description now carries these summary lines and table; traces and trace_digest.py output attached.

aps5-trace-a6abcf9de1ec-8436.json.gz
aps5-trace-a6abcf9de1ec-50196.json.gz
aps5-trace-a59fd393ec6d-25800.json.gz
aps5-trace-a59fd393ec6d-50976.json.gz
digest-tr-main-1.txt
digest-tr-main-2.txt
digest-tr-pr-1.txt
digest-tr-pr-2.txt
validation-main-3a2b919816.txt
validation-pr-7f0e19dd6d.txt

@beardsciences
beardsciences marked this pull request as ready for review October 29, 2026 01:45

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — changes still requested

I reviewed the updated head 7f0e19dd6d15017f8f281c1b3a98d1325f8da3d6 against current main (3a2b91981651569163768854a1fcdc7193b6667b). The implementation review and updated test evidence address the earlier technical questions. I am keeping this review at Changes requested because the four performance summary lines still record APS5_TRACE_DIR; the repository's performance report form instructs reporters not to set other APS5_TRACE_* or APS5_PROFILE_* switches.

What I ran

On Linux x86_64 (GCC), I checked the exact PR head and built the focused mesh test in the isolated review build:

  • python3 tools/check_conventions.py --base upstream/main --head refs/remotes/review/12233
  • git diff --check upstream/main...refs/remotes/review/12233
  • git merge-tree --write-tree upstream/main refs/remotes/review/12233
  • ninja -C /tmp/anyps5-pr12664-build-full agc_driver_mesh_vector_window_waves_tests
  • Native CTest for agc_driver_mesh_vector_window_waves
  • The same executable under qemu-x86_64 -cpu Haswell

I also downloaded the four attached traces and matching digest outputs and recomputed each digest using tools/trace_digest.py with its attached summary. I read both attached Vulkan validation logs; I did not run the Windows/NVIDIA workload locally.

Results

Check Result
Conventions Passed
Diff whitespace Passed
Merge preview with current main Clean (17122221904ecf0f155886fb2f15a7f22fd048c1)
Focused mesh test build Passed (167/167 Ninja actions)
Native CTest Could not reach assertions: host CPU feature guard aborts on this Xeon E5-1620 v2
QEMU Haswell run Skipped with exit 77 because llvmpipe does not provide the required mesh-shader compiler
Four attached trace digests All reproduced exactly (after normalizing CRLF)
Attached Vulkan validation logs Same ten WARNING-VkImageSubresourceRange-layerCount-compatibility warnings in each; no VUID- errors
Raw local check output
Convention checker: passed
Diff check: clean
Merge preview tree: 17122221904ecf0f155886fb2f15a7f22fd048c1
Ninja: 167/167 actions; agc_driver_mesh_vector_window_waves_tests linked
Native CTest: aborted by CPU feature guard before test assertions
QEMU Haswell: test skipped (exit 77); llvmpipe lacks required mesh-shader compiler
Trace digest recomputation: 4/4 matched attached digest output

Negative control and hardware evidence

The author reports that the exact current PR head passes the new mesh test twice on an RTX 3060 Ti, while setting APS5_NO_MESH_WINDOW_BUFFER=1 reproduces wrong results in waves 1–3 twice. Their current-head Windows validation run reports 27/27 tests on the change and 26/26 on its matching main baseline, with no VUID- errors. The author also supplied paired baseline/change traces from the same title, scene, and machine. I independently recomputed all four attached digests successfully. These are author-run hardware results, not locally reproduced runs. Details and attachments are in the author's current-head evidence comment.

Findings

Blocking

  • The four aps5-summary.jsonl lines include APS5_TRACE_DIR. The current performance report form says not to set any other APS5_TRACE_* or APS5_PROFILE_* switch. The current report therefore does not meet the repository's stated capture procedure, even though its traces are attached and their digests reproduce. Please recapture the same baseline and change runs without APS5_TRACE_DIR (write the traces to the working directory), then replace the four summary lines and attach the corresponding traces and digest outputs. Keep the same title, scene, machine, and fixed input so the comparison remains controlled.

Non-blocking

  • The current PR merge preview against main is clean. Its documented #7893 prerequisite is common to both baseline and change captures, and the report explains why it is needed to reach the scene; the comparison isolates this PR's effect.

Not verified locally

I could not run the NVIDIA-specific workload or reproduce the Windows validation/test results on this Linux host. Native CTest stopped at the CPU guard, and QEMU with llvmpipe skipped the test because its Vulkan implementation lacks the required mesh-shader compiler. I therefore do not count those as passing local runtime tests.

To resolve

Please provide the four recaptured summary lines and their matching traces/digests without APS5_TRACE_DIR. I can then re-check the updated report and clear this request if the capture record complies.

@beardsciences

Copy link
Copy Markdown
Author

@MarcZX10 thanks for recomputing the digests. Recaptured without APS5_TRACE_DIR: same builds (main a59fd393ec6d = 3a2b919 + #7893 76bea98, change a6abcf9de1ec = 7f0e19d + the same #7893 head), same machine, same menu and fixed pad input, warm caches, alternating runs, only APS5_TRACE_SECONDS=260:60 set among the trace and profile switches, log not redirected, traces written to the working directory. The four summary lines and the table in the description are replaced; the new traces and tools/trace_digest.py output are attached. GPU busy per present 24.2 / 24.4 ms against 42.9 / 42.9 ms. The PR head is unchanged (7f0e19d), so the validation pair above still applies.

digest-tr-pr-2.txt
aps5-trace-a6abcf9de1ec-5384.json.gz
aps5-trace-a6abcf9de1ec-37332.json.gz
aps5-trace-a59fd393ec6d-2260.json.gz
aps5-trace-a59fd393ec6d-23000.json.gz
digest-tr-main-1.txt
digest-tr-main-2.txt
digest-tr-pr-1.txt

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — approved

The requested performance capture correction is complete. The four current summary lines no longer include APS5_TRACE_DIR, and I reproduced each attached digest from its trace and summary.

What I ran

  • Checked current head 7f0e19dd6d15017f8f281c1b3a98d1325f8da3d6 and its updated description/comment.
  • Downloaded all four newly attached trace files and digest reports.
  • Ran tools/trace_digest.py on each trace with its matching summary line.
  • Refreshed the hosted check state and re-read the exact-head author evidence and previous review findings.

Results

Check Result
Performance summary environment Passed; only APS5_TRACE_SECONDS=260:60 is set among trace/profile switches
Recomputed trace digest: main run 1 Exact match to attached digest
Recomputed trace digest: main run 2 Exact match to attached digest
Recomputed trace digest: change run 1 Exact match to attached digest
Recomputed trace digest: change run 2 Exact match to attached digest
Author-reported NVIDIA validation pair Pass on same RTX 3060 Ti machine; 26/26 baseline and 27/27 change tests, no VUID- errors
Hosted checks Conventions, eligibility, labels, and report pass; Windows pending; Linux and specialized jobs skipped

All four full trace_digest.py outputs match the attached reports exactly. The reported baseline and change use the same title, menu scene, fixed input, machine, driver, and warm-cache alternating capture procedure.

Negative control

Control Result
APS5_NO_MESH_WINDOW_BUFFER=1 Author reports the focused test fails in waves 1–3 in two runs; normal PR configuration passes in two runs
Independent reproduction Not run; this Linux host lacks the required CPU features and does not provide the author’s NVIDIA hardware

Findings

Blocking

None found on this head. The prior report-format issue is resolved, and the attached trace digests verify against the current captures.

Non-blocking

  • The performance report shows a substantial GPU-time cost for this path. The report also demonstrates that the baseline renders corrupted output and the change fixes it, so the measured cost is documented against the behavior being corrected.
  • The Windows hosted job is still pending. The exact-head NVIDIA validation and performance evidence are reported by the author and were not rerun on my machine.

Sources

Not verified

  • I did not execute the Windows/NVIDIA workload, the author-reported CTest validation pair, or the negative control locally.
  • Windows hosted build remains pending at the time of this review; Linux product and specialized workflows were skipped by policy.

To get to a good state

No further changes are required for the findings in this review.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@beardsciences thanks, the fill now happens once in its own submit before the first draw that uses the window, which removes the dropped-command-buffer case. One request left: the Mesh vector windows entry in docs/dev/debt/shader.md still says "PSPHD's clipping geometry program" and "PSPHD's program" (about a third of the way into the line). Debt docs name titles by PPSA id only, so please change both to "the title" (PPSA07550 is already there).

@adriadam10 courtesy: assigned to you; the rest of this update is code-pass.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed exact head 7f0e19dd6d15017f8f281c1b3a98d1325f8da3d6 after the new review comment. The implementation and matched validation/performance evidence remain as reviewed, but docs/dev/debt/shader.md:196 still names “PSPHD” twice. The repository debt convention uses PPSA IDs or “the title”; please replace both names with “the title” (PPSA07550 is already identified in this entry). The current-main merge preview is clean and git diff --check passes. No GPU tests rerun locally.

On NVIDIA's proprietary driver, function arrays that live in local memory
and are read under a data-dependent branch return wrong values in
subgroups 1-3 of a mesh workgroup; subgroup 0 is correct. A wave64
geometry program whose relative addressing keeps its VGPRs in the vector
window therefore loses register writes in every wave but the first. In
the clipping geometry program of Sony's PS5 PSP emulator (PPSA07550) the
empty waves never cleared their candidate primitive slots and wrote
degenerate triangles over wave 0's, so the 3D menu drew spikes.

On that driver a mesh program's window now lives in a device-local
storage buffer of 256 workgroup slots of 65536 dwords, claimed through a
per-slot lock and released after the final barrier. The pool is created
before the first draw that uses it records anything, and its header is
zeroed in a batch of its own that is submitted and waited for; the pool
is published only after that wait, and a failure releases it. Direct
register accesses keep one function variable per register; a direct
store also writes the buffer when a buffer read can see it, relative
reads load the buffer, relative writes store only to it, and a direct
read that may follow a relative write loads the buffer and refreshes the
variable. A buffer entry is zeroed only when a buffer read can see it
before a write. APS5_NO_MESH_WINDOW_BUFFER=1 keeps the arrays.
@beardsciences
beardsciences force-pushed the fix/nvidia-mesh-window-buffer branch from 7f0e19d to 6f79055 Compare October 29, 2026 12:58
@beardsciences

Copy link
Copy Markdown
Author

@MarcZX10 @oneandonlydean thanks, done: both names in the docs/dev/debt/shader.md entry now say "the title" (PPSA07550 stays in the entry). The commit is amended on the same base (3a2b919), and this one line is the only change from 7f0e19d, so the code, the validation pair and the traces above still apply. tools/check_conventions.py passes and the merge with current main is clean.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — new head, approved

The only change since the previously approved head replaces the two title nicknames in the debt entry, resolving the outstanding repository wording request.

What I ran

Reviewed head 6f79055a0001703bc0419fb03a8dabd9ef1a2677 against current main 899ae3d1ad32ccea94dc24e66fdc948ce8a88171. The merge preview is clean (9f92fdb8033189e5630fb23bc56a08ad5b1aaa47). Environment: Debian GNU/Linux 12; GCC 12.2.0; CMake 3.25.1.

git diff 7f0e19dd6d15017f8f281c1b3a98d1325f8da3d6 refs/remotes/upstream/pr/12233-review -- docs/dev/debt/shader.md
git diff --check upstream/main...refs/remotes/upstream/pr/12233-review
git merge-tree --write-tree upstream/main refs/remotes/upstream/pr/12233-review
python3 tools/check_conventions.py --base upstream/main --head refs/remotes/upstream/pr/12233-review --title 'fix(shader): keep mesh register windows out of local memory on NVIDIA' --body /tmp/AnyPS5-12233-body.md

Results

Check Result
Current-head delta One debt-document line changed; shader implementation and tests are unchanged
Conventions, including PR body Passed
Diff check Passed
Merge preview Clean
Hosted conventions and labels Pending when checked
Behavioral tests Not rerun; the new head changes debt wording only
Raw output
All conventions checks passed.

git diff --check produced no output and exited 0. git merge-tree --write-tree returned:

9f92fdb8033189e5630fb23bc56a08ad5b1aaa47

Negative control

Assertion Change made to code under test Did the test fail?
Mesh-window behavior No behavioral code changed on this head; no negative control rerun Not applicable to the documentation-only update

Findings

Blocking

None on this head.

Non-blocking

None on this head.

Sources

Not verified

I did not rerun the GPU workload, Vulkan validation pair, performance captures, or CTest because the changed line is documentation only. Hosted conventions and labels were still pending.

To get this to a good state

No further changes are required for the findings reviewed here. I will rerun these checks if the head changes.

@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head 6f79055 merged onto main ba5b533 · tested 2026-10-29 13:30 UTC · 23 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,218 passed · 5 failed (0 new, 5 also fail on main) · 39 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,262 of 2,891 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (860 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) none
Failing on main too (5, not counted against this PR)
agc_compute_pipeline_inspection
agc_submit_progress
agc_submit_progress_no_timeline
guest_mmap_executable
guest_socket_options_multicast
Validation IDs already present on main (4 IDs in 407 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (407 tests): agc_compute_pipeline_replay, agc_driver, agc_driver_address_passes, agc_driver_address_passes_draw_transitions, agc_driver_address_passes_targets_split, agc_driver_alpha_to_coverage, agc_driver_array_layer_target_cpu, agc_driver_bc_block_copy, and 399 more
VUID-vkCmdCopyImage-srcImage-01548 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_storage_dispatch, and 3 more
VUID-vkCmdCopyImage-srcImage-01551 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_storage_dispatch, and 3 more
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (2 tests): agc_driver_prepared_exports, agc_driver_srgb8_color_target
Skipped (39)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_msaa_dcc_compared_import_2: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_msaa_dcc_compared_import_4: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_msaa_dcc_compared_import_8: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_driver_sparse_dma_buf_import: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_stencil_ref_export: skipped, the device lacks VK_EXT_shader_stencil_export
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_main: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fresh: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_labels: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SD
Reproduce
git fetch origin 6f79055a0001703bc0419fb03a8dabd9ef1a2677 ba5b5330b37724bca4b99f30faab504653635a04
git checkout ba5b5330b37724bca4b99f30faab504653635a04 && git merge --no-ff 6f79055a0001703bc0419fb03a8dabd9ef1a2677
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@adriadam10
adriadam10 merged commit d0cf19b into boykopovar:main Oct 30, 2026
10 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Re-checked the new head: the only change is the debt entry, where both title nicknames now read "the title" (PPSA07550 stays). Code, test and traces are as reviewed before; the NVIDIA CI run on this head passed with no new failures; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants