Repository navigation
perf(agc): pipelined draws on by default - #1202
Conversation
github-actions
Bot
added
needs rebase
|
Test result: Demon's Souls on Linux/NVIDIA with #1202 (and #1201 under it) on by default Setup: Nobara 44 x86-64, kernel 7.2, RTX 5070 Ti, NVIDIA 595.91.07. Demon's Souls (PPSA01341 v1.07), tutorial level (Outpost Passage), steady state for about 150 s after the HUD appears. Base is upstream main df16c4c plus #861, #864 and #157, our local fixes (including the NVIDIA dma-buf import guard) and a job-worker idle sleep for this title. The PR side is the same tree with this PR merged, plus #1071, which only touches shutdown. The merge conflicted in 5 files against our tree (Dispatch.cpp, WrittenBuffers.cpp, PacketExecution.cpp, Submission.cpp, ShaderResources.cpp), all because of our local changes. The only code fix it needed was our own Two runs per side. Numbers are medians of the 10 s windows in the steady state:
Median fps goes from 1.43 to 1.90 (+33 %), and both PR runs beat both base runs. Queue 0 spends about 0.7 s less per 10 s in Graphics::Draw while getting through about 40 % more draw packets. The run logs show no new error signatures, and the picture is unchanged. Driver ctest -j1: 412/413. The one failure (guest_force_tls_destructor) also fails on main. Unrelated to this PR: both PR runs still crash at teardown when the window closes (one double free, one SIGSEGV), and so does the base. AI-assisted: yes (an agent ran the builds, the game runs and the measurements). |
b5be7a1 to
569252b
Compare
|
@MichonGoddijn231849 a follow-up to the test result above, about your next step of keeping compute dispatches from draining the pipeline. On Demon's Souls that looks like the main remaining gap, so I wanted to check where you are with it before we write anything. Per frame in the tutorial level (queue 0, same stack as above, ~2.2 fps, 448 ms on the worker):
The committer is ~4.6% busy here while the worker is pegged, so the time goes to the dispatch path and to indirect draws that #1201 doesn't pipeline. About 2.6k of the dispatches are one chain of small dispatches (program Two questions:
|
Diablo II with this PR: the GPU is now the limit (Nsight Systems trace)Follow-up to the numbers above, same setup: PPSA03912 in the Rogue Encampment, Linux, RTX 3080 Ti (NVIDIA 615.71.09), this PR's head c653e9e on What a frame does
So there is about one pipeline bind per draw, a few thousand descriptor updates per frame, but few barriers. Where the time goes
So on What I could not seeI have no GPU performance counters here (Nsight's GPU metrics need a driver permission I have not changed), so I cannot say which draws or passes make up that 155 ms. Candidates are the 3840x2160 targets, blending over large areas, and what each draw's sampled images need. If someone has a GPU capture tool, a frame of this title is a good case. AI-assisted; the runs are from my machine. |
Castlevania: Belmont's Curse on this PR: about +67% presents per secondTitle: PPSA35399, 01.000.000 (Unity IL2CPP), in game after the opening cinematic ResultsEach row is one run, about 2 minutes in game, steady-state windows of 10 s.
Linux buildThe PR does not build on Linux as pushed: -#ifdef _WIN32
#include "prx/libc/include/HostThreadLocal.hpp"
+#ifdef _WIN32
#include <windows.h>Stability matches what I reported above: this run was clean for about 2 minutes on each build. AI-assisted; the runs are from my machine. |
With APS5_PIPELINE_DMA=1 a DMA_DATA was enqueued before the packet path's metadata notes ran, so a fill of 0 over a depth surface's HTILE (boykopovar#374) or a color target's CMASK (boykopovar#419) never reached the surface: agc_driver_depth_fast_clear_submit failed. The queued commit now notes the fill in order before it stores it, and a DMA copy is checked against multisampled targets at enqueue, as on the packet path. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> (cherry picked from commit fda7dbf)
Queue 0 now pipelines draw commits unless APS5_PIPELINED_DRAWS=0, and draw write ranges use the exact color, DCC, CMASK, depth and stencil sizes unless APS5_EXACT_DRAW_WRITES=0. The base tests now run the pipeline; agc_driver_pm4_failure_inline and agc_driver_draw_cache_reuse_inline keep the inline path covered. Co-Authored-By: Claude <noreply@anthropic.com>
c653e9e to
0340145
Compare
adriadam10
changed the title
|
Rebased after #1201 merged and reshaped (details in the description): this PR is now just the default switch ( |
…#1202); gap 2 applies to the default path
Brings the series onto main 4e9a2a8 (boykopovar#1193, boykopovar#1201, boykopovar#1202, boykopovar#6060, boykopovar#6333 and the descriptor word tracking of 682b07e). Resolutions: - ShaderMemory: the column memo's supplied and spilled blocks now sit on boykopovar#1193's page vector, page pool and word masks. Pages are found by binary search, since a spilled block can add many pages, and a block placed into a lazily read page fills the rest of the 16-word blocks it touches, so a later lazy read never overwrites placed words. - ResourceMaterializer and SharedValues: the selected-record path and the words index carry availableDwordMask, as the snapshot hash does. A selected record's dwords past the end of its buffer count as read zeros, which is what a scalar buffer load returns there, so a key past the heap still selects a null image without reading the heap. - DescriptorCache: kept table sets live beside boykopovar#1201's recycled sets. - Tests and CMake: both sides' tests are kept.
Upstream main now carries the final versions of PRs boykopovar#1193, boykopovar#1201, boykopovar#1202, boykopovar#5216 and boykopovar#4465, so their draft copies on this branch give way to main's: the snapshot ring, the pipeline identity memo, the code-hash memo and APS5_RESIDENT_LINEAR_TARGETS are gone with them. The port of boykopovar#4943 (draw-cache relocation across moved user-data pointers, not merged upstream) is dropped: the draw prefetch replaces it. Kept on top of main: the arm64 FEXCore runner and macOS libraries, the MoltenVK settings, rect-list clip and cull distances, GPU-ordered in-place vertex inputs, the draw pipeline's spin-first locks and lock-free overlap check, SharedVector binding flags, the result-memo admission, the draw prefetch (APS5_DRAW_PREFETCH) and the 1D T# sample modes. New for arm64: fegetenv from the runner's guest control words, and the direct-memory reuse map on macOS. On macOS the guest buffers keep one descriptor array (no buffers_coherent or buffers_u8 aliases): MoltenVK aliases a descriptor array only through Metal argument buffers, which the driver turns off, and Vampire Crawlers' first compute pipeline failed to compile.
Brings the series onto main 4e9a2a8 (#1193, #1201, #1202, #6060, #6333 and the descriptor word tracking of 682b07e). Resolutions: - ShaderMemory: the column memo's supplied and spilled blocks now sit on #1193's page vector, page pool and word masks. Pages are found by binary search, since a spilled block can add many pages, and a block placed into a lazily read page fills the rest of the 16-word blocks it touches, so a later lazy read never overwrites placed words. - ResourceMaterializer and SharedValues: the selected-record path and the words index carry availableDwordMask, as the snapshot hash does. A selected record's dwords past the end of its buffer count as read zeros, which is what a scalar buffer load returns there, so a key past the heap still selects a null image without reading the heap. - DescriptorCache: kept table sets live beside #1201's recycled sets. - Tests and CMake: both sides' tests are kept.
With the queue-0 draw pipeline on, the default since boykopovar#1202, the packet loop hands every label-writing packet to the pipeline and skips the rest of the loop. A RELEASE_MEM that completes a reserved EOP flip (sceVideoOutSubmitEopFlip, boykopovar#5539) is a label write, so its label landed but the flip branch never ran: the reservation was consumed and the flip never presented. video_out_flip_eop and video_out_flip_eop_main failed and video_out_flip_eop_aftergpu timed out. Keep packets that carry a reserved flip out of the pipeline. They take the normal path, which drains the pipeline before the flip as it does for flip packets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the queue-0 draw pipeline on, the default since #1202, the packet loop hands every label-writing packet to the pipeline and skips the rest of the loop. A RELEASE_MEM that completes a reserved EOP flip (sceVideoOutSubmitEopFlip, #5539) is a label write, so its label landed but the flip branch never ran: the reservation was consumed and the flip never presented. video_out_flip_eop and video_out_flip_eop_main failed and video_out_flip_eop_aftergpu timed out. Keep packets that carry a reserved flip out of the pipeline. They take the normal path, which drains the pipeline before the flip as it does for flip packets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
9 participants
What
Turns the queue-0 draw pipeline from #1201 on by default:
APS5_PIPELINED_DRAWSis on (depth 8) unless set to0.APS5_EXACT_DRAW_WRITESis on unless set to0: queued draws carry the exact color, DCC, CMASK, depth and stencil sizes instead of the estimates.DMA_DATAfill notes HTILE and CMASK fills in order, as the packet path does (Michon's commit;agc_driver_depth_fast_clear_submitfailed without it withAPS5_PIPELINE_DMA=1).APS5_PIPELINE_DMAitself stays opt-in here; making it the default is perf(agc): pipeline DMA_DATA when APS5_PIPELINE_DMA=1 #6208.agc_driver,agc_driver_pm4_failure,agc_driver_draw_cache_reuse, ...) now run with the pipeline on, as asked in review;agc_driver_pm4_failure_inlineandagc_driver_draw_cache_reuse_inline(APS5_PIPELINED_DRAWS=0) keep the inline path covered.Reshaped after #1201 merged. Left out of the old head (backup:
adriadam10/AnyPS5branchbackup/MichonGoddijn231849-perf-draw-path-trims, c653e9e):main's own unwritten-snapshot and moved-word caches (98ce8af, b0001cb), and the arena goes in its own PR as agreed above;agc_driver/agc_driver_pm4_failurewith the pipeline off): a failed commit now goes throughReportFailure(perf(agc): pipeline queue-0 draw commits behind the packet worker #1201), so they pass with it on;Open review points from the thread: the loading hang (fill under the GPU lock) is fixed by c653e9e in #1201; failed commits fail like the inline path (#1201); the CB metadata pass drains first (#1201, c50e231). The CMASK-of-DCC abort fma-rc saw on fda7dbf has no rerun since.
Tested
Linux, Ryzen 5 7535HS / Radeon 660M, RADV, GCC,
ANYPS5_ENABLE_SPIRV_TOOLS=ON,ctest -R 'agc|shader'(595 tests):APS5_PIPELINED_DRAWS=0: 594/595APS5_PIPELINE_DMA=1: 593/595; the extra failure isagc_driver'scopiesToHostMemorycount check (the copy is enqueued, the bytes are right), which is perf(agc): pipeline DMA_DATA when APS5_PIPELINE_DMA=1 #6208's to settle--repeat until-fail:20over the pipelined, fill, predication, label, draw-cache, failure, fast-clear and DMA tests withAPS5_PIPELINE_DMA=1: 33/33The only failure in every mode,
agc_driver_tessellation, segfaults onmaintoo on this GPU. Windows full build dispatched next.Game reports on the pipeline (on #1201's c653e9e): mrfelek Diablo II +51 %, Castlevania +67 % presents/s; JAB-dev Besiege 20→33, HSG2 19.5→36.6, Vampire Survivors 25-28→48-58. @fma-rc, an Astro's Playroom run on this head would tell whether the CMASK-of-DCC abort is still there.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt.mdfiles or images added to the repository (attach them to this PR)