Repository navigation
Conversation
… submission binds Driver::Submit waited for every registration preparation queued before it. On a cold shader cache, Ghost of Yotei's first submission waited 76 s for one compute shader it never used. The submission now reads the program address registers its commands write (SET_SH_REG and indirect SH register lists) and waits only for those programs' pending preparations, moving them to the front of the queue. Programs it cannot see still wait in the queue worker at first use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I traced the new scan against the driver paths: I did not find a concrete correctness issue in the diff. I attempted the focused Linux build locally, but GCC 12 stops earlier in unchanged |
|
One small regression-coverage gap: in Could the fixture queue the bound preparation through |
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
jotitadev
left a comment
jotitadev
left a comment
There was a problem hiding this comment.
A Linux run of the regression, which the description (Windows) and the review above (GCC 12 stopped before it) do not have.
Machine: NVIDIA GeForce RTX 3070, driver 595.99.02, Vulkan 1.4.329, plus lavapipe (Mesa 26.0.3, LLVM 21.1.8) on the same machine. Ubuntu 26.04, i7-8700, GCC 15.2, Release, BUILD_TESTING=ON. Head 20cc5ae56 merged into main 452494439 (clean merge). The build of PreparedShaders.cpp and the changed driver sources goes through on GCC 15.2.
agc_driver_prepared_shaders: 3/3 pass on the RTX 3070 (Selected GPU name=NVIDIA GeForce RTX 3070) and 3/3 on lavapipe, about 3.4 to 3.9 s per run (1.1 s onmainwithmain's test).^agc_driver_(prepared|registered), 18 tests,-j1: 18/18 on both devices, no skips.- No negative control: with
main'sSubmission.*andShaderRegistry.*the new test does not compile (ReferencedProgramsis not a member ofAgcDriver::DriverDetail).
This does not address the coverage gap raised above (the bound snapshot's own task queued in RegistrationPreparePool); I did not try that variant.
Not run: Windows, AMD, the full suite, any title.
GPUs tested: NVIDIA GeForce RTX 3070 (595.99.02), lavapipe (Mesa 26.0.3, LLVM 21.1.8), Ubuntu 26.04
Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.
…-wait test ReferencedRegistrationPreparation settled the bound snapshot from a side thread, so the pool never held its task and the prioritization in AwaitReferencedPreparations went untested. The test now occupies every preparation worker, queues the unrelated preparation and then the bound one through SubmitRegistrationPreparation, and frees a single worker once the wait has demanded the bound preparation. The bound task must run first, the unrelated one must still be pending when the wait returns, and a later wait on the unrelated address must settle it. Without the PrioritizeRegistrationPreparation call the test fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, good catch. 7241b05 now queues the bound preparation through |
|
Timed this on Ghost of Yōtei (PPSA26344) on Linux with lavapipe, cold shader cache on every run, base and PR runs interleaved: Base averages 221 s and the PR 171 s (2 runs each), so the first frame came about 50 s (23%) earlier. The ranges do not overlap (base 211–231 s, PR 161–181 s). The time comes from a memory probe that logs every 10 s, so each value can be up to 10 s after the actual first frame. Setup: both builds are For comparison, your Windows runs had the first frame at 40 s with this PR against 101 s on AI-assisted: yes |
|
Data point: RTX 3060 12 GB (NVIDIA 617.42), i5-12400F, Windows 10 19045, MinGW GCC Release. Minecraft Dungeons II (PPSA16064 01.100.100), title screen, no input, 200 s rows,
Neutral on this scene (mean +0.7 %, within the row spread); no regression. 0 fatal. AI-assisted: yes (Claude Code) |
github-actions
Bot
added
perf
|
I have an independent follow-up in progress for #8470: at shutdown, finish the running library prebuild but discard queued speculative prebuilds, leaving registration preparation and device-replacement waits unchanged. The candidate touches |
CnRJay
left a comment
CnRJay
left a comment
There was a problem hiding this comment.
The result is large and measured (Ghost of Yotei, cold cache, first frame 101 s to 40 s, first submission wait 76.3 s to under 3 s), and moving a bound program's preparation to the front of the queue (#4404) is the right use of the scan. The part I would not merge as is is the fallback.
- #5007 made
Driver::Submitwait for every queued preparation for one reason: a title that sees a submission take seconds treats the GPU as hung. Goat Simulator 3 (PPSA04159) failed a 10 s timeout when the wait sat in the queue worker. Waiting "in the queue worker at first use" is what this PR leaves for any program the scan misses: bound by an earlier submission, a half-written address, an address reached another way, or an indirect buffer the scan didn't follow. Those cases reproduce the Goat Simulator situation (the GPU side of a submission blocked on a long preparation), and the PR does not say how long such a wait can be. The cold-cache 55,144-dword shader this was found on took 74 s on one thread. If a program the scan misses is bound by a draw, the queue worker blocks for that long. - Please add a test where the program is reachable only through a path the scan doesn't see (address written in two halves across submissions; address in an indirect buffer), with a slow preparation, and state the resulting wait. If it is long, the scan needs the missing paths, or the fallback needs a bound (after N seconds, report and continue).
- The 2.6 s longest wait for 83 bound programs is still a seconds-long submit. It is under the 10 s limit that failed, but the margin is a property of this title on a 4090; a slower machine's cold cache would be longer. State the worst wait by machine, or the time the scan assumes.
- The numbers are from a tree with #6227 and two local diagnostics. The runs are cold-cache only. Post a warm-cache pair as well to show the scan costs nothing when preparations are already done (
ReferencedProgramswalks every copied command on every submit). - Marked conflicting with
main; rebase before it can be tested again.
This branch has not been deployed
7 participants
Fixes #6846.
What
Since #5007,
Driver::Submithas waited for every registration preparation queued before the call, even those of shaders its commands never use. On a cold shader cache this held Ghost of Yōtei's first submission for 76 s, waiting on one compute shader (0x8000083400, 55,144 code dwords). That shader spent 74 s in the recompiler front end on one thread and was never needed during boot.With this change, a submission waits only for the programs it binds. While it copies the command stream,
Driver::Submitalready follows indirect buffers and reads the indirect register lists.ReferencedProgramsscans those copied commands for the program address registers the submission writes:SPI_SHADER_PGM_LO/HIfor the pixel, geometry-back, export/vertex, hull and local stages (SH 0x008, 0x088, 0x0c8, 0x108, 0x148);COMPUTE_PGM_LO/HI(SH 0x20c).These come from
SET_SH_REGpackets and indirect SH register lists (0x63). The address is built the wayDecodeGraphicsProgramsand the dispatch path build it.AwaitReferencedPreparationsthen looks up the registered snapshots that cover each address. For each one whose preparation is still pending, it moves the preparation to the front of the queue (#4404) and waits for it on the submitting thread. The Goat Simulator 3 case from #5007 stays covered, because there the submission binds the shaders it registered just before.A program the scan does not see still waits in the queue worker at first use, through the existing
AwaitRegisteredPreparation. That covers programs bound by an earlier submission, a submission that writes only one half of an address, and an address reached another way.The pool's sequence tracking (
AwaitSubmitted,unfinished) was used only by the old wait, so it is removed.Measured
Ghost of Yōtei (PPSA26344), RTX 4090 (driver 617.42), Windows 11 Pro 26200.
main82ddae7 plus fix(libSceNetCtl): answer the DHCP host name, PPPoE name and proxy server as empty strings #6227 (without it the title aborts onsceNetCtlGetInfoinfo code 12 at 3 s) and two local diagnostics: a crash-report thread dump and the timing lines below.mainmain+APS5_LAZY_SHADER_PREPARE=1main+APS5_LAZY_SHADER_PREPARE=1main: frame 619 comes 80 s sooner (101 s against 181 s), the same as lazy preparation, and preparation stays eager.mainthe first frame came at 181 s (run180), 80 s (run182) and 101 s (run184) in three cold runs, so themainrow is one sample of a wide spread. Both runs of this PR drew their first frame at 40 s.Where main's two long waits come from (run184)
Driver::Submit→AwaitRegistrationPreparation, one prepare thread inStructurizer::computeDominatorTree, and the other 16 idle.A bounded wait, tried first (not part of this PR)
Ending the submit wait at 2 s per call (run185) still left 51 waits totaling 100 s: every submission paid the full 2 s while the giant was running. Ending it 2 s after the newest queued preparation (run186) left 16 waits totaling 20 s, with frame 619 at 101 s. That version's 2 s is a guess, and a burst of registrations still costs up to 2 s per submission. This PR waits only for what the commands bind, as suggested in #6846.
Tests
PreparedShaders.cppreplaces the oldAwaitedRegistrationPreparationcase with two new ones.ReferencedRegistrationPreparation:ReferencedProgramsfinds a compute program bound withSET_SH_REGand a vertex program bound through an indirect register list, and ignores an address whose high register is not written.SubmitRegistrationPreparation. One worker is freed onceAwaitReferencedPreparationshas demanded the bound preparation.PrioritizeRegistrationPreparationcall removed fromAwaitReferencedPreparations, this case fails.UnrelatedPreparationAtSubmit:sceAgcDriverSubmitDcbis called with a 1.5 s unrelated preparation queued, and must return in under 1 s, before that preparation runs. With the old wait it would block for the full 1.5 s. I haven't run this test against the old code.Tested
Windows 11 Pro 26200, RTX 4090, driver 617.42. This branch at 20cc5ae on
maine0e3ced:agc_driver_prepared_shaders(4.2 s) and its indirect, deferred, lazy and registered-workgroup variants.misaligned_writer_aheadis not built: main'sGpuIndirectArgumentsMisalignedWriterAhead.cppneeds 65536-byte alignment, which MinGW's object format does not allow.agc_driver_prepared_shaderstests pass, and the base test passed 10 runs in a row. With the prioritization call removed, the base test fails with "the queued preparation of a bound program was not moved ahead of unrelated queued work".Technical debt
The entry "Registration preparation at submit" in
docs/dev/debt/libSceAgcDriver.mdnow records:Related
ShaderRegistry.cpp. Each merges cleanly with this branch.0x8000083400is a separate recompiler cost. This change only stops unrelated submissions from waiting for it.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code