Repository navigation
Shader registration runs the recompiler front end under a process-wide lock even on disk-cache hits #2292
Description
Summary
Shader registration is serialized by ShaderPreparationTransaction::State::writers, a process-wide mutex held for the whole Driver::RegisterShader. A registration still runs the full recompiler front end under that lock when the shader disk cache already has the variant, because getSource builds the ResourceProgram before findOrCompileVariant consults the disk cache. A retail Unreal Engine title registers its shaders from UE worker threads while loading. On this title that adds up to more than 30 s of serialized work. The UE game thread waits on those workers, and UE's hang detector (FThreadHeartBeat, which deliberately writes 0xE0000001 to address 3) kills the process.
I'm opening this as an issue rather than a PR because the fix depends on what the transaction lock is meant to guarantee.
Measurements
Windows 11, MinGW-w64 GCC 15.2.0, RTX 5090. Base is main d70b8998 + #1836 + #2141 + CouchRiv's #1789 branch (bd5f22c9) + #2105's change. The shader disk cache was warm (0 misses), and each run hit the hang.
- CDB at the hang: 11 UE workers (
Foreground/Background Worker #n) are blocked inShaderPreparationTransaction::ShaderPreparationTransaction(). The one running worker is inside recompiler IR passes (IrValue::ReplaceUsesWith). - Local timing on
State, logged from its constructor to its destructor: transactions held ≥ 250 ms add up to 32.5 s in one loading sequence (113 such transactions). All come fromDriver::RegisterShader. - Registration times with a warm cache are typically 0.3–0.8 s for 15–30 KB compute shaders. Almost all of it is the front end in
PrepareRegistered→PrepareShader→getSource→ResourceProgram: decode,GraphBuilder,Structurizer, translation and optimization. - A compute registration runs
Structurizeup to three times: theGraphBuilder/Structurizerpre-check inPrepareRegistered(its graph is discarded), thengetSourceonce per variant, since thepartialThreads = {1,1,1}variant gets its own source key. Before perf(recompiler): compute post-dominators visiting successors first #2105, three shaders with a pathological post-dominator case took 3 × 6.2 s = 18.5 s each under the lock.
Possible directions (for maintainers to judge)
- Do the front end (and disk-cache lookup) outside
State::writers, and take the lock only to publish intoPreparedShaders, re-checking for a concurrent publication. - Let the two compute variants of a registration share one source entry and front-end result.
- Reuse the
PrepareRegisteredpre-check's graph instead of rebuilding it inPrepareResourceProgram. - Persist enough of the front-end result (resource plan) for a disk-cache hit to skip it.
Related: #1789 (unpreparable shaders are never cached, which made this worse), #1836 (deferred preparation), #2105 (post-dominator iteration order).
Investigated and written by Claude (Claude Code) on a contributor's machine.
Activity
ThatsLeo commentedon Oct 11, 2026
Fix:
ThatsLeo/AnyPS5:perf/agc-shader-registration-outside-lock(one commit,d64c72d3, onmainf1ae427b).#2583 moved the registration-time preparation off the lock, but ABI resolution (
ResolveShaderAbi,ResolveGraphicsStagesAbi, reached fromsceAgcLinkShaders,sceAgcCreateInterpolantMapping*andsceAgcCreatePrimState) still ranPrepareShaderWithDiagnosticswhile holdingState::writers. Most graphics shaders are prepared there, since registration defers linked vertex programs and pixel programs with interpolants.What
writersguarantees is isolation between transactions:Read/Editcopy the prepared state andCommitswaps it back, so two transactions must not interleave on the same snapshot. Neither the preparation itself nor the reuse lookup needs it (the background pool already prepares without it).writersat its firstReadorEditinstead of in its constructor.RegisterShaderandPublishRegisteredShadertake it at once withLock(), so a registration still waits for an unfinished preparation, asagc_driver_prepared_publicationrequires, and the order stayswriters→Driver::mutex→PreparedShaders::mutex.Peek()reads the published state under the snapshot's own mutex (or the transaction's own changes) withoutwriters.ResolveShaderAbiandResolveGraphicsStagesAbicheck withPeek, prepare outside the lock, thenReadand check again before publishing, so a concurrent resolution of the same ABI publishes once.Tested on Fedora 44, GCC 16.2.1, Release, Intel Arc (Mesa 26.2.3) and lavapipe:
agc_driver_prepared_shaders: a resolved ABI does not wait for an unrelated open transaction, a new ABI is not published inside one, and four racing resolutions publish. It fails onmain("a resolved shader ABI waited for an unrelated shader preparation") and passes 30/30 on the branch. On Intel Arc this binary stops earlier onmaintoo (VK_EXT_depth_range_unrestricted), so the case ran on lavapipe.libtsan,RelWithDebInfo):agc_driver_prepared_publicationandagc_driver_prepared_shaders10/10 with no report.ctest: the same failures asmainf1ae427bon both devices (Arc 7, lavapipe 2).Stray (PPSA02101 v01.002, UE4), warm shader cache, a counting wrapper around
writers(not in the commit). Both builds load the same ~3,300 cached shaders by about 41 s. Onmain, the log then shows a rect-list draw prepared at draw (about 42 s), followed by a singlewritershold of 45 to 96 s, 10 to 16 contended acquisitions and one new shader compile every 12 to 24 s; I read that as the render thread waiting behind ABI preparations of a few slow shaders, but I did not capture its stack. With the change, the stopping draw follows the rect-list one by about 1.7 s. Time until the title reaches the draw that stops it on #2881's error, which this change does not affect:mainf1ae427b(Two runs of each also carried
0467f908from #2881; it does not change this stop.)Left as it was:
ResolveGraphicsAbiwaits for a pending background preparation while the scope opened bysceAgcCreateInterpolantMapping_0100already holdswriters(a 30 s hold in Stray on both builds, with no other waiter there), andResolvePreparedGraphicsstill builds rect-list shaders under it. @mrk1987, a run on your title would show whether this covers the stall you measured.AI-assisted: yes.
jweidenh commentedon Oct 11, 2026
The remaining requirement for #2363 is startup measurements on the original affected title, including whether it gets past the
FThreadHeartBeatkill. We do not have that title or its reproduction locally.Could the original reporter, or someone with the same affected title, compare these two revisions and post the results here?
e8d6de4ae7aeb5e7c0ec40d85371e39113724cb2.58d31147cc886cf0e833dc89a3f69e460fa424c3, the same base plus only perf(agc): prepare registered shaders outside the publication lock #2363. Both include upstream asynchronous preparation from perf(agc): prepare registered shaders in the background #2583.Please use the same Release toolchain, title/version, launch steps, settings and cache condition in both builds. If the title needs additional bring-up patches, apply the same commits to both and list them. Report the GPU/toolchain, per-run time from launch to the same named startup milestone, and whether each build reaches that milestone or still hits
FThreadHeartBeat. Repeated runs would help if practical. If startup ends at a different failure, please identify that endpoint rather than treating it as successful startup.The regression tests and historical synthetic benchmark do not establish this title's outcome. No game files need to be shared.
mrk1987 commentedon Oct 11, 2026
@jweidenh Measurements from the original title, STAR WARS: Galactic Racer (PPSA28416). Windows 11, RTX 5090, MinGW-w64 GCC 15.2.0, Release.
Base. The two revisions you named don't reach this title's startup without its bring-up patches, and those don't apply to
e8d6de4a. So both arms use the same newer base:main1e8fff68plus the same bring-up commits. The only difference between the arms is #2363's five commits (3c855173..58d31147), which cherry-pick cleanly.Bring-up commits in both arms:
fe805bd0), and johnmaia'sd06f097f847cccc63313464f0beaafe6ds_ordered_count(add, counter 0)s_swappc_b64targets loaded through descriptorsSettings in both arms.
APS5_PROFILE_DRAW=1,APS5_NO_SNAPSHOT_CHECK=1.uecommandline.txt, soFThreadHeartBeatis active.Milestone. The first
[present]statistics line, measured from launch. It is printed once per 10 s window that contains presents, so this is a coarse upper bound on the first present.Neither arm hits
FThreadHeartBeatwith warm caches, and the times are within noise of each other.The original stall no longer occurs on this title. The CDB capture in this issue showed 11 UE workers queued on
ShaderPreparationTransaction::State::writers. On the current base with cold caches, the title still gets killed byFThreadHeartBeat, but for a different reason:RegistrationPreparePool::Main, and no thread waits in the transaction.Driver::holdFlipBehindWorker, and the GPU worker is inDriver::waitMemory.nvgpucomp64.dll), building a compute pipeline of about 0.7M SPIR-V words. Pipelines of 0.8M to 2.8M words take 7.7 to 22 s each here.So on this title, the registration-lock stall seems to have been resolved by #2583's background preparation. The remaining startup kill is pipeline compile time, which #2363 does not address. These runs can't show a benefit for #2363 on this title, and they show no regression either.
Unrelated test flake. While running the registration tests, one of
agc_driver_prepared_shaders_debug,agc_driver_prepared_shaders_deferredandagc_driver_compute_user_data_calls_registeredsometimes exits with heap corruption (0xc0000374) when they run in parallel (ctest --parallel 8). Of 10 batches, 3 had a failure with #2363, 3 on the same base without it, and 3 on cleanmain1e8fff68(all inagc_driver_prepared_shaders_debugthere). #2363 doesn't cause it.AI-assisted: yes (Claude Code)
adriadam10 commentedon Oct 11, 2026
#3717 adds opt-in capture and replay of compute pipeline compilation as a diagnostic; the registration lock and the NVIDIA compile stall reported here are still open.
oneandonlydean commentedon Oct 14, 2026
Status: #4883 is on main, so a disk-cache hit now reuses the stored front-end resource plan instead of running the front end again. The process-wide registration lock itself is still open.
perf(recompiler): share the front end between the two compute variant…
perf(recompiler): share the front end between the two compute variant…
perf(recompiler): share the front end between the two compute variant…