[gjonn]

What

Rebased onto current main as one commit. #2583 has landed, so its commit is gone from this branch.

Driver::ResolveShaderAbi for compute and pixel shaders (types 0 and 1) compiles each new ABI variant on the guest thread that calls it. Titles resolve hundreds of these in a row from a loader thread through sceAgcCreateInterpolantMapping, so that thread spends its time compiling. Since #3159, the compile no longer runs under the preparation writer lock. It still runs on the calling thread, one variant after another.

With this change, ResolveShaderAbi does the following instead:

• Returns at once if the ABI key is registered, pending or failed. The registration job's own key counts as pending, so a resolution that arrives while registration prepares the same state queues nothing.
• Otherwise it runs PlanRegistered on the calling thread, so register and validation errors are still reported by the call. It then records the key in pendingAbis, queues the compile on the registration preparation pool as an AbiPreparation, and returns.

Waiting for ABI jobs:

• Draws, dispatches (SourceHandleFor, InvocationFor) and transaction Read / Edit wait for the snapshot's unfinished ABI jobs (abiPreparations).
• A waiting thread that finds a job the pool has not started runs that job itself, so it never waits behind other shaders in the pool queue.
• Read / Edit finish these jobs before they take the preparation writer lock, so no thread holds that lock while it waits on the pool.

A failed ABI job does not reach the guest:

• It is logged, and its key moves to failedAbis, so later resolutions of that key return at once.
• The snapshot stays usable. A draw or dispatch that needs that state prepares it at use, through main's existing prepare-at-use path, and fails there if the shader cannot be prepared.
• This is the same handling a deferred registration gets.

APS5_SYNC_SHADER_PREPARE=1 keeps the synchronous path. ResolveGraphicsStagesAbi and the rectangle-list path are unchanged. docs/dev/ARCHITECTURE.md describes the background ABI resolution and the job list.

Tested

Unit tests. New cases in agc_driver_prepared_shaders:

• --background-abi, run as agc_driver_prepared_shaders_background_abi (default) and _background_abi_sync. It registers a pixel shader whose code fails only once it is built for its ABI.
  • Default mode: ResolveShaderAbi returns, and repeated calls for 500 ms never throw.
  • Sync mode: the call throws as before.
  • On main, the default case fails: ResolveShaderAbi throws branch at program counter 0x00000000 targets invalid program counter 0x00000008 into the caller. This was checked by building the same test against main c13dc7e on Linux.
  • The same mode also dispatches a compute shader right after resolving a new ABI for it, and checks that the dispatch's memory write lands.
• QueuedAbiPreparation, in the main run:
  • A queued ABI job that the pool has not started is run exactly once by the first SourceHandleFor / InvocationFor, and its artifact is reused.
  • A failed ABI job is recorded in failedAbis. It leaves the snapshot readable, and the dispatch prepares at use instead of rethrowing.

Results:

• Windows 11, MinGW-w64 GCC 15, NVIDIA RTX 5090 Laptop GPU, Release: full ^agc_ CTest suite, 452/452 passed (1 skipped).
• Linux (Ubuntu in WSL2, GCC 15.2, Mesa lavapipe), Release: agc_driver_prepared_shaders in all modes passed, including both new ones.

Synthetic before/after on Linux. Same machine and build flags; main c13dc7e against this PR, alternating runs, 5 each. Ubuntu in WSL2, GCC 15.2, Release, ANYPS5_ENABLE_SPIRV_TOOLS=OFF, Mesa lavapipe, 24 threads. The harness source is in the review reply below.

The harness does the following:

• Registers 96 pixel shaders with one interpolant (0x1b6 = 1), so registration does not compile them.
• One thread then registers 192 compute shaders, which floods the pool.
• At the same time, a loader thread resolves the 96 pixel ABIs. Each shader is a 600-instruction VALU chain, about 50 ms of compile.
• "loader" is the loader thread's loop. "total" runs until shutdown has drained the pool.

Scenario	main loader (s)	PR loader (s)	main total (s)	PR total (s)
resolve burst	5.34 4.39 4.42 4.27 3.17 (median 4.39)	0.03 0.03 0.02 0.02 0.03	5.35 4.39 4.42 4.27 3.17 (median 4.39)	1.96 1.89 1.88 1.92 1.56 (median 1.89)
+ 20 ms of other loader work per call	5.07 5.31 5.44 7.15 6.60 (median 5.44)	1.94 in every run	5.08 5.31 5.44 7.15 6.61 (median 5.44)	1.94 1.95 1.96 2.07 1.97 (median 1.96)

Every PR run finished before every main run.

The earlier Linux slowdown (PPSA11386). It was measured on the previous version of this commit. That version had two places where waits got longer:

• Transaction Read / Edit took the global preparation writer lock first, then waited for the snapshot's in-flight jobs. An ABI job queued behind a registration burst kept every other registering thread blocked until the pool reached it.
• Draws waited for queued ABI jobs instead of running them.

This version waits before it takes the lock and runs a job that is still queued on the waiting thread. I have not re-run that title. @SleglJan, if you can repeat your series on this head, that would confirm it.

Not tested here: a performance trace of a title scene in the form the template asks for.

agc_driver crash in default mode. It does not reproduce on current main or on this head:

• ctest -R '^agc_driver$': 0/30 on this PR and 0/30 on main c13dc7e on Linux (WSL, lavapipe). 0/30 on this PR on Windows (RTX 5090).
• On main alone, also 0/480 in parallel runs and 0/96 under ASan.

The likely cause was the one #2976 (2dd453d) fixed:

• Driver::stop reset the device without waiting for the registration pool.
• A job still queued or running then read the freed SpirvTarget extension names at exit.
• Sync mode never crashed because it has no pool jobs.

#2976 merged after the first review's runs, and this branch is based on it. CloseRegistrationPreparation drains ABI jobs as well, because they run on the same pool.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: none verified. PPSA01317 Gran Turismo 7 (PPSA01317) #4697 gets faster menu loads. For PPSA14632, the failure @alfalfa-star described no longer reaches the guest from sceAgcCreateInterpolantMapping, but I have not run that title.
• AI-assisted: yes

[adriadam10]

Thanks. I rebased only this PR's commit onto current main (#2583 landed with a different commit, so it applies cleanly) and ran the libSceAgcDriver tests. Findings:

• agc_driver segfaults intermittently in the default (background) mode: 10/30 runs on this PR, 0/30 with APS5_SYNC_SHADER_PREPARE=1. Plain main (#2583) does the same, 8/30 default and 0/30 sync, so this PR didn't introduce it. But it adds more concurrent preparation (pendingAbis, abiFailures, count of in-flight jobs) on top of a crash we can't yet explain, so we can't merge it until the default mode is stable. Could you find the cause (a backtrace of agc_driver_tests in default mode would be enough to start)? A loop like for i in $(seq 30); do ctest -R '^agc_driver$'; done reproduces it for us.
• The new cases in PreparedShaders.cpp only compile against the new members, so they don't show the behaviour differs from main. Please add one that fails on main for observable behaviour (e.g. ResolveShaderAbi returning before the failure is raised).
• The speed-up numbers are from your Windows/RTX 5090 run only; we can't reproduce them here. A synthetic before/after on Linux (same harness, several runs) would back it up (#218).

Please also add a short paragraph to docs/dev/ARCHITECTURE.md (shader section) covering the background ABI resolution and the in-flight job count.

[alfalfa-star]

A data point on the failure handling here: some titles call ResolveShaderAbi from game code, and re-raising a stored failure there ends the title.

On LEGO Horizon Adventures (PPSA14632), the call comes from sceAgcCreateInterpolantMapping on a game thread (CreateInterpolantMapping → Driver::ResolveShaderAbi). On main 6cc3358, one pixel shader there fails the recompile with #2548's packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?). The exception goes up through libSceAgc into the game and terminate runs, about 5 s after boot (stack in #2548).

With this PR, if I read it right, the first call queues the job and returns, but a later ResolveShaderAbi call for the same key rethrows the stored error, which reaches the same caller. Would it make sense for a failed ABI job to behave like a deferred registration instead (logged, then prepared or refused at the draw that uses it)? Locally, marking the snapshot deferred and recording the key on failure gets LEGO past that shader.

[Mouhand-Kaddo]

Additional platform validation on standalone PR head 4d8dd9dbbc9f6dc5c66b026cf1a8c2614dba2fe6:

Linux: passed — Arch Linux in WSL2 (kernel 6.6.87.2), GCC 16.2.1 20260810; CMake 4.3.1 / Ninja.

Release / BUILD_TESTING=ON: built the selected test targets; 6 CTest cases, 0 failures, 0 CTest skips.

Tests: agc_driver_prepared_shaders, agc_driver_prepared_shaders_indirect, agc_driver_prepared_shaders_uncached, agc_driver_prepared_shaders_debug, agc_driver_prepared_shaders_debug_failure, agc_driver_prepared_shaders_deferred.

Reproduce after building the test targets: ctest --test-dir <build> --verbose --no-tests=error --timeout 30 -R "^agc_driver_prepared_shaders".

Vulkan: Mesa lavapipe 26.2.4 software Vulkan; no native Linux GPU/retail-title validation.

Integration limitation: a local merge against main 948a526c2 has source conflicts; these results validate the standalone PR head, not a resolved merge.

[SleglJan]

Data from 007 First Light (PPSA11386 v01.018.000) on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ryzen 9 7950X3D, RTX 3080 with NVIDIA 615.71.09), Release with ANYPS5_ENABLE_SPIRV_TOOLS=OFF. This measures something different from the description (PPSA01317 menu load, warm caches): 007 presents no frame yet, so it covers only start-up to its first flip.

This head does not merge with current main, so I measured its last commit, 4d8dd9d, cherry-picked onto main 3d1a7d9, which has #2583 as merged. Only tests/execution/PreparedShaders.cpp conflicted; it stays at main's version. ShaderRegistry has not changed on main since. Both builds also carry the changes listed in #2523 that get 007 to its first flip, including that issue's local experiment raising ShaderInfo::MaxImages from 64 to 128 (row 13).

007 currently stops while presenting that flip, on a bindless image table limit. The measure is therefore the process lifetime from exec to that stop. It covers boot, the shader registration burst and the first draws (40 to 46 traced per run). Runs alternated between the two builds.

Build	Shader cache	Runs	Seconds	Median
main + stack	cold	10	25.6 26.6 26.8 27.0 27.5 27.5 27.6 28.4 28.8 30.7	27.5
+ 4d8dd9d	cold	8	29.1 29.3 29.7 29.8 29.8 30.3 30.8 32.5	29.8
main + stack	warm	4	24.1 24.3 24.9 25.3	24.6
+ 4d8dd9d	warm	4	26.1 26.1 26.8 27.4*	26.5

* This run overlapped a git checkout and a CMake configure on the same machine; without it the median is 26.1.

• With this commit, 007 reaches the same stop later by about 2.3 s from a cold cache and 1.5 to 1.9 s from a warm one (medians). Warm runs have 1,712 to 2,027 cache hits and 32 misses. Every warm run with the commit is slower than every warm run without it.
• An earlier series on 007 First Light (PPSA11386) on Linux: start-up stops in order and the PRs that pass them #2523's explore branch went the same way: a median of 38.4 s without the commit and 41.8 s with it, 5 runs each. That branch is d8cfc04: main 17d69d1, perf(agc): prepare registered shaders in the background #2583 at 7fdcc34 and both of that issue's local experiments.
• On the failure handling: with the MaxImages experiment, 32 of 007's shaders fail at registration with shader image heap capacity exceeded.
  • They are logged once per shader: could not be prepared after registration without this commit, could not be prepared in the background with it. Both builds continue to the flip.
  • With APS5_SYNC_SHADER_PREPARE=1 (two runs with this commit), 007 stops on the first of them, at the 733rd shader-cache miss.
  • No ABI-resolution job fails, and none of the stored failures reaches the game before the first flip, so 007 does not exercise the case @alfalfa-star describes.

AI-assisted: yes (Claude Code)

[adriadam10]

Thanks for the work here. Since there was no push after our last review, we looked at the new data in the thread. Still changes:

• Slower on Linux. @SleglJan measured this commit cherry-picked on current main (007 First Light, PPSA11386, RTX 3080, alternating runs): median 27.5 s → 29.8 s cold cache and 24.6 s → 26.5 s warm; every run with the commit is slower than every run without it. The Windows/RTX 5090 numbers in the description come from a local build that is not this branch. Please add a before/after that shows a gain on Linux, or explain where the extra time goes.
• Stored failure reaches the game. As @alfalfa-star describes, a later ResolveShaderAbi call for the same key re-raises the background failure into game code (sceAgcCreateInterpolantMapping), which ends the title where main would not have reached the call yet. A failed ABI job should behave like a deferred registration (logged, then prepared or refused at the draw that uses it).
• agc_driver segfault in default mode is still unexplained (10/30 here, 8/30 on main, 0/30 with APS5_SYNC_SHADER_PREPARE=1). A backtrace of agc_driver_tests in default mode would be a good start; this PR adds more concurrent preparation on top of it.
• The new cases in PreparedShaders.cpp only compile against the new members; add one that fails on main for observable behaviour.
• docs/dev/ARCHITECTURE.md (shader section) needs a paragraph on the background ABI resolution and the in-flight job count.
• The PR conflicts with main (ShaderRegistry.cpp, PreparedShaders.cpp): #2583 landed with a different commit, so please rebase onto it and drop its commit from this branch.

[blawesom]

Hi! Since #2583 merged, this PR's stacked base is gone from its diff and it now conflicts with main, so I rebased the new commit (perf(agc): resolve static shader ABIs on the background pool) onto current main as a single commit — #2583's original commit dropped, since main took it as 6db5c1369.

The rebase needed care because main evolved ShaderRegistry.cpp substantially after #2583 (ReportPreparedAtUse, SubmitRegistrationPreparation, fused hull headers, bit-word ABI walks). The merged result keeps main's newer structure and adds this PR's behavior on top:

• the abiFailures rethrow in SourceHandleFor/InvocationFor is kept, guarded to the original reachability (a settled, non-deferred snapshot with no matching entry) — main's newer prepare-at-use path handles everything else;
• PrepareRegisteredInBackground keeps main's SubmitRegistrationPreparation helper with this PR's added registration parameter and the pending/abi bookkeeping;
• the pendingAbis/abiFailures fields, the duplicate-entry suppression and the async ResolveShaderAbi path applied as in the PR.

It builds; agc_driver_prepared_shaders_tests passes (including BackgroundAbiResolution); full suite 832/832 on Linux (AMD RX 9070 XT / RADV); tools/check_conventions.py --base origin/main passes.

The rebased branch is available to pull:

git fetch https://github.com/blawesom/AnyPS5 rebase/pr-2589
git reset --hard FETCH_HEAD

Feel free to ignore this if you'd rather rebase yourself.

[gjonn]

Thanks for the reviews and the data. I pushed a reworked version, 2f9697b, as one commit on main c13dc7e. Per point:

• Rebase. Done. perf(agc): prepare registered shaders in the background #2583's commit is dropped. @blawesom, thanks for the rebase. main had moved about 1,300 commits past it (including perf(agc-driver): prepare shader ABI variants before taking the preparation lock #3159, which already moves the ABI compile out of the writer lock), so I rebuilt on current main.

• Stored failure reaching the game (@alfalfa-star's report). A failed ABI job is no longer rethrown from ResolveShaderAbi. It is logged and its key is marked failed, so later resolutions of that key return at once. The draw or dispatch that needs that state prepares it at use, through main's existing prepare-at-use path, and fails there if the shader cannot be prepared. This is the same handling as a deferred registration.

• Slower on Linux (@SleglJan's series). The old version had two waits that could run long:

  • Transaction Read / Edit took the global writer lock, then waited for in-flight jobs that were queued behind a registration burst. That blocked every other registering thread.
  • Draws waited for queued ABI jobs instead of running them.

  Now Read / Edit finish the snapshot's ABI jobs before taking the lock, and a waiting thread runs a job the pool has not started. A synthetic before/after on Linux is in the description (main total 4.39 s → 1.89 s median, every PR run faster than every main run). @SleglJan, if you get a chance to repeat your PPSA11386 series on this head, I'd like to know whether the gap is gone.

• agc_driver segfault. It does not reproduce on current main or on this head:

  • 0/30 on Linux for both. 0/30 on Windows for this head.
  • On main alone, also 0/480 in parallel runs and 0/96 under ASan, plus 0/90 on 9a1f6f4, the main at your second review.

  The likely cause is the one fix(agc): drain the registration preparation pool before the device is reset #2976 (2dd453d) fixed: the device was reset at shutdown while registration-pool jobs still read its SpirvTarget. That fix merged after your first measurement. ABI jobs run on the same pool, so CloseRegistrationPreparation drains them too. I couldn't verify on RADV. If it still shows up there, a backtrace would point me at it.

• A test that fails on main. agc_driver_prepared_shaders_background_abi registers a pixel shader that fails only once it is built for its ABI, and expects ResolveShaderAbi to return. On main it throws targets invalid program counter into the caller; I checked this by building the test against main on Linux. _background_abi_sync keeps the old throwing behaviour under APS5_SYNC_SHADER_PREPARE=1.

• ARCHITECTURE.md. Added a paragraph in the shader section.

Still missing: a performance trace of a title scene in the template's form. I have not captured one for this head yet.

Benchmark harness (built as a local test executable linked like agc_driver_prepared_shaders_tests; arguments: pixels computes length work_us)

#include "VulkanTestDevice.hpp"
#include "prx/libSceAgcDriver/Execution/include/Driver.hpp"
#include <algorithm>
#include <array>
#include <cstddef>
#include <chrono>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <memory>
#include <thread>
#include <vector>

namespace {

using Clock = std::chrono::steady_clock;

double Ms(Clock::duration duration) {
    return std::chrono::duration<double, std::milli>(duration).count();
}

std::vector<std::uint32_t> Chain(unsigned length, unsigned seed) {
    std::vector<std::uint32_t> code;
    code.push_back(0x7e0002ffu);
    code.push_back(0x3f800000u + seed);
    for (unsigned index = 0; index < length; ++index) {
        const std::uint32_t op = (index % 3u == 0u) ? 0x08u : (index % 3u == 1u ? 0x03u : 0x10u);
        const std::uint32_t dst = 1u + index % 6u;
        const std::uint32_t src0 = 256u + index % 6u;
        const std::uint32_t src1 = (index + 1u) % 6u;
        code.push_back((op << 25u) | (dst << 17u) | (src1 << 9u) | src0);
    }
    code.push_back(0xbf810000u);
    while (code.size() % 64u != 0u) code.push_back(0xbf800000u);
    return code;
}

struct PixelShader {
    Shader shader{};
    std::array<ShaderRegister, 3> registers{};
    std::array<ShaderRegister, 3> context{};
    std::vector<std::uint32_t> storage;
    std::uint32_t* code = nullptr;
};

struct ComputeShader {
    Shader shader{};
    std::array<ShaderRegister, 7> registers{};
    ShaderSpecialRegs specials{};
    std::vector<std::uint32_t> storage;
    std::uint32_t* code = nullptr;
};

std::uint32_t* Aligned(std::vector<std::uint32_t>& storage, const std::vector<std::uint32_t>& code) {
    storage.assign(code.size() + 64u, 0u);
    auto address = reinterpret_cast<std::uintptr_t>(storage.data());
    address = (address + 255u) & ~std::uintptr_t{255u};
    auto* aligned = reinterpret_cast<std::uint32_t*>(address);
    std::copy(code.begin(), code.end(), aligned);
    return aligned;
}

std::unique_ptr<PixelShader> MakePixel(unsigned length, unsigned seed) {
    auto pixel = std::make_unique<PixelShader>();
    const auto code = Chain(length, seed);
    pixel->code = Aligned(pixel->storage, code);
    const auto address = reinterpret_cast<std::uintptr_t>(pixel->code);
    pixel->context = {{{0x1b3, 2}, {0x1b4, 2}, {0x1b6, 1}}};
    pixel->registers = {{{0x008, static_cast<std::uint32_t>(address >> 8u)}, {0x009, static_cast<std::uint32_t>(address >> 40u)}, {0x00b, 0}}};
    auto& shader = pixel->shader;
    shader.file_header = 0x34333231u;
    shader.version = 0x18;
    shader.header_size = offsetof(PixelShader, storage);
    shader.shader_size = static_cast<std::uint32_t>(code.size() * 4u);
    shader.code = pixel->code;
    shader.type = 1;
    shader.sh_registers = pixel->registers.data();
    shader.num_sh_registers = pixel->registers.size();
    shader.cx_registers = pixel->context.data();
    shader.num_cx_registers = pixel->context.size();
    return pixel;
}

std::unique_ptr<ComputeShader> MakeCompute(unsigned length, unsigned seed) {
    auto compute = std::make_unique<ComputeShader>();
    const auto code = Chain(length, seed);
    compute->code = Aligned(compute->storage, code);
    const auto address = reinterpret_cast<std::uintptr_t>(compute->code);
    compute->registers = {{{0x20c, static_cast<std::uint32_t>(address >> 8u)}, {0x20d, static_cast<std::uint32_t>(address >> 40u)}, {0x207, 64}, {0x208, 1}, {0x209, 1}, {0x212, 0}, {0x213, 0}}};
    compute->specials.dispatch_modifier = 0x8000;
    auto& shader = compute->shader;
    shader.file_header = 0x34333231u;
    shader.version = 0x18;
    shader.header_size = offsetof(ComputeShader, storage);
    shader.shader_size = static_cast<std::uint32_t>(code.size() * 4u);
    shader.code = compute->code;
    shader.sh_registers = compute->registers.data();
    shader.num_sh_registers = compute->registers.size();
    shader.specials = &compute->specials;
    return compute;
}

}

int main(int argc, char** argv) {
    const unsigned pixels = argc > 1 ? std::strtoul(argv[1], nullptr, 10) : 96u;
    const unsigned computes = argc > 2 ? std::strtoul(argv[2], nullptr, 10) : 192u;
    const unsigned length = argc > 3 ? std::strtoul(argv[3], nullptr, 10) : 1500u;
    const unsigned workUs = argc > 4 ? std::strtoul(argv[4], nullptr, 10) : 0u;
    if (!OpenVulkanTestDevice()) return VulkanTestSkipped;
    std::vector<std::unique_ptr<PixelShader>> pixelShaders;
    std::vector<std::unique_ptr<ComputeShader>> computeShaders;
    for (unsigned index = 0; index < pixels; ++index) pixelShaders.push_back(MakePixel(length, index));
    for (unsigned index = 0; index < computes; ++index) computeShaders.push_back(MakeCompute(length, 100000u + index));
    for (auto& pixel : pixelShaders) AgcDriverRegisterShader_nid_postfix(&pixel->shader);
    const auto start = Clock::now();
    std::thread registering([&] {
        for (auto& compute : computeShaders) AgcDriverRegisterShader_nid_postfix(&compute->shader);
    });
    double resolveMs = 0;
    std::thread loader([&] {
        const auto begin = Clock::now();
        for (auto& pixel : pixelShaders) {
            AgcDriverResolveShaderAbi_nid_postfix(&pixel->shader, {}, {});
            const auto until = Clock::now() + std::chrono::microseconds(workUs);
            while (Clock::now() < until) {}
        }
        resolveMs = Ms(Clock::now() - begin);
    });
    registering.join();
    loader.join();
    const auto issued = Clock::now();
    AgcDriverShutdown_nid_postfix();
    const auto drained = Clock::now();
    std::printf("pixels %u computes %u length %u work_us %u loader_ms %.1f issue_ms %.1f total_ms %.1f\n", pixels, computes, length, workUs, resolveMs, Ms(issued - start), Ms(drained - start));
    return 0;
}

Run as ANYPS5_NO_SHADER_CACHE=1 abi_resolve_bench 96 192 600 0 and ... 96 192 600 20000, alternating the main and PR builds.

[adriadam10]

Thanks for the rework. Checked on the current head (merged onto main): build, agc_driver_prepared_shaders* including _background_abi and _background_abi_sync (12/12 pass), and agc_driver plus the prepared-shader tests repeated 40 times with no failure. The design reads sound: ABI jobs never take the preparation lock, and Read / Edit wait before taking it.

What is still open:

• Linux regression not re-measured. The blocker on the previous version was @SleglJan's alternating-run series on 007 First Light (PPSA11386): median 27.5 s → 29.8 s cold, 24.6 s → 26.5 s warm. The description says this version should fix it, but nobody has repeated that series on this head, and the synthetic harness doesn't cover draws waiting on queued jobs. Please post a before/after on a real title on Linux (or get @SleglJan's run on 2f9697bb5); a gain or no change is enough to continue.
• Conflict with #5007. It touches tests/execution/PreparedShaders.cpp near line 669; one of the two will need a rebase after the other lands.

Once the real-title number is in, this is ready for a final pass.

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
#5007	core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp L675

[SleglJan]

@gjonn @adriadam10 Repeated on 2f9697b with 007 First Light (PPSA11386 v01.018.000) on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ryzen 9 7950X3D, RTX 3080 with NVIDIA 615.71.09), Release with ANYPS5_ENABLE_SPIRV_TOOLS=OFF.

Result, warm shader cache, 6 runs each: with 2f9697b the first presented frame comes at about the same time (6.4 against 6.5 s, ranges overlap), frame 169 comes 0.5 s later, and from frame 300 to 1000 007 renders about 3% slower (2.21 against 2.27 FPS).

On main 007 stops about 2 s after start, before any draw (#5714), so both builds are my integration branch share/007-v5 at 65c5b68 (main 34355ee plus the open PRs and local changes listed in #5714), with and without 2f9697b cherry-picked. The binaries were built from a local commit whose tree differs from 65c5b68 only in one code comment. Only the prepared-shader test list in libSceAgcDriver/CMakeLists.txt conflicted; it takes both sides' entries. 34355ee includes #4404, which this PR's base does not; ShaderRegistry.cpp merged without conflict. Both builds have the same ShaderCacheVersion.

On this branch 007 gets past the stop of the earlier series (its first flip), so the measure changed, and the earlier 1.5 to 2.3 s gap does not compare directly. The measure is the time from exec to the first presented frame and to frames 169, 724 and 1000, back-dated from the window-title FPS counter. No input is given, so none of these frames is gameplay. By frame 169 most shader preparation is done: the first 10-second shader cache report, before frame 125 in every run, has 2,091 of the run's 2,193 to 2,199 lookups, and about 100 more follow by frame 200. From about frame 230 007 renders at mostly 2.0 to 2.5 FPS, so frames 724 and 1000 measure draw throughput after that. Runs alternated between the builds (BAAB).

Only warm runs: a cold boot on this branch takes about 13 minutes, most of it NVIDIA compiling a few compute pipelines of up to 4.3 million SPIR-V words on the queue worker. Every run starts from a fresh copy of the same shader cache (the recompiler cache plus the Vulkan pipeline cache), so shader preparations, this PR's ABI jobs included, are cache hits (no miss in 11 of the 12 runs, 6 in the other). The recompile work this PR moves to the background pool is therefore not exercised here. The sandbox gives the driver an empty home, so NVIDIA's own disk cache is empty in every run. Switches: APS5_AHEAD_PIPELINES=2 (a local 2-thread pool that creates compute pipelines ahead of queued dispatches), APS5_SPARSE_RUN_IMPORT=1, APS5_NO_IDLE_STORE=1, APS5_HOST_IMPORT_MIB=12288, APS5_RESOURCE_CACHE_ENTRIES=64, APS5_NO_DRAW_SRT_ENTRIES=1, APS5_PIPELINED_DRAWS=8, APS5_LOCAL_LARGE_CLASSES=1; no trace or profile switches.

Build	Runs	First present	Frame 169	Frame 724	Frame 1000	FPS, frames 300-1000
share/007-v5	6	6.4 (5.8–6.6)	11.9 (11.3–12.2)	261.2 (251.8–292.1)	384.2 (373.5–418.7)	2.27 (2.21–2.31)
+ 2f9697b	6	6.5 (6.4–7.0)	12.4 (12.1–12.8)	267.4 (259.5–280.6)	395.9 (384.2–407.8)	2.21 (2.16–2.26)

Medians in seconds (FPS in the last column), ranges in parentheses. Counts below are of the 36 pairs of a run without the commit and a run with it.

• Frame 169: in 34 of the 36 pairs the run without the commit gets there first.
• Frames 724 to 1000: medians 124.1 s without (119.9–126.6) against 127.1 s with (124.7–129.9); in 32 of the 36 pairs the run without the commit is faster there, and in 31 its FPS over frames 300 to 1000 is higher. At frames 724 and 1000 themselves, the run without the commit is earlier in 25 and 27 of the 36 pairs.
• Five runs (2 without the commit, 3 with it) look up 6 more shaders (2,199 against 2,193). In each build they are the slowest from frame 724 to 1000; within each group every run with the commit is slower there than every run without it.
• The slowest run without the commit (frame 724 at 292.1 s) took 280.1 s from frame 169 to 724, against 240.5 to 251.0 s in the other runs without it, and is the only run with cache misses (6).
• No ABI job failed in any run with the commit.
• A 60-second on-CPU profile (perf record -e cycles:u) of one more run of each build, from frame 400, shows the same split by library (libSceAgcDriver 24.9% and 24.6% of samples). No driver or recompiler function differs by more than 0.1 points, ResolveShaderAbi is under 0.01% of samples in both builds, and AwaitAbiPreparations under 0.01% with the commit. It does not count waiting time, and that pair did not show the gap: frame 1000 at 383.0 and 384.2 s, frames 724 to 1000 in 123.5 and 123.7 s.

Each run (seconds, in run order)

Run	Build	First present	Frame 169	Frame 724	Frame 1000
1	+ 2f9697b	6.4	12.1	272.4	402.0
2	share/007-v5	6.6	12.2	263.3	389.6
3	share/007-v5	6.1	11.8	261.9	386.4
4	+ 2f9697b	6.8	12.5	266.5	393.5
5	+ 2f9697b	6.5	12.2	268.4	398.3
6	share/007-v5	6.4	11.8	258.4	381.9
7	share/007-v5	6.5	11.9	292.1	418.7
8	+ 2f9697b	6.6	12.8	280.6	407.8
9	+ 2f9697b	6.5	12.3	259.5	384.2
10	share/007-v5	6.5	11.9	260.5	380.4
11	share/007-v5	5.8	11.3	251.8	373.5
12	+ 2f9697b	7.0	12.6	261.4	387.3

AI-assisted: yes (Claude Code)

[gjonn]

Thanks @SleglJan for repeating the series, and @adriadam10 for the reviews.

The bar was a gain or no change on a real title, and this head misses it. On PPSA11386 it is about 3% slower in steady rendering, and frame 169 arrives 0.5 s later. The run without the commit wins 31 to 34 of the 36 pairs.

The profile shows no extra CPU work, so the cost is waiting. My reading is this: since #3159, main no longer prepares the ABI under the writer lock. A title that calls ResolveShaderAbi well ahead of the draw already prepares off the critical path. This PR moves that work onto the pool, and the draw that needs it then waits for it or runs it. It only helps when the thread calling ResolveShaderAbi is itself the bottleneck, and that does not hold generally.

Closing. The test for a failure raised in the background (agc_driver_prepared_shaders_background_abi) is tied to this design, so it goes too.

AI-assisted: yes