Repository navigation
feat(agc): store the pipeline journal's modules compressed - #8365
Open
milescoviello wants to merge 4 commits into
Open
milescoviello wants to merge 4 commits into
milescoviello wants to merge 4 commits into
Conversation
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
Contributor
|
Windows data point: RTX 3080 (NVIDIA 617.42) and AMD Radeon RDNA 2 iGPU ( AI-assisted: yes (Claude Code). |
Contributor
Author
|
Thanks @TheSovietWaffle. That covers Windows, which I hadn't run, and adds an AMD GPU. I added both to the Tested section of the description. AI-assisted: yes (Claude Code) |
Contributor
|
@milescoviello PPSA11386 (007 First Light) with f5504d2, compared with the runs in #8309:
Setup:
Cold seed with the default cap (1024 MiB):
Warm run from that seed:
The 2 GiB seed in #8309 gave 2,505 builds taking 5.4 s and 0.70 FPS, so the default cap now does about what that seed did. The entry limit now binds before the byte cap. The replay takes the first
AI-assisted: yes (Claude Code) |
Contributor
Author
|
Thanks @SleglJan, that's the result #8309 was after: the default cap now covers the level load that the 2 GiB seed needed. Good catch on the entry limit. The replay cuts the loaded journal to its first AI-assisted: yes (Claude Code) |
12 tasks done
Contributor
Author
milescoviello
force-pushed
the
feat/journal-compressed-modules
branch
from
October 20, 2026 19:39
f5504d2 to
30c643b
Compare
Contributor
Author
|
Rebased onto AI-assisted: yes (Claude Code) |
This was referenced Oct 20, 2026
12 tasks done
Contributor
Author
|
Windows results for this head (30c643b, which carries #8213), since the description only has Linux. Setup: Windows 11 Enterprise (build 22631), NVIDIA GeForce GT 1030 (driver 582.66), WinLibs GCC 15.2.0 r7 (UCRT), Release,
AI-assisted: yes (Claude Code) |
12 tasks done
12 tasks done
milescoviello
force-pushed
the
feat/journal-compressed-modules
branch
from
October 21, 2026 11:05
30c643b to
a2eb629
Compare
Contributor
Author
|
Rebased onto #8542 is restacked on this head. AI-assisted: yes (Claude Code) |
12 tasks done
12 tasks done
12 tasks done
Collaborator
|
@milescoviello the rebase onto FileFormat 7 looks right. Full CI on a2eb629 fails one Windows test, twice: The test's message names the cause: That's the new Moving it to struct ReadEventStorage;
auto& completion = HostThreadLocal<ReadEvent, ReadEventStorage>();With that change, |
Contributor
Author
|
@oneandonlydean thanks for tracking that down. Pushed #8213 has the same My Windows 11 VM (WinLibs GCC 15.2.0) is building all modules for AI-assisted: yes (Claude Code) |
Contributor
Author
|
Windows 11 VM, WinLibs GCC 15.2.0, Release, every module built ( AI-assisted: yes (Claude Code) |
12 tasks done
…lding the file The journal was read whole into one buffer that the decoded modules viewed, so a replay held the entire file (1 GiB at the default APS5_PIPELINE_JOURNAL_MAX_MIB) until its last entry was built. The load now parses the file a record at a time through positioned reads and keeps it open; a replay thread reads an entry's modules again when it builds the entry and rejects it when a record no longer reads back as loaded. The replay report gains the peak module bytes held at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…indows A handle opened without FILE_FLAG_OVERLAPPED serializes its reads, so replay threads reading modules from the journal waited for each other on Windows even though each ReadFile passed its own offset. The journal is now opened overlapped, and each read waits on a per-thread event with GetOverlappedResult. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A thread_local with a destructor in a module runs after emulated TLS freed it on Windows, which prx_thread_local_destructors refuses. The per-thread read event now lives in HostThreadLocal, as the other per-thread driver state does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Module records are SPIR-V and take almost all of a journal's bytes, so APS5_PIPELINE_JOURNAL_MAX_MIB fills with modules and later pipelines are not recorded (boykopovar#8309). A module record now keeps its identity, stage and encoded size, followed by a zstd frame (level 1) of the encoded result; loading and replay expand it before decoding. The journal format becomes 7 (main took 5 for the recipe codec change of boykopovar#5656 and 6 for the polygon modes of boykopovar#7869). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
milescoviello
force-pushed
the
feat/journal-compressed-modules
branch
from
October 22, 2026 19:40
e867237 to
e582540
Compare
Collaborator
|
@milescoviello #8115 merged with the pipeline journal at Could you rebase when this one's turn comes and set |
Contributor
Author
|
@oneandonlydean will do. When it's this one's turn I'll rebase it and set AI-assisted: yes (Claude Code) |
Collaborator
|
@adriadam10 I've added myself as a co-assignee here so we can run this through our local Astro and Sarah regression gate alongside your review. Your assignment stays as it is, and I'll post the gate result here once it has run. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Pipeline journal module records are SPIR-V, and they take almost all of a journal's bytes. #8309 shows that
APS5_PIPELINE_JOURNAL_MAX_MIB(1024 MiB) fills with them during a level load, after which later pipelines are never recorded. A module record now stores the module's identity, its stage and the size of its encodedRecompileResult, then a zstd frame (level 1) of that encoding. Loading expands each module once to check that it decodes, as it decoded the raw payload before. Replay expands a module when the entry that needs it is built, on the replay thread, building on #8213's per-module reads. The held-module accounting counts expanded bytes, which is what replay holds in memory. The file format becomes 6 (#5656 took 5 onmainfor its recipe codec change), so an older journal starts over. Since the journal's source version already changes with every recompiler build, that costs no extra rebuilds.zstd is the vendored copy (
3rdparty/zstd, already built aslibzstd_staticfor ZArchive).libSceAgcDriverlinks it statically, and on Linux it's kept out of the library's exports with--exclude-libs,libzstd.a, ascmake/ZArchive.cmakedoes.nm -D --defined-only libSceAgcDriver.prx | grep -i zstdprints nothing.Why
The same cap then holds about 3.5 times as many modules:
zstd -1gives 27.4% (3.7 s on one thread), so everything that seed recorded would fit under the default cap. Records compressed one at a time can come out a little larger than the whole stream.zstd -1(0.39 s on one thread).Stacked on #8213
This branch holds #8213's three commits and then its own, e582540, which is the only one to review here. It changes the module read path that #8213 adds, so it waits for #8213.
Tested
Restacked onto #8213's rebased head 8abecc1, on
maind6e7ab4, ase5825406b4. The branch now carries #8213's three commits unchanged and then this PR's one commit. Its copy of #8213'sHostThreadLocalfix was identical, so the rebase dropped it. The change againstmainhas the samegit patch-idas before. On that head, Ubuntu 26.04.1, GCC 15.2, RTX 4070 Ti (615.71.09):agc_driver_pipeline_journalandagc_pipeline_journalpass 3 of 3.pipeline_journal|pipeline_store|pipeline_library|pipeline_optimized_readback, onmaind6e7ab4 and on this head: 16 of 16 pass on each, with noVUID-message and no layer-missing line.FileFormatis 7 here and still 6 onmain. My feat(agc): draw one-sample color targets beside a multisampled depth target with sample 0's coverage #8115 also takes 7, so whichever of the two lands second renumbers.Earlier results, on the heads named in them:
Ubuntu 24.04 (Linux 6.8), GCC 13.3, NVIDIA RTX 4070 Ti, driver 595.99.02.
agc_driver_pipeline_journal:HeldModuleTestsnow also encodes its 16 compute modules and requires the whole journal file to be smaller than those encodings. With fix(agc): read the pipeline journal's modules at replay instead of holding the file #8213's uncompressed records it fails (the journal is 467,960 bytes with 16 of its modules alone encoding to 430,144 bytes), 3 of 3. With this PR both journal tests pass 3 of 3, the held-module bound among them.Khronos validation layer (LunarG SDK 1.4.363), NVIDIA RTX 4070 Ti (595.99.02), Ubuntu 24.04:
VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation ANYPS5_NO_SHADER_CACHE=1 ANYPS5_REQUIRE_VULKAN=1 ctest -R 'pipeline_journal' -Von fix(agc): read the pipeline journal's modules at replay instead of holding the file #8213's head 5778288 and on this PR (f5504d2). 2 of 2 pass on each. Neither log reports the layer missing, and neither has aVUID-error.tools/check_conventions.py --base upstream/mainpasses, apart from the existingdebt-lengthwarning on the pipeline journal entry. This PR doesn't change that entry, since nothing in it becomes false.PPSA11386, by @SleglJan (comment), RTX 3080 (NVIDIA 615.78.08), Linux, on a
main68323ba integration with the title's open PRs, fix(agc): read the pipeline journal's modules at replay instead of holding the file #8213 and this PR: with the default 1024 MiB cap, the first level's load now records completely, and the cap is reached about 455 s into gameplay. Before this, it was reached during that load. The journal records 628 graphics, 694 compute and 1,292 modules, against 330, 377 and 664. The warm level entry has no gap, against 280 s from Resume to the first gameplay frame: 4,107 draws built in 7.2 s, against 772 in 284.5 s.Windows 10 Pro 22H2, WinLibs GCC 15.2.0, Release, by @TheSovietWaffle on
f5504d2ec:agc_driver_pipeline_journalandagc_pipeline_journalpass 3 of 3 on an NVIDIA RTX 3080 (617.42) and on an AMD Radeon RDNA 2 iGPU (1002:164e, 32.0.21045.5002).Checklist
main; no other open PR implements the same functionsFileFormatnow names record layouts among what bumps it)NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code