Repository navigation
fix(agc): bound a journal replay's graphics and compute pipelines apart - #8542
Open
milescoviello wants to merge 4 commits into
Open
milescoviello wants to merge 4 commits into
milescoviello wants to merge 4 commits into
Conversation
12 tasks done
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
github-actions
Bot
added
build
docs
agc
milescoviello
force-pushed
the
fix/journal-replay-limit-per-kind
branch
from
October 20, 2026 19:39
62171b0 to
de02531
Compare
Contributor
Author
|
Restacked on #8365's rebased head (30c643b, on AI-assisted: yes (Claude Code) |
12 tasks done
Contributor
Author
|
Windows run of this head (
The new cases write and replay journals past both limits, so the test takes about a second longer here. The 28.4 s run was the second of the first three. It didn't come back in six more runs, and the VM's disk is a qcow2 on NFS, so I take it as I/O on this machine rather than the change. I haven't confirmed that. AI-assisted: yes (Claude Code) |
12 tasks done
…lding the file The journal was read whole into one buffer that the decoded modules viewed, so a replay held the entire file (1 GiB at the default APS5_PIPELINE_JOURNAL_MAX_MIB) until its last entry was built. The load now parses the file a record at a time through positioned reads and keeps it open; a replay thread reads an entry's modules again when it builds the entry and rejects it when a record no longer reads back as loaded. The replay report gains the peak module bytes held at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…indows A handle opened without FILE_FLAG_OVERLAPPED serializes its reads, so replay threads reading modules from the journal waited for each other on Windows even though each ReadFile passed its own offset. The journal is now opened overlapped, and each read waits on a per-thread event with GetOverlappedResult. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Module records are SPIR-V and take almost all of a journal's bytes, so APS5_PIPELINE_JOURNAL_MAX_MIB fills with modules and later pipelines are not recorded (boykopovar#8309). A module record now keeps its identity, stage and encoded size, followed by a zstd frame (level 1) of the encoded result; loading and replay expand it before decoding. The journal format becomes 7 (main took 5 for the recipe codec change of boykopovar#5656 and 6 for the polygon modes of boykopovar#7869). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The replay kept the first APS5_PIPELINE_JOURNAL_ENTRIES entries of either kind, so compute pipelines used up the bound meant for the graphics pipeline store, and the entries left out were not reported. Graphics entries now count against that bound alone, compute entries against APS5_PIPELINE_JOURNAL_COMPUTE_ENTRIES (1024), and a line says how many of each a replay leaves out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
12 tasks done
milescoviello
force-pushed
the
fix/journal-replay-limit-per-kind
branch
from
October 21, 2026 11:05
de02531 to
ff8ec20
Compare
Contributor
Author
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A journal replay keeps the first
APS5_PIPELINE_JOURNAL_ENTRIESentries (1024) of the loaded journal. That bound is there for the graphics pipeline store, which holds 1024 graphics pipelines (APS5_PIPELINE_STORE_ENTRIES). But the replay counts compute entries against it too, and it truncates silently. So in a title that records both kinds, compute pipelines take graphics slots, and the graphics entries past the cut are dropped without a word.This PR keeps the first
APS5_PIPELINE_JOURNAL_ENTRIESgraphics entries, as before. Compute entries count separately against a newAPS5_PIPELINE_JOURNAL_COMPUTE_ENTRIES(1024). The recorded order is kept. When a replay leaves entries out, it says so:ENVIRONMENT.mddocuments the new variable. The journal debt entry names both bounds. It was over the 1000-character limit, so it is rewritten gap-first under it.Why
@SleglJan found this with #8365. On PPSA11386, the default-cap seed now holds 1,322 entries: 628 graphics, 694 compute and their modules. The warm run replayed exactly the first 1,024 (481 graphics, 526 compute, 17 already present), and the load line still said 1,322. The 298 entries left out were the journal's tail, about 147 of them graphics, while the store had room for all 628.
Tested
Ubuntu 24.04 (Linux 6.8), GCC 13.3, NVIDIA RTX 4070 Ti, driver 595.99.02, on #8365's head f5504d2.
agc_driver_pipeline_journalgains three replays of its two-entry journal (one compute pipeline recorded first, then one graphics pipeline):With
main's replay code it fails 3 of 3:a replay bounded to 1 graphics and the default compute entries replayed 0 graphics and 1 compute pipelines. The compute entry used up the single slot. With this PR it passes 3 of 3, along withagc_pipeline_journalandagc_driver_graphics.tools/check_conventions.py --base upstream/mainpasses.tools/pr_overlap.pylists only fix(agc): read the pipeline journal's modules at replay instead of holding the file #8213 and feat(agc): store the pipeline journal's modules compressed #8365, which this branch builds on.Not tested: Windows, a title run.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code