Skip to content

perf(agc): mirror a sparse heap whose committed span crosses adjacent ranges - #9551

Open
JJenkx wants to merge 4 commits into
boykopovar:mainfrom
JJenkx:perf/agc-sparse-mirror-range-run
Open

JJenkx wants to merge 4 commits into
boykopovar:mainfrom
JJenkx:perf/agc-sparse-mirror-range-run

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What

A sparse heap mirror needs one leased range that covers the heap's committed span: bindSparseMirror calls containingRange(lease, begin, neededEnd). When the guest grows a sparse heap by mapping a second range directly after the first, two adjacent leased ranges cover the span instead of one. The bind is then refused, and every use of the heap falls back to copyRegion, which copies every committed piece on the CPU and diffs it at write-back.

Echoes of the End (PPSA25636) does this with its PRT heap: 0x8fc0000000+0x21000000, then 0x8fe1000000+0x1000000. Its committed span ends at 0x8fe2000000. The first 8 binds succeed while one range covers the span. After that, each use copies about 550 MiB.

With this change, bindSparseMirror falls back to a joined range when no single range covers the span. The joined range covers the run of continuing ranges (crossedRun, the same rule the import path uses). joinedHeapRange keeps it for as long as the run's member ranges are the same objects, so the mirror's identity check (sameRange) holds across builds. When a member changes, the joined range is replaced: the old mirror's weak range expires and the sweep drops it, as it does for a single range that goes away. sweepMirrors first releases the joined ranges whose members have expired, and forces a full sweep when it releases one, so a mirror over ranges the guest released is freed even if no bind follows.

A joined mirror also needs a change in releaseImportedMirrors. An address-based build (AcquireRegistered) releases the heap mirror at the base of each range it imports. A joined mirror starts at its first range, so importing that range alone released the whole mirror, and the next descriptor bind rebuilt it: in Echoes, 140 releases and rebuilds of a 544 MiB mirror in 480 s. A heap mirror is now released only when the imported range covers its reserved span, which for a single-range mirror is unchanged.

Tested

  • Current head a5331de72d on main 12f7e1cd3f. The head was ported onto fix(agc): bind heap mirrors only where the guest holds bytes #7647's resident heap mirrors. After fix(agc): import registered ranges larger than the device's largest allocation in pieces #7292, a piece import records the summed length of its pieces, so a mirror is released only when the pieces cover it whole.
    • Exact-head focused gate: ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_LOADER_DEBUG=layer ctest -V -j1 -R '^(agc_driver_graphics|agc_sparse_.*|agc_heap_mirror_.*|agc_host_import.*)$'. Results: main 16/16; this PR with the partial-import fix (5c7c0ea870) reverted 15/16, failing only the intended "an address build that imported one range of a joined heap mirror released the mirror"; this PR 16/16.
    • The validation layer (1.4.363) was inserted in 4 tests per arm, with 0 VUIDs. Only the two existing Windows guest-arena skips appear.
    • Receipt: perf(agc): mirror a sparse heap whose committed span crosses adjacent ranges #9551 (comment). Logs vvl9551h-*.log in https://github.com/JJenkx/AnyPS5/releases/tag/pr9551-evidence. Earlier heads' gates (98de5968c0, caaed795da) are in the comments.
  • agc_driver_graphics_tests, new case joinedHeapMirrorPartialImportTests: a heap bound across two committed ranges is mirrored over both, and an address build that imports the first range keeps the mirror (same buffer, no rebuild). With only the release fix reverted, it fails with "an address build that imported one range of a joined heap mirror released the mirror".
  • agc_driver_graphics_tests, new case sparseJoinedHeapMirrorTests (mock device; the guest arena write watch on Windows, GuestWriteWatch on Linux). A sparse heap over two adjacent registered ranges, with one committed page in each, binds one mirror. A store to the second range is written back. A second bind over unchanged ranges reuses the mirror without a rebuild. Removing the ranges frees the mirror at the next sweep. On Linux (GuestWriteWatch), the whole agc_driver_graphics_tests run passes with this change. With only this change reverted, the new case fails with "a sparse heap whose committed span crosses an adjacent range was not mirrored".
  • Echoes of the End PPSA25636, Linux 7.2.9, RX 6650 XT, RADV (Mesa 26.2.4). The title needs open PRs to reach its scene (fix(shader): preserve read lane dominance across branches #8089, feat(libSceAgcDriver): import scattered direct memory as a dma-buf run list and probe its coherence #4449, fix(agc): lower import budgets only on memory exhaustion #8091, fix(agc): keep the write-back ready barrier while hole copies are pending #8908 and others), so both arms are the same integration tree, with and without this change, in the same build directory.
    • Instrumented single runs, not seeded (APS5_PROFILE_DRAW=1 APS5_TRACE_BIGBUF=1, 420 s each). Before:
      • 417 CPU copies of the heap, 227,704 MiB read;
      • 90.1 s of guest reads;
      • 221 presents by 340 s.
    • after, same settings:
      • 0 CPU copies of the heap, 333 MiB read in all;
      • 14.9 s of guest reads;
      • 301 presents by 340 s;
      • 0 amdgpu kernel events.
    • Matched-seed captures, no overrides other than APS5_TRACE_SECONDS=290:120 (420 s per arm):
      • The fixed arm moves through the title faster, so the 290-410 s window covers a later and heavier stretch with it (presents 559-672) than without it (516-664). The window summary lines therefore compare different scenes.
      • By time to the same present (Driver.Present timestamps), with this change: present 560 at 290.7 s (330.8 s without), 600 at 317.1 s (355.4), 640 at 343.5 s (379.7), 660 at 364.1 s (403.4). That is 36-40 s earlier, the same title state about 10 % sooner.
      • The stretch from present 560 to 660 takes the same time in both arms (73.4 s with, 72.6 s without). The difference comes from the part where the heap was copied.
      • Summary lines: without seconds 290:120 from present 516: 149 presents, frame p50 644.0 / p90 1389.8 / p99 2297.5 / max 2380.6 ms, GPU busy 386.7 ms per present; with seconds 290:120 from present 559: 114 presents, frame p50 672.1 / p90 1541.4 / p99 7793.8 / max 7978.6 ms, GPU busy 386.8 ms per present. The slowest frames with the change (presents 668-672) come after the last present the other arm reached.
    • Matched-seed pair with APS5_PROFILE_DRAW=1 APS5_TRACE_BIGBUF=1, 420 s per arm, 0 amdgpu kernel events in either:
      • without: 440 CPU copies of the heap (240,752 MiB), 246,603 MiB read by CPU copies in all, 84.9 s of guest reads, 139 sparse heap binds, 25 mirror rebuilds;
      • with: no CPU copy of the heap, 3,838 MiB read by CPU copies in all, 14.6 s of guest reads, 541 sparse heap binds, 127 mirror rebuilds.
      • The joined mirror is rebuilt more often than a single-range one, because a replaced member range replaces the joined range. Fewer rebuilds is a possible follow-up; guest-read time still falls by 83 %.
    • Final version (with the sweep release), same profiled settings, 420 s: no CPU copy of the heap, 3,302 MiB read by CPU copies in all, 15.5 s of guest reads, 0 amdgpu kernel events.

External Windows cross-test

Scramblrrrr reports testing this head c83c5ea1c9 in integration 152c7a80ac70, based on 163fd9b52c, on Windows 11 / RTX 5070 Ti 617.42 / Ryzen 7800X3D / 32 GiB / WinLibs GCC 15.2r7 Release, with SPIRV Tools off. The fetched integration commit has exactly those base/topic parents and contains this topic head.

The external report records 85 focused passes and one expected Linux-only skip, with Khronos 1.4.363 loaded for native Vulkan cases. Two 02275 errors in sampled_storage_base_level_usage also occurred on that tester's baseline; no new validation errors were reported. The graphics regression uses a mock device. Reverting only GuestBufferMemory.cpp to that integration's baseline made it fail at the adjacent-range sparse-heap assertion; restoring the candidate passed with the fixture executable unchanged. The restored focused import/mirror controls passed 16 cases with one expected skip and zero validation errors.

This is external correctness evidence, not the required controlled performance comparison or a native Vulkan claim for the mock graphics case. The tester reached a moving offline menu, not gameplay, and explicitly did not establish an FPS improvement or regression. The requested controlled same-scene performance comparison remains open. A later focused Linux gate receipt records main 4777caca34 and candidate 98de5968c0 passing 15/15 on the same machine, with required Vulkan, shader cache disabled, four actual validation-layer instance insertions and zero VUIDs per arm. The test-only and release-fix-reverted controls each fail the intended assertion (14/15). Two existing Windows guest-arena cases are skipped on Linux in every arm; neither new joined-range case is skipped. These are results on the pinned base above, not a claim of testing every subsequent main revision. The receipt also corrects the older vvl9551e-ctest-change.log provenance: it is from 318ad5365b; vvl9551f-ctest-change.log covers 98de5968c0.

Summary lines

Measured at head 98de5968c0 (the same joined-range and partial-import logic, before the #7647 port and the #7292 piece-import fix; not re-measured on a5331de72d), present-matched warm pair: APS5_TRACE_FRAMES=450:100, 540 s per arm, same seed. Both arms are the integration tree; the baseline has this PR's code reverted. Files: https://github.com/JJenkx/AnyPS5/releases/tag/pr9551-evidence (the current-* assets).

Baseline (without this PR):

{"commit":"6784938b0095","dirty":false,"env":{"APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_DIR":"<path>","APS5_TRACE_FRAMES":"450:100"},"gpu":"AMD Radeon RX 6650 XT (RADV NAVI23)","driver":"radv Mesa 26.2.4-arch1.1","os":"Linux 7.2.9-arch1-1","cpu":"AMD Ryzen 9 7950X 16-Core Processor","cpu_threads":32,"memory_mib":127901,"title_id":"PPSA25636","title":"Echoes of the End","content_version":"01.012.000","resolution":"3840x2160","window":{"spec":"frames 450:100","first_present":450,"ms":114396.035},"presents":100,"frame_ms":{"p50":1003.787,"p90":1681.364,"p99":2596.762,"max":2596.762},"gpu_busy_ms_per_present":472.414,"worker_busy_pct":{"queue worker 0x0":99.817,"queue worker 0x20":97.747},"top_phases_ms_per_frame":{"Driver.Packet":2229.130,"Driver.Packet.dispatch":1204.279,"Driver.Dispatch":1107.533,"Vulkan.Dispatch":711.504,"Resources.Complete":705.573,"Vulkan.Dispatch.shader_resources":701.175,"Driver.Packet.memory_wait":636.887,"GpuMutex.wait.indirect":143.947},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":37.231,"events_lost":478449,"totals_full":false}},"trace":"aps5-trace-6784938b0095-4118577.json.gz"}

With this PR:

{"commit":"1551d42a177d","dirty":false,"env":{"APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_DIR":"<path>","APS5_TRACE_FRAMES":"450:100"},"gpu":"AMD Radeon RX 6650 XT (RADV NAVI23)","driver":"radv Mesa 26.2.4-arch1.1","os":"Linux 7.2.9-arch1-1","cpu":"AMD Ryzen 9 7950X 16-Core Processor","cpu_threads":32,"memory_mib":127901,"title_id":"PPSA25636","title":"Echoes of the End","content_version":"01.012.000","resolution":"3840x2160","window":{"spec":"frames 450:100","first_present":450,"ms":110954.006},"presents":100,"frame_ms":{"p50":1036.931,"p90":1601.300,"p99":2162.815,"max":2162.815},"gpu_busy_ms_per_present":453.019,"worker_busy_pct":{"queue worker 0x0":99.816,"queue worker 0x20":97.643},"top_phases_ms_per_frame":{"Driver.Packet":2153.495,"Driver.Packet.dispatch":1166.453,"Driver.Dispatch":1074.981,"Vulkan.Dispatch":681.489,"Resources.Complete":675.709,"Vulkan.Dispatch.shader_resources":671.282,"Driver.Packet.memory_wait":617.489,"GpuMutex.wait.indirect":162.506},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":36.961,"events_lost":463551,"totals_full":false}},"trace":"aps5-trace-1551d42a177d-112044.json.gz"}
  • tools/performance_compare.py: mean frame -3.0 %, p50 +3.3 %, p90 -4.8 %, p99 -16.7 %, GPU busy -4.1 %. 0 amdgpu kernel events in either.
  • Capture limits: both current traces dropped queue-worker events (478,449 baseline; 463,551 candidate). The event traces therefore omit work and cannot support complete event-by-event attribution. This is one warm integration-tree pair, with median frame time 3.3 % worse; it does not establish a repeatable FPS gain on plain main.
  • In an attempted 480 s pair over presents 560-659, only the arm with this PR reached present 659; the baseline did not, so that window is not compared.
  • The earlier pair over presents 560-659 on head c83c5ea1c9 measured mean -15.1 %; its files are the non-current assets of the same release.
  • The mean difference in this window is within run-to-run variance. The measured effect this PR is sure of is the removed CPU work (heap copies 440 -> 0, guest reads 84.9 s -> 14.6 s, mirror rebuild time 15.5 s -> 3.3 s).

Direct links to the existing capture evidence

These are the already-published integration captures summarized above, not a new title run or a plain-main benchmark. Their digest output has been regenerated from the linked traces and matching summary files with tools/trace_digest.py at db6af9e0fd; both outputs match the existing release digests byte for byte.

Baseline integration without this PR: summary line, trace, trace_digest output.

**frames 450:100** from present 450: 100 presents, frame p50 1003.8 / p90 1681.4 / p99 2596.8 / max 2596.8 ms, GPU busy 472.4 ms per present
AMD Radeon RX 6650 XT (RADV NAVI23), radv Mesa 26.2.4-arch1.1, Linux 7.2.9-arch1-1, 3840x2160, commit 6784938b0095, dropped: yes

Integration with this PR: summary line, trace, trace_digest output.

**frames 450:100** from present 450: 100 presents, frame p50 1036.9 / p90 1601.3 / p99 2162.8 / max 2162.8 ms, GPU busy 453.0 ms per present
AMD Radeon RX 6650 XT (RADV NAVI23), radv Mesa 26.2.4-arch1.1, Linux 7.2.9-arch1-1, 3840x2160, commit 1551d42a177d, dropped: yes

Both traces dropped events. The existing limits still apply: median frame time is 3.3% worse, mean frame time is 3.0% lower, and a repeatable FPS gain is not established. This evidence-link update does not change the implementation or claim new GPU or title validation.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file (no unimplemented path added)
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source (not applicable)
  • Generic behaviour, not specific to one title
  • Depends on: none. fix(agc): bind heap mirrors only where the guest holds bytes #7647 edits the same heap mirror code; neither depends on the other, and git merge-tree shows two mechanical conflicts in GuestBufferMemory.cpp, described on fix(agc): bind heap mirrors only where the guest holds bytes #7647.
  • Fixes: none
  • Unblocks: none. This is a performance change measured on Echoes of the End (PPSA25636), which has no Game bring-up discussion.
  • GPUs tested: AMD Radeon RX 6650 XT, RADV Mesa 26.2.4, Linux 7.2.9
  • AI-assisted: yes

🤖 Generated with Claude Code

@github-actions github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line unblocks title Names a title ID on the Unblocks line labels Oct 22, 2026
@github-actions

Copy link
Copy Markdown

This pull request is labeled perf, so it needs the summary line of a capture on main and one with the change, from the same title, scene and machine, as the Performance report form asks. Please paste both aps5-summary.jsonl lines, attach the two traces and add the output of tools/trace_digest.py.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new synthetic test and the hosted eligibility, conventions, intake, labels, and report checks are useful. I cannot approve this performance PR yet because the repository performance-report rule is not met: the matched-seed APS5_TRACE_SECONDS=290:120 summaries compare different scene stretches (the PR notes presents 516–664 versus 559–672), and the report lacks the required attached base/change traces plus tools/trace_digest.py output. The profile runs with APS5_PROFILE_DRAW/APS5_TRACE_BIGBUF are diagnostic runs, not compliant performance captures under the report form. Please provide a warm-cache capture pair on the same title state/settings/machine with the required summary lines, traces and digests, or remove the speed claim/perf title and retain the measured observations as non-comparative diagnostics. Separately, AGC changes require the same focused Vulkan tests on main and this commit with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and the Khronos validation layer, confirming no new VUIDs and that the requested layer was found; this report does not include that comparison. Hosted GPU jobs were skipped, so I have not treated them as run. I did not run local tests.

@MarcZX10

Copy link
Copy Markdown
Contributor

The AGC rules require the same focused Vulkan test on main and the change on the same machine/GPU with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_LAYER_KHRONOS_validation; compare VUIDs and confirm the layer was found. Also, because this is labeled perf, the PR needs two valid warm-cache summary lines, attached traces, and tools/trace_digest.py output from the same title scene/settings/machine. The current 290–410 s windows cover different present ranges, as the report itself notes; the diagnostic profile traces do not substitute for the report form capture. Please add compliant comparisons or remove the speed claim and perf labeling. The hosted Linux/Windows and relinker test jobs are skipped for this file set; I did not run these tests locally.

@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 thanks, both points are right. Coming next:

  • Focused tests: the same ctest -V run on main and on this branch, on the same RX 6650 XT / RADV machine, with VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, ANYPS5_NO_SHADER_CACHE=1 and ANYPS5_REQUIRE_VULKAN=1. I'll include the layer-load lines and the VUID comparison.
  • Perf capture: warm-cache captures over the same presents in both arms, using APS5_TRACE_FRAMES=<start>:<count> instead of a seconds window. Both aps5-summary.jsonl lines, the traces and the tools/trace_digest.py output will be attached.

One thing a reviewer should know: Echoes does not reach this scene on plain main. It needs #8089, #4449, #8091 and #8908. So the perf baseline will be that integration tree without this change, not main itself.

If that is not acceptable for the perf label, I'll drop the speed claim and keep the change as a fix for the CPU copies, which the counters show directly.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1191 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 2434 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 4448 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 4449 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 5021 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 5372 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 6146 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6182 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6185 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6186 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6197 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 6412 core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 6481 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6548 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6565 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6628 core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 6918 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 7201 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 7261 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 7446 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 7457 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 7514 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 7618 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 7622 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 7642 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 7717 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 7844 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 8896 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 8926 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 8963 core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 9012 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 9176 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 9287 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 9289 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 9402 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 9508 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 9514 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 9852 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 9879 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 9898 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 9940 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 9941 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 10025 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 10141 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 10193 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp
PR 10194 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 10304 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 10369 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 10443 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 10487 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 10668 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp

JJenkx pushed a commit to JJenkx/AnyPS5 that referenced this pull request Oct 22, 2026
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Present-matched capture, as asked.

Setup

  • Echoes of the End (PPSA25636), Linux 7.2.9, RX 6650 XT, RADV (Mesa 26.2.4).
  • Warm cache: shader caches, Mesa cache and save state restored from the same seed before each arm.
  • One build directory, rebuilt incrementally between arms.
  • APS5_TRACE_FRAMES=560:100, so both arms record the same 100 presents (560-659), 480 s runs, no other overrides.
  • Baseline: the integration tree (needed to reach the scene, see above) with this change reverted, 20f928c642. Candidate: the same tree with it, 0560946d6d. The two differ only in GuestBufferMemory.cpp.
  • 0 amdgpu kernel events in either run.

Summary lines (tools/trace_digest.py)

  • without: frames 560:100 from present 560: 100 presents, frame p50 748.4 / p90 1091.6 / p99 6985.3 / max 6985.3 ms, GPU busy 412.5 ms per present
  • with: frames 560:100 from present 560: 100 presents, frame p50 630.0 / p90 1299.8 / p99 2373.5 / max 2373.5 ms, GPU busy 400.7 ms per present

tools/performance_compare.py, candidate minus baseline

Metric Change
mean frame -15.1 %
p50 -15.8 %
p90 +19.1 %
p99 / max -66.0 %
GPU busy per present -2.9 %
Driver.Dispatch -18.4 %
Resources.Complete -27.0 %

The gain is on the CPU side, as expected from removing the per-use copies of the heap. One pair, so p90 and the tails are within run-to-run variance.

Files (both aps5-summary.jsonl lines, both traces, both full digests and the comparison) are on the branch https://github.com/JJenkx/AnyPS5/tree/evidence/pr9551.

The focused ctest -V comparison under the validation layer passed on both main (a6dcea9279) and this branch, 2/2 each, with no VUIDs. I'm rerunning it with VK_LOADER_DEBUG=layer to show the layer was loaded, and will post that with the logs.

@MarcZX10

Copy link
Copy Markdown
Contributor

Thanks for adding the present-matched capture details. Two required pieces are still outstanding in the PR itself: (1) the evidence/pr9551 fork branch is not an attachment to this pull request, so please attach the summary lines, traces, digests, and comparison here; (2) the Vulkan comparison currently cites main a6dcea9279, not current upstream main dce41a89478876a8e3ea9c346b1dac00bb31c8cd, and the requested layer-load confirmation is still pending. Please update the comparison to current main (or document why the current main build cannot reach this scene, then keep the performance claim limited to the explicitly identified integration baseline) and include logs proving the layer loaded and no new VUIDs. The 560:100 integration-tree pair is useful same-scene performance evidence, but its p90 regression and single pair should remain disclosed as you have done.

JJenkx pushed a commit to JJenkx/AnyPS5 that referenced this pull request Oct 22, 2026
JJenkx pushed a commit to JJenkx/AnyPS5 that referenced this pull request Oct 22, 2026
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Focused tests under the validation layer, on main and on this branch.

Setup

  • One build directory, RX 6650 XT, RADV Mesa 26.2.4, Linux 7.2.9.
  • main is this PR's base, a6dcea9279; the change is c83c5ea1c9.
  • Environment: ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_LOADER_DEBUG=layer.
  • Command: ctest -V -R '^(agc_driver_graphics|agc_sparse_.*|agc_heap_mirror_.*|agc_host_import_sparse)$'. That is 12 tests, covering the sparse heap mirror, heap mirror, sparse run import and sparse host import paths, and including the new sparseJoinedHeapMirrorTests case.

Result

  • 12/12 pass on both.
  • The loader shows the layer added: [Vulkan Loader] WARNING | LAYER: env var 'VK_INSTANCE_LAYERS' defined and adding layers "VK_LAYER_KHRONOS_validation" and the inserts, for the tests that create a real instance (agc_sparse_run_import, agc_sparse_run_import_copied).
  • agc_driver_graphics and the mirror cases run on the mock device, so the layer has nothing to validate there.
  • VUIDs are identical on both: one VUID-vkCmdCopyBuffer-srcBuffer-00118 in agc_sparse_run_import_copied. It is present on main too, so this change neither adds nor removes it.

Logs (paths redacted): https://github.com/JJenkx/AnyPS5/tree/evidence/pr9551/vvl

@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

The aps5-summary.jsonl lines of the present-matched pair (APS5_TRACE_FRAMES=560:100). The traces and the tools/trace_digest.py output are on https://github.com/JJenkx/AnyPS5/tree/evidence/pr9551.

Baseline (without this change):

{"commit":"20f928c642f7","dirty":false,"env":{"APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_DIR":"<path>","APS5_TRACE_FRAMES":"560:100"},"gpu":"AMD Radeon RX 6650 XT (RADV NAVI23)","driver":"radv Mesa 26.2.4-arch1.1","os":"Linux 7.2.9-arch1-1","cpu":"AMD Ryzen 9 7950X 16-Core Processor","cpu_threads":32,"memory_mib":127901,"title_id":"PPSA25636","title":"Echoes of the End","content_version":"01.012.000","resolution":"3840x2160","window":{"spec":"frames 560:100","first_present":560,"ms":92408.448},"presents":100,"frame_ms":{"p50":748.353,"p90":1091.598,"p99":6985.317,"max":6985.317},"gpu_busy_ms_per_present":412.483,"worker_busy_pct":{"queue worker 0x0":99.794,"queue worker 0x20":96.700},"top_phases_ms_per_frame":{"Driver.Packet":1775.286,"Driver.Packet.dispatch":983.421,"Driver.Dispatch":913.871,"Vulkan.Dispatch":547.906,"Vulkan.Dispatch.shader_resources":539.003,"Resources.Complete":530.227,"Driver.Packet.memory_wait":471.162,"GpuMutex.wait.indirect":143.345},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":31.770,"events_lost":417448,"totals_full":false}},"trace":"aps5-trace-20f928c642f7-4193406.json.gz"}

With this change:

{"commit":"0560946d6da2","dirty":false,"env":{"APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_DIR":"<path>","APS5_TRACE_FRAMES":"560:100"},"gpu":"AMD Radeon RX 6650 XT (RADV NAVI23)","driver":"radv Mesa 26.2.4-arch1.1","os":"Linux 7.2.9-arch1-1","cpu":"AMD Ryzen 9 7950X 16-Core Processor","cpu_threads":32,"memory_mib":127901,"title_id":"PPSA25636","title":"Echoes of the End","content_version":"01.012.000","resolution":"3840x2160","window":{"spec":"frames 560:100","first_present":560,"ms":78426.489},"presents":100,"frame_ms":{"p50":630.016,"p90":1299.811,"p99":2373.513,"max":2373.513},"gpu_busy_ms_per_present":400.723,"worker_busy_pct":{"queue worker 0x0":99.763,"queue worker 0x20":96.499},"top_phases_ms_per_frame":{"Driver.Packet":1521.026,"Driver.Packet.dispatch":823.749,"Driver.Dispatch":745.761,"Driver.Packet.memory_wait":418.504,"Vulkan.Dispatch":410.589,"Vulkan.Dispatch.shader_resources":402.112,"Resources.Complete":387.016,"GpuMutex.wait.indirect":111.581},"trace_dropped":true,"trace_dropped_threads":{"queue worker 0x0":{"events_full_after_s":25.439,"events_lost":428881,"totals_full":false}},"trace":"aps5-trace-0560946d6da2-163082.json.gz"}

@github-actions github-actions Bot removed the perf numbers missing perf pull request without a before and after summary line label Oct 22, 2026
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 your review's points are all addressed now. Please take another look when you have time.

  • Same title state: a warm-cache pair over the same presents, APS5_TRACE_FRAMES=560:100 instead of the seconds window. Both arms record presents 560-659 from one seed of shader caches and save state, with no diagnostic overrides. The results are in the comment above and in the description's "Summary lines" section.
  • Summary lines, traces and digests: both aps5-summary.jsonl lines are in the description. The two traces, both tools/trace_digest.py outputs and the tools/performance_compare.py table are at https://github.com/JJenkx/AnyPS5/tree/evidence/pr9551.
  • Focused Vulkan tests: 12 sparse and heap-mirror ctests on this PR's base and on the change, with ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. All pass on both. The loader shows the layer added for the tests that create a real instance, and the VUIDs are identical on both (one pre-existing vkCmdCopyBuffer-srcBuffer-00118, present on main). The logs are in vvl/ on the same branch.
  • Diagnostic runs: the APS5_PROFILE_DRAW / APS5_TRACE_BIGBUF runs remain in the description as observations (CPU copy counts) and are not presented as the performance capture.

@github-actions github-actions Bot added agc libSceAgc* and removed unblocks title Names a title ID on the Unblocks line labels Oct 22, 2026
@Scramblrrrr

Copy link
Copy Markdown
Contributor

Windows cross-test completed for c83c5ea on integration 152c7a8 (base 163fd9b):
https://github.com/Scramblrrrr/AnyPS5/tree/codex/first-light-oct22-sparse-heap-runs

Windows 11 / RTX 5070 Ti 617.42 / Ryzen 7800X3D / 32 GiB, WinLibs 15.2r7 Release, SPIRV Tools OFF. 85 focused cases passed, one expected Linux-only skip (36.81 s), with the actual Khronos 1.4.363 layer loaded for native Vulkan cases. Two existing 02275 validation errors in sampled_storage_base_level_usage match the baseline; no new validation errors. The graphics regression itself uses the mock device, so its passing result is not a native Vulkan validation claim.

Production-only negative: restoring baseline 163 GuestBufferMemory.cpp while retaining the new fixture makes graphics fail with “a sparse heap whose committed span crosses an adjacent range was not mirrored.” Restoring the exact candidate passes; fixture executable hashes were unchanged. Restored focused import/mirror controls: 16 pass, one expected skip, 8.78 s, zero validation errors.

007 observation, not a controlled performance claim: six contained summary intervals in the observed warm moving offline menu gave 78 presents / 63.204949 s = 1.2341 FPS, GPU busy 191.76 ms/present. Previous 163 menu observation was 1.2788 FPS. This does not establish an improvement or regression; the diagnostic startup settings and scene timing are not a strict performance pair. Large CPU buffer copies persist, so I have not established that the dominant title copies are this adjacent-range sparse case. No gameplay was reached in this run; it ended at the bounded 30-minute deadline. #9652 remains excluded pending isolation of a separate startup failure.

Original author commits retained, no duplicate implementation. Raw title code/requests/captures remain local. AI-assisted: Codex.

@JJenkx

JJenkx commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

Added c65f593739: keep a heap mirror that an import covers only in part.

While profiling the joined mirror's rebuilds, I found an interaction with the existing release rule:

  • An address-based build (AcquireRegistered) releases the heap mirror at the base of each range it imports.
  • A joined mirror starts at its first range but spans the others, so importing that first range alone released the whole mirror.
  • The next descriptor bind then found no single import covering the run and rebuilt the mirror.

The fix releases a heap mirror only when the imported range covers its reserved span. For a single-range mirror these are the same range, so its behaviour does not change.

Echoes (PPSA25636), one 480 s run each, same settings (APS5_PROFILE_DRAW=1 APS5_TRACE_BIGBUF=1), integration tree:

before after
heap mirror rebuilds 141 27
heap mirrors released for imports 140 (76,160 MiB) 24 (12,672 MiB)
time in heap mirror builds 15.5 s 3.3 s
amdgpu kernel events 0 0
  • The remaining releases are single-range ones from the loading phase, before the heap grows into a joined run; releasing those is the existing, correct behaviour.
  • Cumulative presents are within run-to-run variance (265 vs 274 by 400 s). I am reporting this as removed CPU work, about 12 s of build time and about 100 ms per avoided rebuild, not as a measured frame-rate change.

The mock-device tests cannot exercise this path, because host imports fail there and nothing is released. The focused validation-layer tests on this new head are queued, and I will post them.

@MarcZX10

Copy link
Copy Markdown
Contributor

I reviewed the new c65f593 change as well. The reserved > bytes guard is aimed at the partial-import release case, but sparseJoinedHeapMirrorTests currently binds twice and then removes the registrations; it does not pass through a host import that covers only the first member range. Please add a focused regression asserting that this partial import keeps the joined heap mirror and does not force a rebuild. I also found the open #7647 sparse-heap mirror PR edits GuestBufferMemory.cpp and BdaResources.cpp; a three-way merge of both heads is text-clean, but these are closely related mirror paths. Please coordinate with its author on merge order/composition and record the decision in the checklist, adding Depends on if sequencing is required. Current main is 4777cac; this PR is 3 ahead/119 behind (merge base a6dcea9). Diff-check, conventions and merge-tree against main pass. The focused Vulkan report in the PR/comments is on earlier head c83c5ea and old base a6dcea9; the author says the c65 validation rerun is queued. Since this changes AGC, please complete the prescribed same-machine/GPU main and current-head run with required Vulkan, no shader cache and validation settings, layer-load confirmation and VUID comparison. The performance summaries/traces also need to match the current head and be attached per the report form; the existing pair is from the earlier head and the trace artifacts are linked from a fork. I did not run GPU tests locally.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added partial-import guard addresses a real lifecycle issue, but its new behavior lacks a targeted test: the sparse joined heap case never imports only one member range after creating the mirror. Please add that regression. Also coordinate with open #7647, which edits the same mirror implementation and tests, and record whether either depends on the other. This head is 3 ahead/119 behind current upstream/main 4777cac (merge base a6dcea9). Diff-check, conventions and merge-tree against main pass, but the GPU evidence and performance pair in the PR cover earlier head c83c5ea on older base a6dcea9. The required AGC validation comparison and report artifacts must be refreshed on current main and this head, with the layer confirmed loaded and VUIDs compared. I did not run GPU tests locally.

@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Focused validation-layer tests on the new head c65f593739, as promised in my last comment.

Setup:

  • Same machine as before; this PR's base a6dcea9279 against c65f593739.
  • ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_LOADER_DEBUG=layer ctest -V -R '^(agc_driver_graphics|agc_sparse_.*|agc_heap_mirror_.*|agc_host_import.*)$' (15 tests).

Result:

  • 15/15 pass on both.
  • The loader shows VK_LAYER_KHRONOS_validation added in both (8 lines each).
  • The VUIDs are identical: VUID-vkCmdCopyBuffer-srcBuffer-00118 in agc_sparse_run_import_copied, the same pre-existing message as before. It is reported on this older base; current main no longer reports it.

@JJenkx
JJenkx force-pushed the perf/agc-sparse-mirror-range-run branch from c65f593 to 318ad53 Compare October 22, 2026 11:28
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 this covers your latest review, on head 98de5968c0 (rebased onto main 4777caca34):

  1. Partial-import regression: joinedHeapMirrorPartialImportTests, commit 98de5968c0. A heap bound across two committed ranges is mirrored over both, and an address build on an importing context imports the first range. The mirror must stay: same buffer, no rebuild. With only the release fix (318ad5365b) reverted, it fails with "an address build that imported one range of a joined heap mirror released the mirror". With it, 15/15 focused tests pass, the layer is added, and there are no VUIDs.
  2. fix(agc): bind heap mirrors only where the guest holds bytes #7647: coordinated on that PR. Neither depends on the other; git merge-tree shows two mechanical conflicts in GuestBufferMemory.cpp, and the resolution is described there. The description's Depends on line says so.
  3. Current main and head:
    • Paired focused validation-layer gate on main 4777caca34: 15/15 pass. main plus the test commit only fails the new case as intended. The change passes 15/15. The loader adds the layer in every arm, and no arm reports a VUID.
    • The perf pair is refreshed on the current code (present-matched, presents 450-549). The summary lines in the description are replaced, and the traces, digests, comparison and ctest logs are release assets at https://github.com/JJenkx/AnyPS5/releases/tag/pr9551-evidence.
    • The mean frame difference in this window is small (-3.0 %, p99 -16.7 %), and the description now says plainly that the certain result is the removed CPU work.

@JJenkx

JJenkx commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

Focused gate receipt for the current head, 98de5968c0, on base 4777caca34. This covers what was run and what was skipped. It makes no performance claim.

Build: one Ninja build directory for every arm, CMAKE_BUILD_TYPE=Release, GCC 16.2.1, ctest 4.4.4. Each arm rebuilt agc_driver_graphics_tests, agc_driver_recorder_tests, agc_host_import_sparse_tests and agc_host_import_retire_tests from a clean tree before testing.

Environment, the same for every arm:

ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_LOADER_DEBUG=layer
ctest -V -j1 -R '^(agc_driver_graphics|agc_sparse_.*|agc_heap_mirror_.*|agc_host_import.*)$'
  • Validation layer: Khronos validation 1.4.363 (implementation_version 1), loaded from a local install through VK_ADD_LAYER_PATH.
  • Machine: Linux 7.2.9, AMD Radeon RX 6650 XT, RADV, Mesa 26.2.4.
arm source result layer inserted VUIDs
main 4777caca34, clean 15/15 4 tests 0
main + the first test commit only 4777caca34 + df453b32e2, uncommitted 14/15: agc_driver_graphics fails with "a sparse heap whose committed span crosses an adjacent range was not mirrored" 4 tests 0
this PR 98de5968c0, clean 15/15 4 tests 0
this PR with the release fix reverted 98de5968c0 with 318ad5365b reverted, uncommitted 14/15: agc_driver_graphics fails with "an address build that imported one range of a joined heap mirror released the mirror" 4 tests 0

"Layer inserted" counts the loader's Insert instance layer "VK_LAYER_KHRONOS_validation" lines: the four tests that create a real Vulkan instance.

Skipped cases. Every arm prints the same two lines from agc_driver_graphics:

  • guest arena unavailable or not write-watched: heap mirrors not tested
  • guest arena unavailable or not write-watched: heap mirrors of imported ranges not tested

These are existing cases that need the Windows guest arena; they do not run on Linux on main either. Neither of this PR's new cases is skipped. Their skip messages ("sparse heap mirrors over adjacent ranges not tested" and "joined heap mirrors under a partial import not tested") appear in no log, and both of their assertions fire in the negative arms above.

Logs, with local paths redacted, in the pr9551-evidence release:

  • vvl9551e-ctest-main.log, vvl9551e-ctest-testonly.log
  • vvl9551f-ctest-change.log, vvl9551f-ctest-nofix.log

vvl9551e-ctest-change.log is from 318ad5365b, the head before the partial-import test commit, and is superseded by vvl9551f-ctest-change.log.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 23, 2026
@JJenkx
JJenkx force-pushed the perf/agc-sparse-mirror-range-run branch from 98de596 to caaed79 Compare October 23, 2026 17:53
@JJenkx

JJenkx commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on main e94545a592 (head caaed795da). Main had merged #7647, which binds heap mirrors only where the guest holds bytes, so this is a port onto that model, not a mechanical rebase:

Exact-head focused gate: one build directory, the same command for every arm, submodules synced per arm. Command: ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_LOADER_DEBUG=layer ctest -V -j1 -R '^(agc_driver_graphics|agc_sparse_.*|agc_heap_mirror_.*|agc_host_import.*)$'. Machine: Linux 7.2.9, RX 6650 XT, RADV (Mesa 26.2.4), GCC 16.2.1, Release.

arm source result validation layer inserted VUIDs
main e94545a592 16/16 4 tests 0
this PR with the partial-import fix reverted caaed795da with b5ed6fd8de reverted 15/16: agc_driver_graphics fails with "an address build that imported one range of a joined heap mirror released the mirror" 4 tests 0
this PR caaed795da 16/16 4 tests 0
  • Every arm prints only the two existing Windows guest-arena skips ("heap mirrors not tested", "heap mirrors of imported ranges not tested"). Neither of this PR's new cases is skipped.
  • The "past the budget … copying the sparse heap instead" line in each log is the expected output of main's agc_sparse_heap_mirror_budget test, not a failure.
  • Logs, with local paths redacted: vvl9551g-{main,nofix,change}.log in https://github.com/JJenkx/AnyPS5/releases/tag/pr9551-evidence

The performance evidence in the description is unchanged and keeps its limits: one warm integration pair, with dropped trace events, measured before this rebase.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 23, 2026
@JJenkx
JJenkx force-pushed the perf/agc-sparse-mirror-range-run branch from caaed79 to a5331de Compare October 23, 2026 18:45
@JJenkx

JJenkx commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Rebased again, onto main 12f7e1cd3f (head a5331de72d). After the previous rebase, #7292 merged and added two more imported.push_back(range->address) sites in AcquireRegistered. The branch still merged without a textual conflict but no longer compiled, because this PR records (address, bytes) pairs. The fix is folded into the partial-import commit (5c7c0ea870):

  • The piece-import path (importPieces, ranges over the import size limit) now records the summed length of the pieces it imported.
  • releaseImportedMirrors therefore drops a mirror only when the pieces cover it whole, the same rule as for a single import.

Exact-head gate: same command and environment as the receipt above, with submodules synced per arm.

arm source result validation layer inserted VUIDs
main 12f7e1cd3f 16/16 4 tests 0
this PR with the partial-import fix reverted a5331de72d with 5c7c0ea870 reverted 15/16: agc_driver_graphics fails with "an address build that imported one range of a joined heap mirror released the mirror" 4 tests 0
this PR a5331de72d 16/16 4 tests 0

Only the two existing Windows guest-arena skips appear. Logs: vvl9551h-{main,nofix,change}.log in https://github.com/JJenkx/AnyPS5/releases/tag/pr9551-evidence

There is no new test for the piece-import path itself. That path's existing #7292 tests pass with this change.

@adriadam10 adriadam10 self-assigned this Oct 23, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the joined-range lifetime, the partial-import guard, the piece-import byte sum and the two new bda_resources cases on the current head, merged onto 11d6efa06; the code compiles against the moved main (no stale imported call sites). Requesting changes on the process side, not the code:

  • Performance label: please attach to this PR the two aps5-summary.jsonl lines, both traces, and the tools/trace_digest.py output for main and for the change, from the same title, scene and machine. The fork branch evidence/pr9551 is not an attachment, and the description's summary has performance_compare output but no trace_digest output. (AGENTS.md: a speed claim shows trace_digest summaries of both arms.)
  • Your last review, @MarcZX10, is CHANGES_REQUESTED on 2026-10-22 11:27 and its points were answered on later heads: please re-review the current head or retire the request.
  • The description already says the warm pair measured p50 +3.3 % (worse) and mean -3.0 %, and that FPS is not established; keep that wording unless the attached traces show otherwise.

@JJenkx

JJenkx commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

@adriadam10 @MarcZX10 Added a “Direct links to the existing capture evidence” section with both summary files, both traces and both trace_digest outputs as individual GitHub release asset links, plus the two digest summary lines inline. I downloaded the linked captures and regenerated both digests with the current db6af9e tool; each matches its previously published digest byte for byte. These remain the already-reported integration runs (6784938b0095 vs1551d42a177d), not new captures. Both traces dropped events; the p50 +3.3% regression, mean -3.0% and FPS-not-established limits remain explicit. Implementation head a5331de is unchanged. Please re-review the current head and evidence links when convenient.

JJenkx and others added 4 commits October 24, 2026 04:21
… ranges

A sparse heap the guest grows by mapping an adjacent range is no longer
covered by one leased range, so bindSparseMirror refused it and every
use copied the heap's committed pieces on the CPU. The mirror now takes
a joined range over the run of continuing ranges (crossedRun), kept
while its member ranges stay the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… ranges

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An address-based build releases the heap mirror at the base of each range
it imports. A mirror over a joined run of ranges starts at its first range
but spans the others too, so importing that range alone released it, and
the next descriptor bind rebuilt it: in Echoes of the End, 140 releases
and rebuilds of a 544 MiB mirror in 480 s. A heap mirror is now released
only when the imported range covers its reserved span.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e of its ranges

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JJenkx
JJenkx force-pushed the perf/agc-sparse-mirror-range-run branch from a5331de to bc7d801 Compare October 24, 2026 09:34
@JJenkx

JJenkx commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on main aeeaefad98 as bc7d80164a: a merge with current main conflicted in GuestBufferMemory.cpp, although GitHub still showed this PR as mergeable. git range-diff shows the four patches unchanged.

Exact-head gate, Linux / RX 6650 XT / RADV (Mesa 26.2.4), ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_LOADER_DEBUG=layer ctest -V -j1 over agc_driver_graphics, agc_sparse_*, agc_heap_mirror_* and agc_host_import*:

  • main aeeaefad98: 16/16;
  • this PR with the partial-import fix (7a535e3333) reverted: 15/16, agc_driver_graphics fails with "an address build that imported one range of a joined heap mirror released the mirror";
  • this PR: 16/16.

The validation layer was inserted in 4 tests per arm, with 0 VUIDs. Logs: vvl9551-bc7d-*.log in https://github.com/JJenkx/AnyPS5/releases/tag/pr9551-evidence

Note for whichever of this PR and #8874 merges second: both add one line at the start of sweepMirrors (releaseExpiredJoinedHeaps() here, ReferenceStorage::Shared().ReleaseIdle() there). Keeping both lines resolves it; I will rebase the remaining one.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* needs rebase Conflicts with main, rebase needed perf Performance change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants