Skip to content

perf(agc): answer small guest read checks without /proc/self/maps - #6179

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
milescoviello:perf/verify-read-probe
Oct 18, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
milescoviello:perf/verify-read-probe

Conversation

@milescoviello

@milescoviello milescoviello commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

What

On Linux kernels without PROCMAP_QUERY (before 6.11, including the stock Ubuntu 24.04 kernel), GuestMemory's verify() reads and parses /proc/self/maps up to the covering mapping for every range check that misses its page cache. That cache only covers the guest arena and the image, so a range anywhere else misses every time. ASTRO BOT (PPSA21564) checks the 96-byte Shader headers it passes, which live in the host heap, from RegisteredHeader (through SameHeader). It does this on every sceAgcCreatePrimState, ResolveGraphicsAbi and interpolant mapping while it loads.

Measured with a temporary counter around the /proc/self/maps fallback (the process has about 3,600 mappings at that stage):

PPSA21564, 200 s, warm shader cache main ab9a476 this change
/proc/self/maps reads in verify() 404,000, 25.7 s of CPU (63.6 µs each) 110,000, 1.8 s (16.4 µs each)
read checks answered by process_vm_readv — 17,260,000
at 140 s still in the warp tunnel past it, in the space scene with the planet and asteroids
frame counter in the window title at 190 s 2,589 3,474

The loader's checks run on its own thread, so the slow path throttled loading. With the fast path the title does about 40 times as many checks in the same time and gets further. The 110,000 remaining reads are write checks and ranges over 4 KiB.

Now, on such a kernel, a read-only check of at most 4 KiB first reads the range into a thread-local scratch buffer with process_vm_readv on its own process. If every byte comes back, the range is readable, which is all a read-only check asks. Anything else takes the existing path, which also builds the failure message:

  • a fault or a partial read;
  • EPERM (for example under a seccomp profile that blocks the call);
  • a write check;
  • a larger range;
  • a kernel with PROCMAP_QUERY.

agc_driver_graphics is registered a second time on Linux with APS5_NO_PROCMAP_QUERY=1, so the fallback is also exercised on kernels that have PROCMAP_QUERY. LiveStack gains read-only checks for a readable range and for two ranges that end in a guard page.

Tested

Ubuntu 24.04, kernel 6.8.0 (no PROCMAP_QUERY), GCC 13.3, Release with CI's configure flags, on main ab9a476:

  • agc_driver_graphics and agc_driver_graphics_maps_fallback pass.
  • PPSA21564, 200 s from a warm shader cache, with the temporary counter added to both builds and screenshots at 60, 100, 140 and 190 s: the table above.
  • The full ctest suite passes (1,225 tests).
  • With fix(agc): cache inaccessible guest pages #6184, which records inaccessible pages in the same page table, the probe would have answered for a page the table already holds as inaccessible (found by @nicolasfalesy). 2108971 runs the probe only for ranges with no recorded page state. Merged with fix(agc): cache inaccessible guest pages #6184, the previous head fails agc_driver_graphics_maps_fallback 2 of 2; this head passes it and agc_driver_graphics 3 of 3, and all 663 ^agc_ tests pass alone.
  • tools/check_conventions.py --base upstream/main passes.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

🤖 Generated with Claude Code

@github-actions github-actions Bot added build agc libSceAgc* labels Oct 15, 2026
@MarcZX10 MarcZX10 mentioned this pull request Oct 15, 2026
11 tasks done
@milescoviello
milescoviello force-pushed the perf/verify-read-probe branch 5 times, most recently from fedd378 to 4e97c2c Compare October 15, 2026 11:39
This was referenced Oct 15, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1341 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1368 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1395 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 2434 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 4448 core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 4650 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 4891 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5021 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 5371 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5426 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5499 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5925 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6044 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6185 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6186 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6188 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6192 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6273 core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6280 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6292 core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6300 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6320 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6360 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6412 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6481 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6486 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6490 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6499 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6501 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6503 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6535 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6615 core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6741 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6747 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6763 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6788 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6808 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6868 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6905 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6937 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6941 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6963 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7201 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 7223 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7332 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 7642 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp

@milescoviello
milescoviello force-pushed the perf/verify-read-probe branch from 4e97c2c to 0621f21 Compare October 15, 2026 12:17
@milescoviello
milescoviello force-pushed the perf/verify-read-probe branch from 0621f21 to 2bd87d2 Compare October 15, 2026 13:00
@milescoviello
milescoviello force-pushed the perf/verify-read-probe branch 2 times, most recently from e21b63a to 081d730 Compare October 17, 2026 18:41
@nicolasfalesy

Copy link
Copy Markdown
Contributor

Tested on Linux (Arch), c++ (GCC) 16.2.1 20260810, Release, BUILD_TESTING=ON, main at e3838332ee31af787ad311b7eadbe53d8d012e95. GPU tests ran on lavapipe (llvmpipe, LLVM 22.1.8).

#6179 and #6184 together fail #6179's new test agc_driver_graphics_maps_fallback. Each PR alone passes it (main has no such test). With both merged it fails (2 of 2):

AGC graphics: a checked inaccessible arena page was queried again instead of read from the page table

The message is the check at GuestAllocations.cpp:259 that #6184 adds. #6179 registers the test with APS5_NO_PROCMAP_QUERY=1 (CMakeLists.txt:1641 on its head), so it runs agc_driver_graphics_tests without PROCMAP_QUERY. In that case #6179's verify() (GuestMemory.cpp:680 on its head; the call is at :684) answers a read check of up to 4096 bytes with process_vm_readv before it looks at the page table (readableWithoutQuery, :667). The page in that check is readable behind the registry, so the direct read succeeds and Accessible returns true. #6184's cache for inaccessible pages is never consulted, and the check expects false.

With the readableWithoutQuery call at :684 removed from the merge, agc_driver_graphics_maps_fallback passes (2 of 2). So whichever of the two lands second has to look at the cache before the direct read, or the fast path must not answer for a page the cache already holds as inaccessible.

AI-assisted: yes

@milescoviello

Copy link
Copy Markdown
Contributor Author

Thanks @nicolasfalesy, that's the bug. The probe answered before the page table was consulted, so a page #6184 holds as inaccessible read as accessible whenever the host page was readable.

2108971 makes the probe step aside for any range with a recorded page state. readableWithoutQuery now returns false when any page of the range has a non-zero entry in the arena or image page table. That covers main's readable entries and #6184's PageInaccessible, so the check goes through describePages and the table answers it, as before this PR. Ranges with no recorded state, such as the Shader headers on the host heap that this PR is for, still take the probe.

Checked on Ubuntu 24.04 (kernel 6.8, no PROCMAP_QUERY), GCC 13.3, Release, RTX 4070 Ti:

So the two should now land in either order.

AI-assisted: yes (Claude Code)

milescoviello and others added 2 commits October 18, 2026 16:45
On Linux kernels without PROCMAP_QUERY (before 6.11, which includes the
stock Ubuntu 24.04 kernel), every guest range check that misses the page
cache reads and parses /proc/self/maps up to the covering mapping. The
cache only covers the guest arena and the image, so ranges in the host
heap always miss. ASTRO BOT (PPSA21564) checks 96-byte shader headers
there from RegisteredHeader (through SameHeader) on every
sceAgcCreatePrimState and interpolant mapping, about 3,700 times per
second while it loads.

A read-only check of at most 4 KiB on such a kernel now first reads the
range into a scratch buffer with process_vm_readv on the own process.
If every byte is read, the range is readable. Anything else (a fault,
a partial read, a refused call, a write check or a larger range) takes
the path it took before, which also builds the failure message.

agc_driver_graphics runs a second time with APS5_NO_PROCMAP_QUERY=1, so
the fallback path is covered on kernels that have PROCMAP_QUERY too, and
LiveStack gains read-only checks: a readable range, and ranges that end
in a guard page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…age table

The process_vm_readv probe answered a small read check before the page
table was looked at, so a page the table holds as inaccessible (as boykopovar#6184
records them) read as accessible whenever the host page was readable. The
probe now runs only for ranges with no recorded page state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@milescoviello
milescoviello force-pushed the perf/verify-read-probe branch from 2108971 to 4c50918 Compare October 18, 2026 16:48
@milescoviello

Copy link
Copy Markdown
Contributor Author

Rebased onto main 889cee5 (4c50918). The only conflict was in libSceAgcDriver/CMakeLists.txt, where #6054 added agc_driver_image_srgb_sample_copy_forced beside this branch's agc_driver_graphics_maps_fallback; both are kept. On the rebased head the 441 tests matching ^agc_driver_graphics|guest|stack|procmap|maps pass, agc_driver_graphics_maps_fallback among them (RTX 4070 Ti, 595.99.02, Ubuntu 24.04).

AI-assisted: yes (Claude Code)

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@adriadam10 adriadam10 self-assigned this Oct 18, 2026
@adriadam10
adriadam10 merged commit ae4d901 into boykopovar:main Oct 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants