[Swastik-51]

What

Fixes #1791.

On Linux, describePages queried PROCMAP_QUERY (or /proc/self/maps) but did not store the readable pages it found in the page table, so every uncached guest-range check repeated the query.

• Adds PageStates::record(), shared by both platforms, to populate the cached page state.
• Reuses the page-table caching the Windows VirtualQuery path already does.
• Stores Linux PROCMAP_QUERY results in the page table, and does the same for the /proc/self/maps path.
• Keeps the existing generation/invalidation mechanism and the memory validation semantics.

Depends on #861: on current main the Linux arena span is empty, so the cache stays inactive on Linux until #861 lands.

Tested

• python -m unittest discover tools/tests: 14/14 passed.
• python tools/check_conventions.py --base upstream/main: passed.
• git diff --check: passed.
• Not run locally: tools/check_test_registration.py (no Ninja/CMake build tree) and the Linux C++ build/tests (Windows environment).
• Linux and Windows runs by others are in the thread (Linux: PPSA15657 and PPSA28997 with fix(guest-arena): place guest address space in the PS5 window on Linux #861; Windows: agc_driver_graphics).

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: fix(guest-arena): place guest address space in the PS5 window on Linux #861
• AI-assisted:

[MarcZX10]

I traced the current-main dependency before reviewing the cache path. On Linux, GuestArena::Arena still leaves _base and _end at zero; GuestMemory::PageStates::initialize() therefore gets an empty arena span, and the image span is initialized only under _WIN32. The new Linux pages.record() calls consequently have no span to populate on this base. The new arena test also returns early when GuestArenaAvailable_nid_postfix() is false, so it does not exercise the cache on current Linux main.

This change needs #861 first to make its intended Linux arena behavior active. CONTRIBUTING asks dependent PRs to say so explicitly; please add Depends on #861 to the body and make the test evidence clear that Linux cache behavior is gated until that dependency lands. I have not approved the PR: its check rollup is empty, and the body says the Linux C++ build/tests were unavailable locally.

[MarcZX10]

Thanks for adding Depends on #861 and clarifying the gated Linux behavior. I checked the new upstream workflow attempts for head 8dc179c; Build, Conventions and Progress are all action_required, so no upstream checks have run yet. CONTRIBUTING says a maintainer must approve fork workflows. Since #861 is still open, this PR remains dependent on it even after the checks are approved.

[MarcZX10]

I checked the branch base against current upstream main 0447793: head 8dc179c diverges by 2 commits ahead and 4 behind. CONTRIBUTING requires PRs to be based on current main; please rebase when refreshing and rerun upstream checks after a maintainer approves the fork workflows. Keep the Depends on #861 note because that PR is still required for the Linux arena path.

[johnmaia]

Linux results, since the PR could only be tested on Windows: this fixes a large slowdown on kernels before 6.11.

Setup. Ubuntu 24.04.5, kernel 6.8 (no PROCMAP_QUERY), i3-12100, RX 7600 (RADV, Mesa 25.2.8), GCC 13.3 Release. Title: Disney Epic Mickey: Rebrushed (PPSA15657, UE4), on a stack of open PRs that gets it to the title screen (see #2255). Both runs below use the same build, 25 minutes, headless. The only difference is this PR plus #861.

Before. perf record on the busiest thread: about 85% of its time was under GuestMemory::(anon)::describePages, 74% from verify() and 11% from CompareMapped(). Nearly all of it was std::istringstream parsing /proc/self/maps. The process has about 5,400 mappings (518 KB of text), re-read for every uncached query. That thread sat at 100% while the CPU as a whole was about 80% idle.

	Without	With #1810 + #861
Logos and intro movies	5-6 fps	57-60 fps
Title screen	not reached in 25 min (3 fps in the opening cinematic)	reached at ~14 min, 11.4 fps
Frames presented in 25 min	4,323	23,801
First frame	~535 s	~541 s (loading-bound either way)

With the change, describePages no longer shows up. The busiest thread is about 64% busy, and its time is now in GPU submission ioctls under Recorder::Submit.

On kernel 7.0, where PROCMAP_QUERY is available, the build without this PR runs about the same as the build with it on 6.8 (logos ~60 fps, title screen ~10.6 fps). So the gain is on kernels before 6.11, which includes Ubuntu 24.04's default 6.8.

Tests on Linux (main + #861 + #1810): agc_driver_graphics_tests and guest_memory pass. One thing to check: the new arena test in tests/GuestAllocations.cpp returns early, because GuestArenaAvailable_nid_postfix() is false in the test binary even with #861. The same run prints guest arena unavailable or not write-watched: heap mirrors not tested. So the Linux path passes without being exercised there. In the title's process, where the arena is set up, the cache clearly works (the numbers above).

Tested and written by Claude (AI assistant) on a contributor's machine.

[jeffrey12cali]

Independent confirmation on a second title, with a measured frame-rate outcome, and one thing this change
appears to still need.

The measurement. A retail title (PPSA28997) on Linux, NVIDIA RTX 4070 Laptop, Hyprland/XWayland.
perf record on the two hot threads during gameplay: 50.8 % of all samples were inside libstdc++
std::istream, top self symbol std::istreambuf_iterator at 19.7 %, callers describePages ->
verify / DescribeCommittedUncached / CompareMapped. The game's own module was 3.0 %. Counted directly:
1,172,045 map queries and 64,207,427 parsed lines in one run, because the page table was never populated.

After the store-back fix, on the same tree, same seed, same input, three runs per arm:

	before	after
map queries per run	1,172,045	0 (query 0 calls)
42,772 verifications	~50 % of CPU	0.1 s
CPU-bound frame rate	7.40 FPS	60.32 FPS (56.5 median)
main menu	1635 s headless, never on a compositor	66 s

Zero fatal lines over a 13 minute soak with continuous input, and screenshots showed correct scenes
throughout.

One thing this change seems not to cover on its own. The store-back only helps if a span exists to store
into, and on Linux PageStates::initialize gets base = 0 size = 0 from GuestArenaRange, because the
Arena constructor in core/libs/prx/libc/src/GuestArena.cpp is entirely inside #ifdef _WIN32.
PageSpan::allocate() returns false on size == 0, spanOf() answers null, and the store has nowhere to
go. #1791 says the same thing ("the Linux arena span is empty ... so the table is not used on Linux at
all"), so this may just be a note that #1810 depends on #861 to be effective on Linux rather than a gap in
the diff. It is worth stating in the PR description either way, because with #1810 alone on current main
the measurable effect on Linux is nil.

A second, smaller gap. Two munmap calls in core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
trim the alignment prefix and suffix inside the aligned-mapping helper, and that helper runs outside any
GuestAllocations::Mutation. GuestAllocationsEnd is what fires the registered invalidator, so those two
ranges are unmapped without ForgetPages ever seeing them. Once readable bits can be cached, a bit set for
a prefix or suffix range before it is trimmed would survive the trim. Invalidating both ranges explicitly
closes it; that is two calls, and I can send them as a commit on this branch if useful.

One consequence worth knowing: enabling the span also activates the invalidator, because
PageStates::initialize only registers ForgetPages when a span actually allocated. Before this, no
invalidation was installed on Linux at all.

Written with AI assistance; the numbers above were measured on the hardware listed.

[popcoder565]

Native Windows run, since no Windows results were posted. Setup: Windows 10 (19045), RTX 3060 Laptop (NVIDIA 616.92), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release. Head 966d503a, merged onto main d70b899 in one build with 33 other AGC/shader PRs that had no Windows run.

• This PR's tests pass: agc_driver_graphics. None of them reported skipping a case.
• The change is Linux-only; on Windows this checks that it builds and that agc_driver_graphics (which includes GuestAllocations.cpp) passes.
• Full ctest in that build: 514 of 515. The one failure is fix(agc): defer a shader whose code does not decode instead of aborting the title #2257's new test, which is registered without the Windows PATH setup.

No title was run.

🤖 Addressed by Claude Code

[oneandonlydean]

Conventions fails because the description isn't in the PR template: it needs the What, Tested and Checklist sections (including AI-assisted yes/no). The change itself waits on #861, since the Linux arena span is empty without it.

[adriadam10]

Thanks for this, the Linux measurements in the thread (@johnmaia, @jeffrey12cali) show it is worth landing. Ran build + the AgcDriver ctest scope here: green. What's left:

• Conventions is red (as @oneandonlydean noted): the PR body has no ### What, ### Tested and ### Checklist sections (template in .github/pull_request_template.md). Please move the text under those headings.
• The new test never runs on Linux (core/libs/prx/libSceAgcDriver/tests/GuestAllocations.cpp:190): it returns when GuestArenaAvailable_nid_postfix() is false, so agc_driver_graphics also passes with GuestMemory.cpp reverted. Either make it exercise the cache without the arena (e.g. a span that doesn't depend on #861), or land it after #861 and show it failing without the change.
• Stale cached bits on the alignment trim (core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp:523-531): the prefix/suffix munmap calls run outside any GuestAllocations::Mutation, so ForgetPages never sees them. Once readable bits are cached they can outlive the unmap. Invalidate both ranges (as @jeffrey12cali offered) or explain why it can't matter.
• Keep Depends on #861: on current main the Linux span is empty and this change is inactive. It can merge once #861 does.

[oneandonlydean]

@Swastik-51 Rebased onto current main and pushed to the branch; your commits are unchanged. One commit on top:

• The arena test calls now pass the gpu argument that Mutation::Add/Protect take since c11a9cb (it didn't compile on main).
• The test now pins the cache: after a checked arena page, it revokes the page with mprotect behind the registry and requires the stored answer. With the GuestMemory.cpp change reverted it fails ("a checked arena page was queried again instead of read from the page table"). With fix(guest-arena): place guest address space in the PS5 window on Linux #861 the arena block runs on Linux; on main alone it still returns early.

Alignment trim (DirectMemory.cpp prefix/suffix munmap): with #861 that path is unreachable on Linux. MapPlaced returns through the arena first, and the arena either initialises or throws, so no cached page can outlive a trim.

Linux, GCC, Release: ctest 694/695 on main + #861 + this branch, and on main + this branch. agc_driver_graphics passes in both. The one failure, agc_driver_dcc_keys_first_write, fails the same way on main.

I also moved the description into the PR template so conventions can pass. Please fill in the AI-assisted answer at the end; only you can answer that.

[adriadam10]

The code is fine: build and the AgcDriver ctest scope are green, and agc_driver_graphics fails with the GuestMemory.cpp change reverted, so the cache test pins the fix.

One thing left: Conventions fails with Fill in the pull request template: answer AI-assisted with yes or no. @Swastik-51 please replace <!-- yes / no --> in the AI-assisted: line of the description with yes or no. Once that check is green this can be merged.

[Raghav314]

@Swastik-51 I tried this in Hades II (PPSA36082) on my machine.

Setup: Linux, RTX 4090, Ryzen 7 7800X3D, main aa79dbd plus a few local changes the game needs to get in game, with this PR's head (6f82d9f) merged on top. Warm caches, standing still in the hub for 20 s, one run each, same session:

	fps	p95 frame time
without this PR	25.8	41.2 ms
with this PR	32.7	32.5 ms

That's about 27% faster. It merged cleanly onto current main.

LLM assistance used.

[MarcZX10]

One status update from the current branch: #861 merged into main on October 10, so the stated dependency has landed. The latest conventions run still fails because the AI-assisted: line remains the template placeholder. The new overlap notice for #3454 is for the same GuestMemory.cpp file; #3454 extracts the procmap query helper while this PR adds the page-state cache, so they are distinct changes but the later merge may need a rebase.

[eliyarson]

Another Hades II (PPSA36082) measurement, on different hardware from @Raghav314's.

Setup: Linux (CachyOS, KDE Wayland), Ryzen 9 5950X, RTX 3080 Ti (NVIDIA 610.57.04), Release build. Base: main 0a8b1d3 plus #3850, #419, #2169 and #2121's own commit, which the title needs to reach gameplay. This PR's three commits were cherry-picked on top, resolving the ProcMapsQueryFd() → HostMapping::QueryFd() rename from #3454 by keeping main's name. A/B in one session with the same runtime and only libSceAgcDriver.prx swapped, standing still in the Crossroads hub, nothing else compiling, APS5_PROFILE_DRAW=1:

	without	with this PR
Window caption FPS (7 samples over ~20 s)	25.1–26.0	31.1–32.7
AGC worker time per draw packet	81.0 µs	62.6 µs
decode / capture / key+lookup+validate / Graphics::Draw (µs)	10.5 / 21.1 / 10.6 / 27.0	4.9 / 16.1 / 7.8 / 23.7

Before the change ioctl (PROCMAP_QUERY from describePages) was the top frame in 116 of 400 gdb samples of the AGC worker thread. agc_driver_graphics fails with the GuestMemory.cpp change reverted and passes with it. The worker thread stays at ~100% of one core afterwards, so the title remains CPU-bound on draw submission, but this is the largest single cost we found.

[oneandonlydean]

Closing: carried in #4395 with your commits and authorship kept, rebased over #3454, because the AI-assisted line here was never answered and only you can fill it in.