Skip to content

feat(libkernel): count committed ranges in sceKernelGetPageTableStats - #4725

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
akkadbakkad89:feat/kernel-page-table-stats
Oct 14, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
akkadbakkad89:feat/kernel-page-table-stats

Conversation

@akkadbakkad89

@akkadbakkad89 akkadbakkad89 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What

Builds on the sceKernelGetPageTableStats implementation merged in #4101 (issue #4067) by adding the usage counting it explicitly deferred: GuestPageTableUsage iterates the recorded protection ranges, charges one page-table entry per 2 MiB granule a range spans to the gpu pool when the range has GPU-access protection bits (0x30) and to the cpu pool otherwise, and follows changes through map/protect/unmap — so available reflects committed usage instead of staying at 2048. Totals, EFAULT-on-null and the KytyPS5 model (4 GiB pool / 2 MiB granules / 2048 entries) are unchanged from #4101.

Unverified constants remain recorded in docs/dev/debt/libkernel.md per conventions.

Tested

WSL Ubuntu 24.04, llvmpipe, on current main including #4101. guest_page_table_stats covers: EFAULT for each null argument, cpu accounting of a flexible mapping and its restore on unmap, cpu/gpu reclassification through sceKernelMprotect, and gpu accounting of a direct mapping. guest_memory (which now contains #4101's CheckPageTableStats), guest_memory_pool_stats, guest_addresses, guest_mlock, guest_msync all pass; check_conventions.py --base upstream/main clean. No console run — budget constants unverified on hardware.

Checklist

@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

Overlap note: #4101 implements the same function and predates this PR — my open-PR search missed it when I picked up #4067, apologies.

The difference between the two: #4101 reports the 2048/2048 totals statically ("KytyPS5 additionally counts the 2 MiB entries of the committed ranges as used; this PR does not"). This PR does that counting — GuestPageTableUsage iterates the recorded protection ranges, charges each 2 MiB granule a range spans to the gpu pool when it has GPU-access protection bits and to the cpu pool otherwise, and follows protection changes through mprotect, so available reflects actual committed usage rather than staying at 2048.

Same signature, constants, sources and EFAULT behavior otherwise. Happy to defer to whichever approach maintainers prefer.

@MarcZX10

Copy link
Copy Markdown
Contributor

The live-range accounting is a useful distinction from #4101’s fixed available count, and the added mapping/protection tests exercise that behavior. I checked the exact PR branch against current main: it is nine commits ahead and includes two merge commits carrying unrelated discussion-template and shader/AGC changes (the page-table implementation itself is a focused commit). Please rebase/cherry-pick the page-table work onto current main before merge, then coordinate with #4101 on which implementation should proceed; the two PRs modify the same export, and the dynamic accounting here is the more complete behavior if its assumptions/tests are validated. No pressure on merge order—maintainers can choose the path.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#4101 is merged now, so this conflicts with main in DirectMemory/Export.cpp and docs/dev/debt/libkernel.md. Please rebase onto current main (and drop the two unrelated merge commits MarcZX100 mentioned) so the diff is just the accounting change on top of the merged function.

Points to fix, since the PR says it follows KytyPS5's CountPageTableEntries:

  • DirectMemory.cpp:1124: Kyty charges a range to the CPU pool when it has any CPU access bit and to the GPU pool when it has GPU access bits, so a range with both counts in both. Here a range with 0x30 is charged to GPU only.
  • DirectMemory.cpp:1121: Kyty only counts committed range types; g_protections records every guest mapping, including ones that are not committed memory. Say which ranges are meant to count and filter accordingly.
  • DirectMemory.cpp:1123: entries are summed per range, so contiguous ranges smaller than 2 MiB that share a granule (any partial mprotect splits a range) each charge it again. Count each granule once per pool, or state that the double count matches Kyty and pin it in the test.
  • Export.cpp:399: with unverified constants, available can now drop to 0 where main reports 2048. Please show the numbers a typical start-up produces (e.g. the values printed at the call in PPSA30601) so we know the dynamic count doesn't starve a title that the static answer lets through.
  • Test: add a case with two sub-2 MiB neighbours in one granule and one for a CPU+GPU protection, with expected values taken from Kyty's rules, not from this implementation's output.

@akkadbakkad89
akkadbakkad89 force-pushed the feat/kernel-page-table-stats branch 2 times, most recently from c84d3c9 to 59391b6 Compare October 13, 2026 05:37
@akkadbakkad89 akkadbakkad89 changed the title feat(libkernel): implement sceKernelGetPageTableStats feat(libkernel): count committed ranges in sceKernelGetPageTableStats Oct 13, 2026
@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (88107b7, which includes the merged #4101) as requested — single focused commit, no merge commits, and the conflict with #4101's sceKernelGetPageTableStats is resolved by layering the live counting on top of it. Their CheckPageTableStats in guest_memory stays valid and passes under the counting implementation.

Since #4101 merged first, this PR's diff now reads as: static totals kept, available now computed from committed protection ranges. Rebuilt and retested on this base (WSL): guest_page_table_stats, guest_memory, guest_memory_pool_stats, guest_addresses, guest_mlock, guest_msync pass; check_conventions.py --base upstream/main clean.

AI-assisted (Devin).

@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

All five points addressed in 08774c0:

  1. Dual-access ranges — fixed: prot & 0x7 now charges the cpu pool and prot & GuestProtGpuReadWrite (0x30) the gpu pool, independently, so a 0x33 range counts in both like Kyty. The direct-mapping test (prot 0x33) now asserts both pools drop, which caught the earlier either/or mistake.

  2. Which ranges count — g_protections holds committed mappings only: DoReserveVirtual records into the separate g_reservations map and calls RecordProtection(addr, len, -1), so reservations are never recorded here. Within the map, each range is charged to each pool whose access bits it carries; a range with no cpu or gpu access bits (e.g. PROT_NONE or ACP-only protections) charges neither, matching Kyty's committed-range access classes. The debt entry now says this explicitly.

  3. Shared granules — kept Kyty's per-range sum (each range charges the granules it covers, so split neighbours in one granule charge it once each) rather than deduplicating, and pinned it in the new CheckSharedGranule: two adjacent 16 KiB maps inside one 2 MiB granule drop cpu_available by 2, not 1.

  4. Startup numbers — measured in guest_page_table_stats on this branch (WSL): a fresh process reports cpu 2048/2048, gpu 2048/2048 (guest image segments aren't recorded, so baseline use is 0); after a 6 MiB flexible map 2045/2048; a 2 MiB 0x33 map 2047/2047; two 16 KiB neighbours 2046/2048. available reaches 0 only past 4 GiB of committed coverage on a side — the same ceiling the constants imply on console — and reservations cannot inflate usage since they are not recorded.

  5. Tests — added CheckDualAccessMapping (0x33 charges both pools, restore on unmap) and CheckSharedGranule (the Kyty per-range double charge above), both with expected values derived from Kyty's rules. guest_page_table_stats, guest_memory, guest_memory_pool_stats, guest_addresses, guest_mlock, guest_msync pass; check_conventions.py clean.

AI-assisted (Devin).

@nicolasfalesy

Copy link
Copy Markdown
Contributor

Linux test (Arch, GCC 16.2.1, glibc, Release), this PR merged onto main 7e181fc together with #4725, #4719, #4585 and #4582 (no conflicts): guest_page_table_stats passes. With this PR's source change reverted (test kept), it fails on Linux, so the check guards the change on Linux too.

@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

Verified on WSL Ubuntu 24.04 as well — guest_page_table_stats and the neighboring guest_memory* tests pass here too.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the counting against KytyPS5's CountPageTableEntries (per committed range, cpu and gpu pools independent, no-access ranges skipped, clamped at 2048): same results. The five earlier points are addressed. The only conflict was the sceKernelGetPageTableStats line in docs/dev/debt/libkernel.md next to clock_settime; rebased onto main, kept your entry. Scoped build and ctest 150/150, and guest_page_table_stats fails without the change. Merging once the full Build is green.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 13, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 13, 2026
@akkadbakkad89
akkadbakkad89 force-pushed the feat/kernel-page-table-stats branch from b5d80ad to f9cc81d Compare October 13, 2026 19:05
@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (dacd785): single commit, the only conflict was the sceKernelGetPageTableStats entry in docs/dev/debt/libkernel.md (kept this entry next to the new pthread_atfork/clock_settime ones). Rebuilt on this base: guest_page_table_stats, guest_memory, guest_memory_pool_stats, guest_addresses, guest_mlock, guest_msync pass; check_conventions.py --base upstream/main clean.

AI-assisted (Devin).

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 13, 2026
@blawesom

Copy link
Copy Markdown
Contributor

Rebase handoff — rebased onto current main (docs-only conflict, both entries kept), authorship unchanged.

Verified on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, AMD RX 9070 XT / RADV):

To pick it up:

git fetch https://github.com/blawesom/AnyPS5 rebase/4725
git checkout <your branch>
git merge --ff-only FETCH_HEAD

Head is f9cc81d86.

@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

f9cc81d is the current head — already rebased onto dacd78525, so nothing to pick up on this side. Thanks for the RADV run and the suite verification.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
Charge each recorded protection ranges 2 MiB granules to the gpu
pool when the range has GPU-access protection bits and to the cpu
pool otherwise, so the available counts reflect committed usage
instead of staying at the 2048-entry budget. Follows KytyPS5s
counting model.
@akkadbakkad89
akkadbakkad89 force-pushed the feat/kernel-page-table-stats branch from f9cc81d to 975ed12 Compare October 14, 2026 05:14
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
@adriadam10

Copy link
Copy Markdown
Collaborator

Rebuilt on current main: scoped build and ctest green, guest_page_table_stats fails without the change and passes with it; the counting matches KytyPS5's CountPageTableEntries (cpu and gpu pools independent, clamped at 2048). Merging once the full Build is green.

@adriadam10
adriadam10 merged commit aa6b757 into boykopovar:main Oct 14, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants