Repository navigation
feat(libSceFiber): add APS5_FIBER_GUARD guard pages on Windows - #4892
Closed
fdfawef324 wants to merge 1 commit into
Closed
fdfawef324 wants to merge 1 commit into
fdfawef324 wants to merge 1 commit into
Conversation
This was referenced Oct 13, 2026
Titles place fiber contexts back to back in guest memory without guard pages, so a fiber that overflows its stack overwrites the context below it and the title fails much later, far from the cause. With APS5_FIBER_GUARD=1 on Windows, _sceFiberInitializeImpl makes the lowest 4 KiB page of the context PAGE_NOACCESS and sceFiberFinalize, or initializing the object again, restores the protection the page had. A context that does not start on a page or is smaller than two pages gets no guard and the first one is named on stderr; any other value of the variable throws. With the context size check on, the fill skips the guarded page and sceFiberGetInfo starts its scan above it. On Linux the variable throws a not implemented error at the first initialize. A guest unmap of arena memory there maps the range again as a PROT_NONE reservation (GuestArenaReset), which the earlier Linux path took for the guard and gave the old read-write protection at the last finalize; the Linux mapping and protection paths in libkernel do not go through one registry that could drop or keep a guard. Each guard is owned by the fiber that set it through a ticket. Fibers that share a context share the guard and the last finalize removes it. Releasing a guard whose ticket is not registered throws, except once for a ticket whose guard a guest unmap or mapping dropped, so a fiber whose page was unmapped and given to a new fiber cannot remove the new fiber's guard. Sharing a guard and the last finalize check that the page is still committed PAGE_NOACCESS and throw otherwise, so a direct VirtualProtect by host code is reported instead of the page being silently left unguarded or given a stale protection. A fiber whose guard is refused is left uninitialized and unpinned, and a finalize that throws leaves the fiber initialized. The guard is registered in WindowsMappings, and every guest protection change and unmap goes through it: - HandleWrite no longer takes a write fault in a guard page for a write to an armed view; before, it reopened the 4 KiB page writable and resumed the overflowing write; - the tracker's VirtualProtect calls in Commit, Pin, HandleWrite, BeginHostWrite and the per-page arming in Collect go through one helper that protects the pages around each guard in the range, never the guard itself, and records the protection to restore at finalize, so arming or opening a run of 4 KiB pages of a view leaves its guard PAGE_NOACCESS; Map keeps its direct VirtualProtect because reset has already dropped every guard in the range; - Collect neither reports, settles nor arms a guarded 4 KiB page of a shared view, so the guard is never taken for a CPU write, and the page's pending state from before the guard is reported after the guard is removed; every other page keeps the per-page tracking; - fresh collection (boykopovar#4535) skips a guarded page the way Collect does: it neither activates nor reports the page, and does not count it as fresh when it decides whether anything is fresh or whether an inaccessible view fails. Guard takes the page out of the fresh ranges, and removing or dropping the guard puts it back if it was never collected; every other page keeps the deferred activation; - the libkernel mprotect loop sends every committed region through that helper (GuestArenaProtect_nid_no_patch), inside the guest arena and outside it. Memory outside the arena, such as the title image, went to VirtualProtect directly before: a guest PROT_NONE of the guarded page left it PAGE_NOACCESS, the check above passed, and the last finalize made the page writable again with the protection read at initialize. A range that crosses the arena boundary took the same direct path; - unmapping or remapping arena memory (reset), and the libkernel munmap of memory outside the arena (GuestArenaDecommit_nid_no_patch and GuestArenaReleaseAllocation_nid_no_patch, which call VirtualFree), drop the guard and record its tickets. An unmap outside the arena left the guard registered before, and the next finalize threw. Guard refuses a range that is not made of whole 4 KiB pages. The context is pinned writable before the guard is set, so no view of the guarded page is armed while it is guarded, and pinned pages other than the guard are collected as written as before. A protection change is not a write for GetWriteWatch (Microsoft VirtualProtect and GetWriteWatch documentation), so the guard adds no written page. Test: fiber_guard, registered on Windows only, runs fiber_tests guard with APS5_FIBER_GUARD=1 on host, flexible and two-view direct memory: the lowest page is inaccessible after initialize and readable again after finalize, the fibers run and return, HandleWrite refuses a fault in the guard, collection, a second pin and guest mprotect keep it, no write is collected in the guard page, a pinned shared context is collected as written except its guard page, and a write to the former guard page is collected as one page in both views after finalize. It also unmaps a guarded context, maps it again at the same address for a second fiber and finalizes the first fiber, which must not remove the second fiber's guard; reinitializes one of two fibers sharing a guarded context with the size check on; releases unknown, twice used and dropped tickets, which throw except for the first release of a dropped one; and changes the guard page with VirtualProtect, after which sharing the guard and the last finalize throw and finalize succeeds once the guard is back. CheckGuardProtectedOutsideArena guards a context in the test executable's .bss, makes it PROT_NONE with sceKernelMprotect and finalizes; the page must stay PAGE_NOACCESS. The same with read-only must keep the guard and be restored at finalize. CheckGuardUnmappedOutsideArena maps flexible memory at a fixed address outside the arena, guards both halves, unmaps them (decommit, then release) and finalizes each fiber, which must succeed once, then maps the range again PROT_NONE and guards it for a new fiber. CheckSharedWritesAroundGuard in guest_memory guards the lowest 4 KiB page of a 16 KiB shared view, writes the other three, collects and re-arms: the guard stays PAGE_NOACCESS, a fault in it is not resumed, exactly the three written pages are reported in each of two rounds, and after the guard is removed the page is reported once, because it was never collected, then armed, and a write to it is reported alone. CheckFreshAroundGuard in windows_write_watch writes the second 4 KiB page of a freshly mapped shared view, guards it and collects fresh: the other fifteen pages are reported and activated, the guard stays PAGE_NOACCESS through activation and a re-arm beside it, and it is neither reported, activated nor counted as fresh, also in a PROT_NONE view. After the guard is removed it is reported once by fresh collection, activated, and a write to it is reported. Without the guard handling in fresh collection it fails at "fresh collection reported a guarded page or lost a fresh one". fiber_tests renames its .eh_frame like the other test executables so that it can catch exceptions thrown in libc.prx. fiber checks that nothing is guarded without the variable. Without the mprotect change fiber_guard fails at "finalize keeps the guest PROT_NONE of the former guard page"; without the munmap change it fails at "a guard dropped by a decommit outside the guest arena is released once". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fdfawef324
force-pushed
the
feat/fiber-guard-page
branch
from
October 13, 2026 05:57
6acb312 to
46a9b9d
Compare
Contributor
Author
|
Rebased onto ea2bbef, after #4661 (per-4 KiB write tracking of shared views) and #4535 (deferred write tracking for fresh Windows shared imports, with b24e1cc's early return). The rebase conflicted in
|
This was referenced Oct 13, 2026
This was referenced Oct 13, 2026
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Titles place fiber contexts back to back in guest memory with nothing between them. A fiber that overflows its stack overwrites the context below it, and the title fails much later, far from the cause.
APS5_FIBER_GUARD=1is an opt-in debugging switch, implemented on Windows. With it set:_sceFiberInitializeImplmakes the lowest 4 KiB page of the contextPAGE_NOACCESS, so an overflow faults at the instruction that causes it.sceFiberFinalize, or initializing the same object again, puts back the protection the page had.mallocare usually aligned to 16 bytes only, so in such titles most contexts may get no guard;USAGE.mdsays so.sceFiberGetInfostarts its scan above it.On Linux the variable throws a not implemented error at the first initialize. A guest unmap of arena memory there maps the range again as a
PROT_NONEreservation, which cannot be told apart from the guard, and the Linux mapping and protection paths do not go through one registry. This is recorded indocs/dev/debt/libSceFiber.md.Ownership of a guard
Each guard is owned through a ticket, which the fiber stores next to its guard address. Fibers that share one context share the guard, and the last finalize removes it. Releasing a ticket that is not registered throws
std::logic_error, except once for a ticket whose guard a guest unmap or mapping dropped. So a fiber whose context was unmapped and given to a new fiber cannot remove the new fiber's guard. Sharing a guard and the last finalize check that the page is still committedPAGE_NOACCESSand throw otherwise. A fiber whose guard is refused is left uninitialized and unpinned, and a finalize that throws leaves the fiber initialized.One registry for every guest protection change and unmap
The guard is registered in
WindowsMappings. Every path that changes a guarded page goes through it:HandleWrite. A write fault in the guard page of an armed shared view used to reopen that 4 KiB page and resume the overflowing write. It now returns false for a guarded address, so the fault reaches the crash report.VirtualProtectcalls inCommit,Pin,HandleWrite,BeginHostWriteand the per-page arming inCollectgo through oneprotecthelper. It protects the pages around each guard in the range and never the guard itself, and records the requested protection for finalize to restore.Mapkeeps its direct call becauseresethas already dropped every guard in the range.Collectneither reports, settles nor arms a guarded 4 KiB page of a shared view. The guard is never taken for a CPU write, and the page's pending state from before the guard is reported after the guard is removed. Fresh collection (fix(agc): defer write tracking for fresh Windows shared imports #4535) skips the guarded page the same way.mprotect. The libkernel loop now sends every committed region through that helper (GuestArenaProtect_nid_no_patch, which no longer requires arena memory). Before this revision, memory outside the guest arena, such as the title image, went toVirtualProtectdirectly. A guestPROT_NONEof the guarded page left itPAGE_NOACCESS, the intact check passed, and the last finalize made the page writable again with the protection read at initialize. A range that crosses the arena boundary took the same direct path.resethandles arena memory. New in this revision, the libkernelmunmapof memory outside the arena goes throughGuestArenaDecommit_nid_no_patchandGuestArenaReleaseAllocation_nid_no_patch. Both drop the guard and record its tickets. Before, an unmap outside the arena left the guard registered and the next finalize threw.Host code that calls
VirtualProtectdirectly on guest memory (the restore of guest thread stack pages at thread exit,APS5_WATCH_WRITE) is still outside the registry. The intact check reports such a change unless it leaves the pagePAGE_NOACCESS. The debt entry lists this case.Per-page write tracking (#4661) and deferred activation (#4535)
Since #4661 the tracker keeps the write generation, seen generation and write protection per 4 KiB page of a 16 KiB shared view, and arms or opens each contiguous run of pages with one
VirtualProtect. The guard sits on top of that state: every run goes throughprotect, which skips the guard and records the run's protection as the one finalize restores, andCollectleaves the guarded page out, so the guard staysPAGE_NOACCESSthrough every arm and re-arm, finalize hands the page back with the protection and armed state the tracker expects, and every page that is not a guard is tracked exactly as onmain.Guardrefuses a range that is not made of whole 4 KiB pages.Since #4535 a freshly imported shared view is not armed at import: each view keeps a per-4 KiB collected mask, the never-collected pages are kept as fresh address ranges, armed collection skips pages that were never collected, and fresh collection activates them (reports and arms them) before
MarkWrittenstamps a GPU write. A guard page is never armed, activated, reported or reopened by either path. Fresh collection leaves a guarded page out the wayCollectdoes, so it is neither reported nor activated, and it does not count the page as fresh:Guardtakes the page out of the fresh ranges, which keeps the early return when nothing else is fresh, and an inaccessible view does not fail fresh collection because of its guard. When the guard is removed or dropped, a page that was never collected is fresh again and is reported once by the next fresh collection. Every page that is not a guard keeps #4535's deferred activation.The context is pinned writable before the guard is set, so no view of the guarded page is armed while it is guarded; pinned pages other than the guard are still collected as written. A protection change is not a write for
GetWriteWatch, so the guard adds no written page.Sources: Microsoft documentation for
VirtualProtect,VirtualFreeandGetWriteWatch.Tested
OS Windows Server 2025, RTX 3060, GCC 15.2 (winlibs MinGW-W64 UCRT),
build-dev. Rebased onupstream/mainea2bbef, after #4661 and #4535. The rebase conflicted inWindowsMappings.hpponly, inCollect(#4535's fresh selection next to the guard skip) and in the member list; the runs below are on the rebased commit.fiber_guardrunsfiber_tests guardwithAPS5_FIBER_GUARD=1on host, flexible (write-watched) and two-view direct memory. It covers:HandleWriterefusing a fault in the guard;sceKernelMprotectkeeping the guard, and write collection around it;VirtualProtectof the guard, after which sharing and the last finalize must throw;CheckGuardProtectedOutsideArenaandCheckGuardUnmappedOutsideArena, a guestmprotectandmunmapof a guarded context outside the guest arena.CheckGuardedSharedContextthe pinned four-view context is collected as 15 pages, none of them in the guard (16 before, the guard included), and a write to the former guard page after finalize is collected as one page in each view (4 before, the whole view).CheckSharedWritesAroundGuardinguest_memoryguards the lowest 4 KiB page of a freshly mapped 16 KiB shared view, writes the other three pages, collects and re-arms. The guard must stayPAGE_NOACCESS, a fault in it must not be resumed, and exactly the three written pages must be reported, in two rounds. After the guard is removed, the page is reported once, because it was never collected (Collectkept that state while it was guarded), then armed, and a write to it is reported alone. A second guard on the now armed page must survive a pin of the whole view.CheckFreshAroundGuardinwindows_write_watchwrites the second 4 KiB page of a freshly mapped shared view, guards it and collects fresh. The other fifteen pages must be reported and activated, and the guard must stayPAGE_NOACCESSthrough the activation and a re-arm of the page beside it. The guarded page must be neither reported, activated nor counted as fresh, also in aPROT_NONEview, where it must not make fresh collection fail. After the guard is removed, the page must be reported once by fresh collection, activated, and a write to it reported.VirtualProtectdirectly, as a plain take of themainside of the conflict gives,guest_memoryfails at the check after the pin (GuestMemory.cpp:1379).Collectnot leaving the guard out,guest_memoryfails at the first collect (GuestMemory.cpp:1364), andfiber_guardfails atthe pinned shared context but its guard page is collected as written.Guardleaves it in the fresh ranges),windows_write_watchfails atfresh collection reported a guarded page or lost a fresh one. With only the last two removed it fails atguarded page made an inaccessible view fresh, and without putting the page back in the fresh ranges atUnguardit fails atunguarded page was not reported as fresh.ctest -j 4):fiber,fiber_guard): 2/2.log_macros, which also fails onmain.guest_memoryincluded: 122/122.windows_write_watch: 1/1.agc_write_tracking: 1/1.agc_shared_write,agc_shared_write_partial_collect,agc_shared_write_partial_collect_two_pass: 3/3.check_conventions.pyfromupstream/mainpasses.Export.cppand the non-Windows branch ofFiber.cppdid not change in this revision.agc_write_trackingandagc_shared_write*were not run.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code