Skip to content

feat(libSceFiber): add APS5_FIBER_GUARD guard pages on Windows - #4892

Closed
fdfawef324 wants to merge 1 commit into
boykopovar:mainfrom
fdfawef324:feat/fiber-guard-page
Closed

fdfawef324 wants to merge 1 commit into
boykopovar:mainfrom
fdfawef324:feat/fiber-guard-page

Conversation

@fdfawef324

@fdfawef324 fdfawef324 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Titles place fiber contexts back to back in guest memory with nothing between them. A fiber that overflows its stack overwrites the context below it, and the title fails much later, far from the cause.

APS5_FIBER_GUARD=1 is an opt-in debugging switch, implemented on Windows. With it set:

  • _sceFiberInitializeImpl makes the lowest 4 KiB page of the context PAGE_NOACCESS, so an overflow faults at the instruction that causes it.
  • sceFiberFinalize, or initializing the same object again, puts back the protection the page had.
  • A context that does not start on a 4 KiB page, or is smaller than two pages, gets no guard. Only the first such context is named on stderr. Contexts from malloc are usually aligned to 16 bytes only, so in such titles most contexts may get no guard; USAGE.md says so.
  • Any other value throws at the first initialize.
  • With the context size check on, the fill skips the guarded page, and sceFiberGetInfo starts its scan above it.

On Linux the variable throws a not implemented error at the first initialize. A guest unmap of arena memory there maps the range again as a PROT_NONE reservation, which cannot be told apart from the guard, and the Linux mapping and protection paths do not go through one registry. This is recorded in docs/dev/debt/libSceFiber.md.

Ownership of a guard

Each guard is owned through a ticket, which the fiber stores next to its guard address. Fibers that share one context share the guard, and the last finalize removes it. Releasing a ticket that is not registered throws std::logic_error, except once for a ticket whose guard a guest unmap or mapping dropped. So a fiber whose context was unmapped and given to a new fiber cannot remove the new fiber's guard. Sharing a guard and the last finalize check that the page is still committed PAGE_NOACCESS and throw otherwise. A fiber whose guard is refused is left uninitialized and unpinned, and a finalize that throws leaves the fiber initialized.

One registry for every guest protection change and unmap

The guard is registered in WindowsMappings. Every path that changes a guarded page goes through it:

  1. HandleWrite. A write fault in the guard page of an armed shared view used to reopen that 4 KiB page and resume the overflowing write. It now returns false for a guarded address, so the fault reaches the crash report.
  2. The tracker's own VirtualProtect calls in Commit, Pin, HandleWrite, BeginHostWrite and the per-page arming in Collect go through one protect helper. It protects the pages around each guard in the range and never the guard itself, and records the requested protection for finalize to restore. Map keeps its direct call because reset has already dropped every guard in the range.
  3. Collect neither reports, settles nor arms a guarded 4 KiB page of a shared view. The guard is never taken for a CPU write, and the page's pending state from before the guard is reported after the guard is removed. Fresh collection (fix(agc): defer write tracking for fresh Windows shared imports #4535) skips the guarded page the same way.
  4. Guest mprotect. The libkernel loop now sends every committed region through that helper (GuestArenaProtect_nid_no_patch, which no longer requires arena memory). Before this revision, memory outside the guest arena, such as the title image, went to VirtualProtect directly. A guest PROT_NONE of the guarded page left it PAGE_NOACCESS, the intact check passed, and the last finalize made the page writable again with the protection read at initialize. A range that crosses the arena boundary took the same direct path.
  5. Guest unmap. reset handles arena memory. New in this revision, the libkernel munmap of memory outside the arena goes through GuestArenaDecommit_nid_no_patch and GuestArenaReleaseAllocation_nid_no_patch. Both drop the guard and record its tickets. Before, an unmap outside the arena left the guard registered and the next finalize threw.

Host code that calls VirtualProtect directly on guest memory (the restore of guest thread stack pages at thread exit, APS5_WATCH_WRITE) is still outside the registry. The intact check reports such a change unless it leaves the page PAGE_NOACCESS. The debt entry lists this case.

Per-page write tracking (#4661) and deferred activation (#4535)

Since #4661 the tracker keeps the write generation, seen generation and write protection per 4 KiB page of a 16 KiB shared view, and arms or opens each contiguous run of pages with one VirtualProtect. The guard sits on top of that state: every run goes through protect, which skips the guard and records the run's protection as the one finalize restores, and Collect leaves the guarded page out, so the guard stays PAGE_NOACCESS through every arm and re-arm, finalize hands the page back with the protection and armed state the tracker expects, and every page that is not a guard is tracked exactly as on main. Guard refuses a range that is not made of whole 4 KiB pages.

Since #4535 a freshly imported shared view is not armed at import: each view keeps a per-4 KiB collected mask, the never-collected pages are kept as fresh address ranges, armed collection skips pages that were never collected, and fresh collection activates them (reports and arms them) before MarkWritten stamps a GPU write. A guard page is never armed, activated, reported or reopened by either path. Fresh collection leaves a guarded page out the way Collect does, so it is neither reported nor activated, and it does not count the page as fresh: Guard takes the page out of the fresh ranges, which keeps the early return when nothing else is fresh, and an inaccessible view does not fail fresh collection because of its guard. When the guard is removed or dropped, a page that was never collected is fresh again and is reported once by the next fresh collection. Every page that is not a guard keeps #4535's deferred activation.

The context is pinned writable before the guard is set, so no view of the guarded page is armed while it is guarded; pinned pages other than the guard are still collected as written. A protection change is not a write for GetWriteWatch, so the guard adds no written page.

Sources: Microsoft documentation for VirtualProtect, VirtualFree and GetWriteWatch.

Tested

OS Windows Server 2025, RTX 3060, GCC 15.2 (winlibs MinGW-W64 UCRT), build-dev. Rebased on upstream/main ea2bbef, after #4661 and #4535. The rebase conflicted in WindowsMappings.hpp only, in Collect (#4535's fresh selection next to the guard skip) and in the member list; the runs below are on the rebased commit.

  • fiber_guard runs fiber_tests guard with APS5_FIBER_GUARD=1 on host, flexible (write-watched) and two-view direct memory. It covers:
    • guard and restore, and fibers running on a guarded context;
    • HandleWrite refusing a fault in the guard;
    • collection, pins and guest sceKernelMprotect keeping the guard, and write collection around it;
    • unmap and remap at the same address for a second fiber;
    • two fibers sharing a context with the size check on;
    • ticket release rules;
    • a direct VirtualProtect of the guard, after which sharing and the last finalize must throw;
    • CheckGuardProtectedOutsideArena and CheckGuardUnmappedOutsideArena, a guest mprotect and munmap of a guarded context outside the guest arena.
  • Changed for fix(libc): track writes to shared guest memory per 4 KiB page on Windows #4661: in CheckGuardedSharedContext the pinned four-view context is collected as 15 pages, none of them in the guard (16 before, the guard included), and a write to the former guard page after finalize is collected as one page in each view (4 before, the whole view).
  • New: CheckSharedWritesAroundGuard in guest_memory guards the lowest 4 KiB page of a freshly mapped 16 KiB shared view, writes the other three pages, collects and re-arms. The guard must stay PAGE_NOACCESS, a fault in it must not be resumed, and exactly the three written pages must be reported, in two rounds. After the guard is removed, the page is reported once, because it was never collected (Collect kept that state while it was guarded), then armed, and a write to it is reported alone. A second guard on the now armed page must survive a pin of the whole view.
  • New for fix(agc): defer write tracking for fresh Windows shared imports #4535: CheckFreshAroundGuard in windows_write_watch writes the second 4 KiB page of a freshly mapped shared view, guards it and collects fresh. The other fifteen pages must be reported and activated, and the guard must stay PAGE_NOACCESS through the activation and a re-arm of the page beside it. The guarded page must be neither reported, activated nor counted as fresh, also in a PROT_NONE view, where it must not make fresh collection fail. After the guard is removed, the page must be reported once by fresh collection, activated, and a write to it reported.
  • Red and green:
    • With the subpage runs going to VirtualProtect directly, as a plain take of the main side of the conflict gives, guest_memory fails at the check after the pin (GuestMemory.cpp:1379).
    • With Collect not leaving the guard out, guest_memory fails at the first collect (GuestMemory.cpp:1364), and fiber_guard fails at the pinned shared context but its guard page is collected as written.
    • With the guard handling removed from the fresh-collection path (the fresh selection and the inner loop no longer leave the guard out, the fresh check of an inaccessible view counts it, and Guard leaves it in the fresh ranges), windows_write_watch fails at fresh collection reported a guarded page or lost a fresh one. With only the last two removed it fails at guarded page made an inaccessible view fresh, and without putting the page back in the fresh ranges at Unguard it fails at unguarded page was not reported as fresh.
    • At HEAD all of them pass.
  • Library test sets at HEAD (ctest -j 4):
    • libSceFiber directory (fiber, fiber_guard): 2/2.
    • libc directory: 82/83. The failure is log_macros, which also fails on main.
    • libkernel directory, guest_memory included: 122/122.
    • windows_write_watch: 1/1.
    • agc_write_tracking: 1/1.
    • agc_shared_write, agc_shared_write_partial_collect, agc_shared_write_partial_collect_two_pass: 3/3.
  • check_conventions.py from upstream/main passes.
  • Not run:
    • The Linux path was not built or run (no Linux host). Export.cpp and the non-Windows branch of Fiber.cpp did not change in this revision.
    • The AGC driver tests other than agc_write_tracking and agc_shared_write* were not run.
    • Not tested with a title.
  • No speed change is claimed.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes

🤖 Generated with Claude Code

Titles place fiber contexts back to back in guest memory without guard
pages, so a fiber that overflows its stack overwrites the context below
it and the title fails much later, far from the cause.

With APS5_FIBER_GUARD=1 on Windows, _sceFiberInitializeImpl makes the
lowest 4 KiB page of the context PAGE_NOACCESS and sceFiberFinalize, or
initializing the object again, restores the protection the page had. A
context that does not start on a page or is smaller than two pages gets
no guard and the first one is named on stderr; any other value of the
variable throws. With the context size check on, the fill skips the
guarded page and sceFiberGetInfo starts its scan above it.

On Linux the variable throws a not implemented error at the first
initialize. A guest unmap of arena memory there maps the range again as
a PROT_NONE reservation (GuestArenaReset), which the earlier Linux path
took for the guard and gave the old read-write protection at the last
finalize; the Linux mapping and protection paths in libkernel do not go
through one registry that could drop or keep a guard.

Each guard is owned by the fiber that set it through a ticket. Fibers
that share a context share the guard and the last finalize removes it.
Releasing a guard whose ticket is not registered throws, except once for
a ticket whose guard a guest unmap or mapping dropped, so a fiber whose
page was unmapped and given to a new fiber cannot remove the new fiber's
guard. Sharing a guard and the last finalize check that the page is
still committed PAGE_NOACCESS and throw otherwise, so a direct
VirtualProtect by host code is reported instead of the page being
silently left unguarded or given a stale protection. A fiber whose guard
is refused is left uninitialized and unpinned, and a finalize that
throws leaves the fiber initialized.

The guard is registered in WindowsMappings, and every guest protection
change and unmap goes through it:
- HandleWrite no longer takes a write fault in a guard page for a write
  to an armed view; before, it reopened the 4 KiB page writable and
  resumed the overflowing write;
- the tracker's VirtualProtect calls in Commit, Pin, HandleWrite,
  BeginHostWrite and the per-page arming in Collect go through one
  helper that protects the pages around each guard in the range, never
  the guard itself, and records the protection to restore at finalize,
  so arming or opening a run of 4 KiB pages of a view leaves its guard
  PAGE_NOACCESS; Map keeps its direct VirtualProtect because reset has
  already dropped every guard in the range;
- Collect neither reports, settles nor arms a guarded 4 KiB page of a
  shared view, so the guard is never taken for a CPU write, and the
  page's pending state from before the guard is reported after the guard
  is removed; every other page keeps the per-page tracking;
- fresh collection (boykopovar#4535) skips a guarded page the way Collect does: it
  neither activates nor reports the page, and does not count it as fresh
  when it decides whether anything is fresh or whether an inaccessible
  view fails. Guard takes the page out of the fresh ranges, and removing
  or dropping the guard puts it back if it was never collected; every
  other page keeps the deferred activation;
- the libkernel mprotect loop sends every committed region through that
  helper (GuestArenaProtect_nid_no_patch), inside the guest arena and
  outside it. Memory outside the arena, such as the title image, went
  to VirtualProtect directly before: a guest PROT_NONE of the guarded
  page left it PAGE_NOACCESS, the check above passed, and the last
  finalize made the page writable again with the protection read at
  initialize. A range that crosses the arena boundary took the same
  direct path;
- unmapping or remapping arena memory (reset), and the libkernel munmap
  of memory outside the arena (GuestArenaDecommit_nid_no_patch and
  GuestArenaReleaseAllocation_nid_no_patch, which call VirtualFree),
  drop the guard and record its tickets. An unmap outside the arena
  left the guard registered before, and the next finalize threw.
Guard refuses a range that is not made of whole 4 KiB pages. The
context is pinned writable before the guard is set, so no view of the
guarded page is armed while it is guarded, and pinned pages other than
the guard are collected as written as before. A protection change is not a write for
GetWriteWatch (Microsoft VirtualProtect and GetWriteWatch documentation),
so the guard adds no written page.

Test: fiber_guard, registered on Windows only, runs fiber_tests guard
with APS5_FIBER_GUARD=1 on host, flexible and two-view direct memory:
the lowest page is inaccessible after initialize and readable again
after finalize, the fibers run and return, HandleWrite refuses a fault
in the guard, collection, a second pin and guest mprotect keep it, no
write is collected in the guard page, a pinned shared context is
collected as written except its guard page, and a write to the former
guard page is collected as one page in both views after finalize. It also unmaps a
guarded context, maps it again at the same address for a second fiber
and finalizes the first fiber, which must not remove the second fiber's
guard; reinitializes one of two fibers sharing a guarded context with
the size check on; releases unknown, twice used and dropped tickets,
which throw except for the first release of a dropped one; and changes
the guard page with VirtualProtect, after which sharing the guard and
the last finalize throw and finalize succeeds once the guard is back.
CheckGuardProtectedOutsideArena guards a context in the test
executable's .bss, makes it PROT_NONE with sceKernelMprotect and
finalizes; the page must stay PAGE_NOACCESS. The same with read-only
must keep the guard and be restored at finalize.
CheckGuardUnmappedOutsideArena maps flexible memory at a fixed address
outside the arena, guards both halves, unmaps them (decommit, then
release) and finalizes each fiber, which must succeed once, then maps
the range again PROT_NONE and guards it for a new fiber.
CheckSharedWritesAroundGuard in guest_memory guards the lowest 4 KiB
page of a 16 KiB shared view, writes the other three, collects and
re-arms: the guard stays PAGE_NOACCESS, a fault in it is not resumed,
exactly the three written pages are reported in each of two rounds, and
after the guard is removed the page is reported once, because it was
never collected, then armed, and a write to it is reported alone.
CheckFreshAroundGuard in windows_write_watch writes the second 4 KiB
page of a freshly mapped shared view, guards it and collects fresh: the
other fifteen pages are reported and activated, the guard stays
PAGE_NOACCESS through activation and a re-arm beside it, and it is
neither reported, activated nor counted as fresh, also in a PROT_NONE
view. After the guard is removed it is reported once by fresh
collection, activated, and a write to it is reported. Without the guard
handling in fresh collection it fails at "fresh collection reported a
guarded page or lost a fresh one".
fiber_tests renames its .eh_frame like the other test executables so
that it can catch exceptions thrown in libc.prx. fiber checks that
nothing is guarded without the variable. Without the mprotect change
fiber_guard fails at "finalize keeps the guest PROT_NONE of the former
guard page"; without the munmap change it fails at "a guard dropped by
a decommit outside the guest arena is released once".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fdfawef324
fdfawef324 force-pushed the feat/fiber-guard-page branch from 6acb312 to 46a9b9d Compare October 13, 2026 05:57
@fdfawef324

Copy link
Copy Markdown
Contributor Author

Rebased onto ea2bbef, after #4661 (per-4 KiB write tracking of shared views) and #4535 (deferred write tracking for fresh Windows shared imports, with b24e1cc's early return). The rebase conflicted in WindowsMappings.hpp only.

  • fix(libc): track writes to shared guest memory per 4 KiB page on Windows #4661: I kept the per-page code and routed protectSubpages through the guard-aware protect. protect protects the pages around a guard and never the guard itself, and it records each run's protection for finalize. Collect neither reports, settles nor arms a guarded 4 KiB page. Guard refuses ranges that are not whole 4 KiB pages. Pages that are not guards are tracked exactly as on main.
  • fix(agc): defer write tracking for fresh Windows shared imports #4535: fresh collection leaves a guarded page out the way Collect does. It neither activates nor reports it, and does not count it as fresh: Guard takes the page out of the fresh ranges, so the early return still applies when nothing else is fresh, and the check of an inaccessible view skips it. Removing or dropping the guard puts a page that was never collected back in the fresh ranges, so it is reported once by the next fresh collection. Armed collection is unchanged; it already skipped the guard through Collect's guard skip. Every page that is not a guard keeps the deferred activation.
  • CheckGuardedSharedContext expects 15 pinned pages, none in the guard, and one page per view for the write after finalize.
  • CheckSharedWritesAroundGuard (guest_memory): it guards the lowest 4 KiB of a 16 KiB shared view, writes the other three pages, collects and re-arms twice. The guard stays PAGE_NOACCESS and exactly the three pages are reported. After the guard is removed, the page is reported once because it was never collected, then armed. It also checks that a guard on an armed page survives a pin. It fails if the subpage runs bypass protect or if Collect reports the guard.
  • New CheckFreshAroundGuard (windows_write_watch): a guarded page in a view collected fresh for the first time is neither activated nor reported, is not counted as fresh in a PROT_NONE view, and is reported exactly once after Unguard. It fails with the guard handling removed from the fresh-collection path.
  • fiber, fiber_guard, windows_write_watch, agc_write_tracking, agc_shared_write* and the libkernel set (guest_memory included) pass. The libc set passes except log_macros, which also fails on main.

@fdfawef324

Copy link
Copy Markdown
Contributor Author

Closing this: #4265 merged with its Windows half (8136438), so a fiber no longer runs host calls on the guest's context and each fiber's host stack has its own guard page. Guarding the guest context, which is what APS5_FIBER_GUARD did, adds nothing now.

@fdfawef324 fdfawef324 closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant