Skip to content

perf(libkernel): prefault mapped Windows direct memory in the background - #4290

Closed
moong15 wants to merge 2 commits into
boykopovar:mainfrom
moong15:perf/prefault-direct-memory
Closed

moong15 wants to merge 2 commits into
boykopovar:mainfrom
moong15:perf/prefault-direct-memory

Conversation

@moong15

@moong15 moong15 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What

Opt-in background prefaulting of mapped Windows direct memory. With APS5_PREFAULT_MEMORY set, a worker processes mapped ranges of at least 2 MiB using VirtualLock followed by VirtualUnlock in 2 MiB chunks. It requests soft working-set headroom of at least 64 MiB minimum and 128 MiB maximum. The worker does not write the pages. Linux is unchanged.

This aims to move first-touch page residency work away from guest loading threads. It is separate from the write-protection exception tracking fixed by #4535, which is now merged. The option can increase memory residency and remains disabled by default. A successful lock is followed immediately by an unlock, so residency is not guaranteed afterward.

Reopened after #4535 merged and rebased without conflicts onto main ea2bbef. Current head: b2e1166. The diff remains one file, 56 added lines. The earlier adjacent #4281 change is already in main. An open-PR search for prefault/VirtualLock found no competing implementation; automated overlap review remains pending.

Tested

Reopening validation: git diff --check passed. No build, runtime test or game comparison has been rerun on the rebased b2e1166 head. This remains a draft while that validation and the current repository's performance-report evidence are pending.

Historical evidence, not validation of the rebased head: Windows 11 Pro 26200, Ryzen 7 9800X3D, 47 GB RAM, Radeon RX 9070 XT (driver 32.0.32015.2008), WinLibs GCC 15.2, Release. guest_memory previously passed with the option unset and set.

Earlier Hades II PPSA36082 v01.006 warm-cache runs used a local d146ec1 build with this branch's DirectMemory.cpp, unpublished diagnostics, two cache options, APS5_HOST_IMPORT_MIB=16384 and APS5_NO_WRITE_WATCH=1 in both arms. Without prefault, 4/4 runs crashed with a LoadTaskScheduler null write around 7.4-7.5 seconds; with prefault, 5/5 survived their observation windows (three 69-77 seconds and two 40 seconds) and the longer runs opened menu audio banks. Those results do not establish performance or correctness with normal write tracking on current main, nor prove the crash's cause.

Subsequent combined-fix investigation and its limitations are recorded in #4535. Fresh matched main-versus-prefault measurements with write tracking enabled are needed here. No current-head speed, cold-cache, rendered-menu or gameplay claim is made. Before making a speed claim, collect the same-machine/same-scene summary lines and attach traces as required by the Performance report issue form. Full upstream CI requires a maintainer's manual workflow run.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none; fix(agc): defer write tracking for fresh Windows shared imports #4535 is merged and included in the base
  • Unblocks: none; investigating PPSA36082 Hades II (PPSA36082) #1965, current-head confirmation pending
  • AI-assisted: yes (Claude Code, OpenAI Codex)

@moong15
moong15 marked this pull request as draft October 12, 2026 05:51
@moong15
moong15 force-pushed the perf/prefault-direct-memory branch from 60f0e98 to 1ad61fe Compare October 12, 2026 13:53
@moong15 moong15 closed this Oct 12, 2026
Windows materializes the pages of a paging-file section view on first
touch, so a guest that streams into freshly mapped direct memory pays one
soft page fault per page on its own threads. With APS5_PREFAULT_MEMORY set,
a background thread makes each mapped direct-memory range of 2 MiB or more
resident with VirtualLock and VirtualUnlock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@moong15 moong15 reopened this Oct 13, 2026
@moong15
moong15 force-pushed the perf/prefault-direct-memory branch from 1ad61fe to b2e1166 Compare October 13, 2026 06:07
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
#4508 core/libs/prx/libkernel/CMakeLists.txt L1117
#4725 core/libs/prx/libkernel/CMakeLists.txt L1117

@xXPBSXx

xXPBSXx commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Tested on Windows 10 (19045), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release: this PR merged onto main c080fc2d5 (no conflict).

  • windows_prefault_enabled (0.64 s) and windows_prefault_disabled (0.38 s) pass with the PR.
  • With the PR's change to DirectMemory.cpp taken back to main, windows_prefault_enabled fails (Prefault check failed at WindowsPrefault.cpp:39, 6.4 s); windows_prefault_disabled still passes. So the new check fails without the change and the test finishes in well under 30 s either way.

Not tested: Linux (Windows-only code), a console, a title run.

AI-assisted (Claude Code).

@moong15 moong15 closed this Oct 13, 2026
@moong15
moong15 deleted the perf/prefault-direct-memory branch October 18, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants