Skip to content

fix(agc): read the process's private memory from task_info on macOS - #9971

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
tkcm01:fix/agc-overlay-private-bytes-macos
Oct 28, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
tkcm01:fix/agc-overlay-private-bytes-macos

Conversation

@tkcm01

@tkcm01 tkcm01 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

What

ReadProcessPrivateBytes (Execution/src/PerformanceOverlay.cpp) reads RssAnon and VmSwap from /proc/self/status on every host other than Windows. macOS has no /proc, so the overlay's RAM line stays empty and agc_performance_overlay_statistics fails with the process's private memory cannot be read. On macOS it now returns the task's phys_footprint from task_info(TASK_VM_INFO): the process's resident and compressed private memory, the figure Activity Monitor shows. Linux and Windows are unchanged.

@jotitadev measured that this makes the test pass on macOS (#7886's description) and left it out of #7886 because #7332 had it; #7332 is closed. The same change is part of #8658 (@iamthehimansh). Both are credited as co-authors; this splits it out as a small change, as for the other macOS pieces on #1. #8658 has since been closed in favour of topic PRs based on main, so no other open PR carries this change.

Tested

macOS 27.0.1, Apple M4 (10-core GPU), Apple clang 21, x86-64 build under Rosetta 2, MoltenVK 1.4.2 from the Vulkan SDK 1.4.363.0 (its universal loader, through VK_DRIVER_FILES). The libraries do not link on macOS from main alone, so the run is on mugurc's ci/macos-series-2 (53db0ba, main 3fd1cc3) with #7524, the code commits of #7937 and the libSceSsl change from #7332, as described on #1, plus my local macOS fixes listed there and #7886, so that agc_performance_overlay loads libvulkan.1.dylib and runs. ReadProcessPrivateBytes is the same in that tree as on main.

  • ctest -R '^agc_performance_overlay(_statistics)?$' -V with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation:
    • without this change: both fail with the process's private memory cannot be read;
    • with it: both pass. agc_performance_overlay shows RAM: 0.1 GB; one read takes 1.3 us.
    • Neither log reports was not found but was requested by env var VK_INSTANCE_LAYERS. Both logs have the same two validation messages, VUID-VkDeviceCreateInfo-pProperties-04451 from test(agc): load the Vulkan loader with portability enumeration on macOS #7886's test instance (it doesn't enable VK_KHR_portability_subset); this change adds none.
  • Full ctest -j4 on the same tree with Vulkan, with and without this change: 1648 tests each. Without it 60 failed, with it 59 in two runs; the two overlay tests are the only consistent difference. The others that differ are flaky under load on both builds and never reach this code (their recorders don't present): agc_driver_vertex_target_range_imported also fails without the change (6 of 100 serial runs), agc_driver_sgpr_pattern_fill_keys failed in a full run without it, and precise_wait is a known flake.
  • python3 tools/check_conventions.py --base upstream/main: passed. tools/pr_overlap.py lists only feat: macOS host support (Apple Silicon via Rosetta 2, Vulkan on MoltenVK) #8658 for this file, with this same change.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: Apple M4 (10-core GPU), MoltenVK 1.4.2 (Vulkan SDK 1.4.363.0), macOS 27.0.1
  • AI-assisted: yes

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 7642 core/libs/prx/libSceAgcDriver/Execution/src/PerformanceOverlay.cpp
PR 9508 core/libs/prx/libSceAgcDriver/Execution/src/PerformanceOverlay.cpp
PR 10193 core/libs/prx/libSceAgcDriver/Execution/src/PerformanceOverlay.cpp
PR 11920 core/libs/prx/libSceAgcDriver/Execution/src/PerformanceOverlay.cpp

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second Mac, different chip and OS build from the M4 run in the description: red→green reproduces here, and phys_footprint matches what footprint(1) reports for a translated process.

Machine: Apple M2 Max (30-core GPU), 32 GB, macOS 27.2, Apple clang 21, x86-64 build under Rosetta 2, Release, BUILD_TESTING=ON.

Base/method: main doesn't link on macOS, so the tree is #8658 (feat/macos-port) @ c5fa39620a, full cmake --build -j4. #8658's ReadProcessPrivateBytes hunk is the same as this PR's at 8f2db802d3 except for the one-line code comment, so the green build is #8658 as is. The red build is the same tree with only the #elif defined(__APPLE__) branch of ReadProcessPrivateBytes removed, so macOS falls through to the /proc/self/status path as on main.

  • Red: agc_performance_overlay_statistics fails 3/3 with the process's private memory cannot be read.
  • Green: passes 3/3. costReport gives 1.3, 1.7 and 1.4 us per read, in line with the 1.3 us on the M4.

agc_performance_overlay_statistics doesn't create a Vulkan device, so this result doesn't depend on the GPU. I didn't run the Vulkan agc_performance_overlay test in this window.

Metric check. I wanted to confirm that phys_footprint is the right figure under Rosetta, so I wrote a small C probe (not part of the tree) with the same task_info(mach_task_self(), TASK_VM_INFO, ...) call. It reads the value before and after a 256 MiB malloc + memset, then I ran footprint <pid> against it while it slept:

build returned count delta after touching 256 MiB final phys_footprint footprint
-arch x86_64 (Rosetta) 93 (TASK_VM_INFO_COUNT) 256.5 MiB 271,455,696 B 64-bit (translated) Footprint: 259 MB
arm64 93 256.1 MiB 269,763,112 B Footprint: 257 MB

So the value is the one footprint reports, also for a translated process. The kernel fills the whole struct, which includes phys_footprint (it arrived in rev1, TASK_VM_INFO_REV1_COUNT = 38). The info{} zero-init plus the == 0 check would also catch an older kernel that returned a shorter struct, so the error handling looks complete to me.

Diff: <mach/mach.h> is under #elif defined(__APPLE__) next to the Windows includes, and the /proc path is still the #else, so Linux and Windows don't change.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ReadProcessPrivateBytes implementation in this PR is also present in the open macOS-port PR #8658 (same TASK_VM_INFO/phys_footprint hunk; #8658 currently remains open). Your description explains this split and preserves credit, but the two PRs do not record which copy should land first or how the other branch will drop its duplicate. CONTRIBUTING asks authors to agree on shared work and declare dependencies where needed. Please coordinate with #8658’s author/maintainers and record the integration order: either make #9971 the standalone change and remove the duplicate from #8658 when it rebases, or keep this hunk in #8658 and make the split PR dependent on that work. I compared the exact hunk: Linux/Windows paths remain unchanged, current-main sync is clean, and local check_main_sync, conventions and git diff --check pass (the branch is 136 commits behind, but this file has no intervening main edits). The reported two-platform macOS red/green evidence is relevant; this host cannot independently run macOS tests. This request is for avoiding duplicate implementation and clarifying integration, not queue size.

@maximilianfv

Copy link
Copy Markdown
Contributor

Independent test: Apple M1 Pro, macOS 27.0.1

Apple M1 Pro, macOS 27.0.1, Apple clang 21, x86-64 Release build under Rosetta 2, BUILD_TESTING=ON, no Vulkan loader. Base: mugurc's ci/macos-series-2 at 27e968974, with this pull request's commit cherry-picked (together with #9972 and #8920, which touch other files).

  • agc_performance_overlay_statistics: fails on the series alone (the process's private memory cannot be read), and passes with this pull request in 2 of 2 runs.
  • Full ctest -j 6 (mic tests excluded): 15 failures with this change against 20 without, and no test that passed before fails now. The other differences are precise_wait and guest_refused_connect, which are flaky here either way.

AI-assisted: yes (Claude Code ran the build and the tests).

@tkcm01

tkcm01 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 thanks. Recorded in the description: this lands on its own first, and #8658 drops the same hunk when it rebases. I've asked @iamthehimansh on #8658; if they'd rather keep it there, I'll close this one.

AI-assisted: yes

@tkcm01

tkcm01 commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 an update on the order: @iamthehimansh closed #8658 and is landing it as topic PRs based on main (#10419 first). No open PR carries this ReadProcessPrivateBytes hunk now, so there's no duplicate to coordinate. I've updated the description. Could you take another look?

AI-assisted: yes

@imellag imellag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested head 8f2db802d3 on Apple M4 Pro, macOS 27.0.1, Apple clang 21, x86-64 under Rosetta 2, with MoltenVK 1.4.2 from the Vulkan SDK 1.4.363.0.

libSceAgcDriver doesn't link on main on macOS yet, so I tested on main 3fcb5e4d76 plus the open macOS build PRs and the AGC parts of mugurc's series. On that tree, with ANYPS5_REQUIRE_VULKAN=1:

  • Without this PR, agc_performance_overlay_statistics fails with the process's private memory cannot be read.
  • With it, the test passes 3 runs out of 3.
  • agc_performance_overlay and agc_performance_overlay_device_selection fail with and without it: they load libvulkan.so.1 themselves (AGC graphics: cannot load Vulkan), which is unrelated to this change.

AI-assisted: yes

@maximilianfv

Copy link
Copy Markdown
Contributor

Tested on Apple M1 Pro, macOS 27.0.1 (x86_64 build under Rosetta 2)

Follow-up to my 2026-10-23 test, which ran on the series alone without a Vulkan loader: this run uses the same head on the series merged with main, with Vulkan, so agc_performance_overlay itself runs.

Applied at 8f2db802d to mugurc's macOS series (ci/macos-series-2, 27e9689) merged with main at 2742004, with local macOS build fixes, then toggled only this PR. Vulkan SDK 1.4.363.0 (MoltenVK) with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. The diff still applies cleanly to current main (a5a780e), whose ReadProcessPrivateBytes still has only the Windows and /proc/self/status paths, so it is still needed.

agc_performance_overlay (3 runs each) agc_performance_overlay_statistics (1 run each)
Without this PR Failed 3/3: the SDR, HDR10 and corner checks pass, then AGC graphics: the process's private memory cannot be read Failed: the process's private memory cannot be read
With this PR Passed 3/3: full panel drawn bit for bit with RAM: 0.1 GB Passed: private memory read in 1.1 us
  • Validation Error lines: 1 in every run, both ways, the same VUID-VkDeviceCreateInfo-pProperties-04451 from the test's own device (created without VK_KHR_portability_subset). This change adds none.
  • No log reports was not found but was requested by env var VK_INSTANCE_LAYERS.
  • Code read: no issues. task_info(TASK_VM_INFO) with TASK_VM_INFO_COUNT and the phys_footprint == 0 fallback to std::nullopt are correct; Linux and Windows are unchanged.

AI-assisted: yes (Claude Code applied the PR, ran the tests, read the code and wrote this report).

@babyhuey

Copy link
Copy Markdown
Contributor

Windows run of this head 8f2db802d34, merged onto main 4e43321bd6c (clean merge, local only), with the selection CI's skipped Windows product job uses: Windows 11, Ryzen 7 5800X, RTX 3080, WinLibs GCC 15.2, Release, BUILD_TESTING=ON. The tree was configured fresh after the merge, then select_tests.py select / ninja -f select.ninja selected / --target libs / select_tests.py ctest --parallel 4 --timeout 120, with check_skips.py --job windows (all skips expected).

AI-assisted: yes (Claude Code)

@adriadam10 adriadam10 self-assigned this Oct 26, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 26, 2026
ReadProcessPrivateBytes reads RssAnon and VmSwap from /proc/self/status
on every host other than Windows. macOS has no /proc, so the overlay's
RAM line stays empty and agc_performance_overlay_statistics fails with
"the process's private memory cannot be read". On macOS it now returns
the task's phys_footprint from task_info(TASK_VM_INFO): its resident and
compressed private memory, which is what Activity Monitor shows.

jotitadev measured that this makes the test pass on macOS (boykopovar#7886).
The same change is part of the macOS port in boykopovar#8658.

Co-Authored-By: iamthehimansh <67262056+iamthehimansh@users.noreply.github.com>
Co-Authored-By: jotitadev <139649225+jotitadev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tkcm01
tkcm01 force-pushed the fix/agc-overlay-private-bytes-macos branch from 8f2db80 to 78afe4c Compare October 27, 2026 12:03
@tkcm01

tkcm01 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main ca0e9557a (now 78afe4c2b8). main added #include <sys/resource.h> for every non-Windows build in the same include block, so <mach/mach.h> now sits inside that branch under #ifdef __APPLE__. ReadProcessPrivateBytes itself is unchanged from the head tested above.

AI-assisted: yes

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 27, 2026
@adriadam10
adriadam10 dismissed MarcZX10’s stale review October 27, 2026 12:11

Dismissing a stale review: the requested changes were addressed after it (new head) and the merge queue blocks on it. Please request changes again if anything is still open.

@adriadam10
adriadam10 merged commit 9fc181d into boykopovar:main Oct 28, 2026
10 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Re-checked on the rebased head: ReadProcessPrivateBytes is unchanged and the only new line is the <mach/mach.h> include, inside __APPLE__, so Linux and Windows are untouched. The point in @MarcZX10's request (coordinate with #8658) is met: #8658 is closed, #10419 doesn't touch this file and no open PR carries this hunk. Confirmed on macOS by @jotitadev, @maximilianfv and @imellag; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants