Skip to content

feat(libc): answer malloc_stats_fast for the default heap - #6217

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
milescoviello:fix/libc-malloc-stats-default-heap
Oct 18, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
milescoviello:fix/libc-malloc-stats-default-heap

Conversation

@milescoviello

@milescoviello milescoviello commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

What

Ship of Harkinian (PPSA99320, catalog release) stops about 10 s after start-up on main:

[libc] unhandled exception 'St13runtime_error' thrown from ...
  what():  malloc_stats_fast: implemented only for an application allocator that provides it

The title uses the default allocator. Every 10 s it logs its heap use with malloc_stats_fast: it fills in size = 0x28 and version = 1, then prints currentInuseSize and currentSystemSize. Its struct is the same 0x28-byte layout that sceLibcMspaceMallocStatsFast already fills. So far only an application allocator's table entry 8 could answer the call.

Now the default heap answers it too:

  • GuestHeap counts the bytes it holds from the arena (the 1 MiB spans of its small-block classes and its committed large blocks, the cached free ones included) and the bytes of the blocks it has handed out (each at its size-class or 64 KiB-granule size, header included), with the peak of each. Without the arena, both count the host blocks.
  • malloc_stats_fast with the default allocator (table entry 8 of defaultApi) fills the block from those counts. A block smaller than 0x28 bytes, or a null pointer, returns 22, as the mspace version does.
  • MallocManagedSize moves from Mspace.cpp into a shared header, so both use one definition.
  • An application allocator without entry 8 still throws, as before.

The counters change on every heap malloc and free. Those calls already run under the guest allocation registry's lock (GuestAllocations::Mutation), so the four relaxed atomics add no new point of contention. I measured it with a temporary benchmark (not part of this PR) of ApplicationHeapAllocate/Free of 32 to 543-byte blocks on the guest arena, 3 runs each. One thread: 112 ns per call on main, 115 ns with this change. Eight threads: 900 ns on main, 875 ns with this change, which is within noise.

libc.md records how these counts differ from Sony's allocator.

Tested

Ubuntu 24.04, GCC 13.3, Release, on main a077a88:

  • application_heap_stats_default now checks the filled block with the default allocator. Its four sizes are ordered (max ≥ current, system ≥ in use). The in-use size grows by at least the 100,016 bytes of two blocks, returns to its earlier value when they are freed, and the peak keeps the high mark. Nothing past 0x28 bytes is written, and a short block or a null pointer gives 22. With main's ApplicationHeap.cpp and GuestHeap.cpp it fails with the title's message. application_heap_stats and application_heap_stats_missing still pass.
  • All guest_*, application_heap* and mspace tests pass (385).
  • PPSA99320, relinked with --to-intel and run on Linux (RTX 4070 Ti) for 75 s without input. On main it stops at 11 s, on the first call. With this change it runs the whole 75 s at 60 FPS and logs perf: mem heap 0 MB, sys 3 MB, ... every 10 s. Screenshots at 20, 45 and 70 s show its "Game files needed" dialog, since the catalog build ships no ROM. When the 75 s timeout ends the run, its exit stops on Mutex has been destroyed. A run with pad input stops the same way 8 s in, when the input starts. That's a separate issue, which this change doesn't touch.
  • tools/check_conventions.py --base upstream/main passes, and tools/pr_overlap.py --branch fix/libc-malloc-stats-default-heap finds no overlap.

@nicolasfalesy found a catalog title that needs this: Ship of Harkinian, Vulkan (PPSA99320), on Linux with an Intel Arc GPU (Mesa ANV). On main 5d9dcc4 it stops with malloc_stats_fast: implemented only for an application allocator that provides it. With this PR it renders for the whole 40 s test (avg 59.5 fps at 3840x2160).

Not checked on Windows or on a console.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Progress report (3e83446 → dde480e)

➖ System libraries: 89.65% (+0.0%, +1 implemented functions)

✅ 1 implemented
Library Name
libc defaultStatsFast

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 docs/dev/debt/libc.md
PR 5303 docs/dev/debt/libc.md
PR 6044 core/libs/tests/ApplicationHeapStats.cpp
PR 6226 docs/dev/debt/libc.md
PR 6315 docs/dev/debt/libc.md
PR 6334 docs/dev/debt/libc.md
PR 6541 docs/dev/debt/libc.md
PR 6565 docs/dev/debt/libc.md
PR 7447 docs/dev/debt/libc.md

@nicolasfalesy

Copy link
Copy Markdown
Contributor

A title that reaches this: Ship of Harkinian, Vulkan (PPSA99320) from the homebrew catalog, on Linux (Intel Arc, Mesa ANV).

  • main 5d9dcc4: stops with malloc_stats_fast: implemented only for an application allocator that provides it.
  • main 5d9dcc4 + this PR: that line is gone. It renders for the whole 40 s test ([info] perf: 3840x2160 avg 59.5 fps, 1% low 30.0 fps, max 33.5 ms, >20ms 7, >40ms 0, 596 frames, mem heap 0 MB, sys 3 MB). It was stopped by the test after 40 s; the Mutex has been destroyed line in the log comes after that stop.

AI-assisted: yes

@milescoviello

Copy link
Copy Markdown
Contributor Author

Thank you, @nicolasfalesy. A catalog title with a before-and-after run on Intel Arc is exactly the evidence this needed. I've added it to the description.

AI-assisted: yes

@milescoviello
milescoviello force-pushed the fix/libc-malloc-stats-default-heap branch from f33e70e to 5691669 Compare October 17, 2026 18:34
Ship of Harkinian (PPSA99320) logs its heap use with malloc_stats_fast
every 10 s and stopped on the first call, since only an application
allocator could answer it. The default heap now keeps its system and
in-use byte counts with their peaks and fills the same managed-size
block as sceLibcMspaceMallocStatsFast, whose layout moves to a shared
header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@milescoviello
milescoviello force-pushed the fix/libc-malloc-stats-default-heap branch from 5691669 to dde480e Compare October 17, 2026 20:54
@adriadam10 adriadam10 self-assigned this Oct 18, 2026
@adriadam10
adriadam10 merged commit 48d61f5 into boykopovar:main Oct 18, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants