[orezraey]

What

With the default heap, the malloc family followed FreeBSD 11.4's jemalloc for failures and alignments, and realloc(ptr, 0) freed the block; the debt entry said none of it was measured on the console. In a payload process on a PS5 (firmware 13.60), libSceLibcInternal's heap behaves differently, and this PR follows it:

Call	Payload process, firmware 13.60	main before this PR	here now
realloc(block, 0), reallocalign(block, 0, a)	the same block, malloc_usable_size 0; a later realloc of it to 64 bytes returns a new block	frees the block, returns null	the same block with 0 usable bytes
memalign(a, n), reallocalign(p, n, a) with a 0 or not a power of two	a block aligned to a rounded up to a power of two, at least 32, as for malloc (100 → 128, 1000 → 1024, 4097 → 8192, 12288 → 16384)	null, errno EINVAL	the same rounding
aligned_alloc(a, n) with a 0 or not a power of two	null, errno untouched	null, EINVAL	null, errno untouched
malloc, calloc, realloc, memalign, aligned_alloc, reallocalign failing for a size up to the limit below	null, errno EINVAL (22), from 2^33 up	null, ENOMEM	null, EINVAL
a size above SIZE_MAX - 0x1008f, or above SIZE_MAX - (A + 0x100af) for an alignment above 32 rounded up to A; a calloc product that overflows	null, errno untouched	null, ENOMEM	null, errno untouched
memalign, aligned_alloc, reallocalign with an alignment of 2^63 or more	null, errno untouched (2^62 + 1 and 3·2^61 fail with EINVAL)	null, EINVAL for a non-power of two, ENOMEM for 2^63	null, errno untouched
posix_memalign failing	returns ENOMEM and sets errno to EINVAL, except for a request above the limit or an alignment of 2^63, where errno is untouched; an alignment below 8 or not a power of two returns EINVAL without touching errno or the output	returns ENOMEM, errno untouched	as on the console
a failed realloc	the block is left as it was	the same	the same

The limit was found by bisection: malloc(0xfffffffffffeff70) fails with EINVAL, malloc(0xfffffffffffeff71) leaves errno alone, and calloc(1, n), realloc(block, n), aligned_alloc(16, n), reallocalign(p, n, 16) and posix_memalign(&p, 16, n) switch at the same size. For memalign and reallocalign with an alignment above 32 it moves by the rounded alignment: 64 → 0xfffffffffffeff10, 128 → …fed0, 256 → …fe50, 512 → …fd50, 1024 → …fb50, 2048 → …f750, 4096 → …eef50, 8192 → …edf50, 65536 → …dff50, 2^30 → 0xffffffffbffeff50, 2^40 → 0xfffffefffffeff50, 2^62 → 0xbffffffffffeff50, and 3·2^61 → 0x7ffffffffffeff50, the same as 2^63. Alignments up to 32 (0, 3, 32) use the malloc limit.

Heaps a title replaces through its process parameters keep the old behaviour: the console was measured only with its own heap, so ApplicationHeapDefault_nid_no_patch tells the two apart and the replacement path keeps its old behaviour. reallocalign(block, 0, a) for a block that does not already have the rounded alignment was not measured and throws. reallocf now calls this realloc and frees the block only when it fails for a nonzero size, as FreeBSD 11's reallocf.c does, so it reports the same errno and reallocf(block, 0) keeps the block. reallocf itself was not called on the console.

Debt: the heap entry in docs/dev/debt/libc.md now records the measured rules, what is still unmeasured (the title-replaced heap, a title process, reallocalign to 0 bytes with a stricter alignment), and the reallocf entry says it now inherits realloc's result and errno.

How the console was measured

• Setup:
  • The console runs firmware 13.60 and is jailbroken.
  • The probes are ELF payloads built with ps5-payload-dev/sdk v0.44 and clang 18.1.8 and started through elfldr, so they ran in a payload process, not in a title. A title process was not measured.
• Provenance: the resolve lines show that malloc, calloc, realloc, memalign, aligned_alloc, reallocalign, posix_memalign, malloc_usable_size and free were taken from libSceLibcInternal.sprx by dlsym and called through those pointers.
• Method:
  • errno was set to 1234 before each call, so "untouched" means it was still 1234 after the call.
  • The limits were found by bisecting between 2^40 (fails with EINVAL) and SIZE_MAX (leaves errno alone).
  • Alignments were read from the lowest set bit of 48 blocks of different sizes held at once.
  • The library prints its own [SceLibcInternalHeap] A too large request for the heap. report on each failed request; those lines are left out of the excerpt.
• Files: the probe sources and the full logs are in this gist, with each probe's main.c renamed <probe>_main.c.

Console output (excerpt, lines copied unchanged)

heap3: meta fw=0x13600007 pid=116 sdk=v0.44 compiler=Clang 18.1.8 built=Oct 20 2026 18:43:48 log=/data/anyps5-probes/heap3.log
heap3: resolve sym=realloc module=libSceLibcInternal.sprx exported=yes
heap3: resolve sym=memalign module=libSceLibcInternal.sprx exported=yes
heap3: resolve sym=posix_memalign module=libSceLibcInternal.sprx exported=yes

heap3: case realloc ptr=block100 size=0 same=1 usable=0 errno_after_preset_1234=1234
heap3: case realloc ptr=shrunk size=64 result=block same=0 first_byte=0xaf
heap3: case reallocalign ptr=align256_block100 size=0 align=256 same=1 usable=0

heap3: case threshold low=0x8000000000000000 errno=22 high=0xffffffffffffffff errno=1234
heap3: case threshold last_with_errno_22=0xfffffffffffeff70 first_with_errno_1234=0xfffffffffffeff71
heap3: case calloc n=SIZE_MAX size=2 result=null errno_after_preset_1234=1234
heap3: case calloc n=2^20 size=2^20 result=null errno_after_preset_1234=22
heap3: case posix_memalign align=16 size=2^40 ret=12 out=untouched errno_after_preset_1234=22

heap5: case memalign_align0_errno last_value_22=0xfffffffffffeff70 first_value_1234=0xfffffffffffeff71
heap5: case memalign_align33_errno last_value_22=0xfffffffffffeff10 first_value_1234=0xfffffffffffeff11
heap5: case memalign_align100_errno last_value_22=0xfffffffffffefed0 first_value_1234=0xfffffffffffefed1
heap4: case memalign_align4096_errno last_value_22=0xfffffffffffeef50 first_value_1234=0xfffffffffffeef51
heap5: case memalign_align6917529027641081856_errno last_value_22=0x7ffffffffffeff50 first_value_1234=0x7ffffffffffeff51

heap3: case memalign align=2^63 size=16 result=null errno_after_preset_1234=1234
heap3: case memalign align=2^62 size=16 result=null errno_after_preset_1234=22
heap4: case memalign align=2^62+1 size=16 result=null errno_after_preset_1234=22
heap4: case memalign align=3*2^61 size=16 result=null errno_after_preset_1234=22
heap6: case align=0x8000000000000000 reallocalign_null=null errno=1234 reallocalign_block=null errno=1234 posix_memalign_ret=12 out=untouched errno=1234

heap2: case memalign align=3 size=40 min_trailing_zero_bits=6 first_low16=0x9c00 usable=40
heap2: case memalign align=100 size=40 min_trailing_zero_bits=7 first_low16=0x9c00 usable=40
heap2: case memalign align=4097 size=40 min_trailing_zero_bits=13 first_low16=0xa000 usable=40
heap2: case memalign align=12288 size=40 min_trailing_zero_bits=14 first_low16=0xc000 usable=40
heap2: case aligned_alloc align=3 size=96 result=null errno_after_preset_1234=1234
heap2: case posix_memalign align=3 ret=22 out=untouched errno=1234
heap2: case posix_memalign align=24 ret=22 out=untouched errno=1234

The heapN: prefix names the probe; the rest of each line is copied unchanged.

Tested

Build: Windows 11 x86-64, full build with WinLibs MinGW-w64 GCC 15.2.0 (posix-seh, ucrt, r7) and -DBUILD_TESTING=ON. The full suite ran on main 2962d21; after rebasing onto 339217e the heap tests below were rebuilt and run again.

Tests:

• guest_allocator_failures, guest_reallocalign, guest_heap_rounding_overflow and the nine application_heap_* tests pass. They now check:
  • every limit above at its last failing size and the next one;
  • the rounded alignments;
  • realloc and reallocalign to 0 bytes;
  • the posix_memalign return values and errno;
  • that reallocalign(block, 0, 4096) on a block without that alignment throws.
• The full suite: 1782 of 1786 pass, and none of the four failures runs this PR's code.
  • agc_driver_image_array_window_past_layer_limit does not compile with MinGW (alignas(65536), fixed by test(agc): align the array window test's texels at run time #8310).
  • select_tests, a test of tools/select_tests.py, fails here while creating a directory link in TEMP.
  • agc_driver_lookup_log_fits_build failed once under -j8 and passed when run alone.

Old code, new tests: with the four libc sources reverted to main and the new tests kept:

• guest_allocator_failures reports 24 failed checks (errno 12 where the console gives 22 or leaves it alone, and memalign(3, 40) returning null);
• guest_reallocalign and application_heap_default abort.

Not run here: Linux and macOS. The tests are portable except GuestHeapRounding.cpp, which is Windows only.

Console: behaviour measured on a PS5 (firmware 13.60), as described above.

Overlaps:

• tools/pr_overlap.py reports no conflicting lines.
• It could not compare test: organize library tests and label CTest suites #6044, which moves these test files and already conflicts with main.
• fix(libs): share Sony CRT handler and lconv state between libc and libSceAmpr #7081 and Macos port #7201 change other functions in FileAndHeap.cpp, and other open PRs change other entries of docs/dev/debt/libc.md.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Fixes: none
• Unblocks: none
• GPUs tested: n/a
• AI-assisted: yes

[MarcZX10]

I noticed reallocf_nid_postfix still bypasses heapAllocation/defaultHeapAllocation and sets errno = 12 for any caught allocation failure, while the new default-heap path reports EINVAL up to the measured size limit and preserves errno above it. The PR only states reallocf(block, 0) matches realloc, so this may be an intentional evidence boundary; could you clarify that larger failing reallocf requests remain unmeasured, or apply the same measured error classification while retaining reallocf’s free-on-failure behavior if the probe covered it?

[orezraey]

Thanks, good catch. reallocf wasn't called on the console, but the debt entry already said it follows FreeBSD 11's reallocf.c on top of this libc's realloc, and the old code didn't. 55e4819 makes it exactly that: it calls realloc_nid_postfix and frees the block only when that returns null for a nonzero size. So it now gets the measured classification (EINVAL up to the limit, errno untouched above it, ENOMEM with a replaced heap), and reallocf(block, 0) keeps the block. application_heap_default checks both sides of the limit.

[github-actions]

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 3965	docs/dev/debt/libc.md
PR 5021	docs/dev/debt/libc.md
PR 5303	docs/dev/debt/libc.md
PR 6044	core/libs/tests/ApplicationHeap.cpp core/libs/tests/GuestAllocatorFailures.cpp core/libs/tests/GuestHeapRounding.cpp core/libs/tests/GuestReallocalign.cpp
PR 7081	core/libs/prx/libc/src/FileAndHeap.cpp docs/dev/debt/libc.md
PR 7201	core/libs/prx/libc/src/FileAndHeap.cpp core/libs/tests/ApplicationHeap.cpp
PR 7544	docs/dev/debt/libc.md
PR 7648	docs/dev/debt/libc.md
PR 7715	docs/dev/debt/libc.md
PR 8150	docs/dev/debt/libc.md
PR 8306	docs/dev/debt/libc.md
PR 8417	docs/dev/debt/libc.md
PR 8863	docs/dev/debt/libc.md
PR 8965	docs/dev/debt/libc.md
PR 9133	docs/dev/debt/libc.md
PR 9232	docs/dev/debt/libc.md
PR 9408	docs/dev/debt/libc.md
PR 9447	docs/dev/debt/libc.md
PR 9526	core/libs/prx/libc/src/FileAndHeap.cpp docs/dev/debt/libc.md
PR 9527	core/libs/prx/libc/src/FileAndHeap.cpp docs/dev/debt/libc.md
PR 9552	docs/dev/debt/libc.md
PR 9669	docs/dev/debt/libc.md
PR 9833	docs/dev/debt/libc.md
PR 9844	docs/dev/debt/libc.md
PR 9873	docs/dev/debt/libc.md
PR 9929	docs/dev/debt/libc.md
PR 10002	docs/dev/debt/libc.md
PR 10003	docs/dev/debt/libc.md
PR 10007	docs/dev/debt/libc.md
PR 10113	docs/dev/debt/libc.md
PR 10226	docs/dev/debt/libc.md
PR 10229	docs/dev/debt/libc.md
PR 10234	docs/dev/debt/libc.md
PR 10245	docs/dev/debt/libc.md
PR 10247	docs/dev/debt/libc.md
PR 10253	docs/dev/debt/libc.md
PR 10265	docs/dev/debt/libc.md
PR 10274	docs/dev/debt/libc.md
PR 10283	docs/dev/debt/libc.md
PR 10306	docs/dev/debt/libc.md
PR 10314	docs/dev/debt/libc.md
PR 10315	docs/dev/debt/libc.md
PR 10316	docs/dev/debt/libc.md
PR 10317	docs/dev/debt/libc.md
PR 10321	docs/dev/debt/libc.md
PR 10325	docs/dev/debt/libc.md
PR 10328	docs/dev/debt/libc.md
PR 10346	docs/dev/debt/libc.md
PR 10349	docs/dev/debt/libc.md

[MarcZX10]

Thanks for the update and the added two-sided reallocf boundary coverage. The new delegation to realloc_nid_postfix followed by freeing only on a null result for nonzero size matches the stated FreeBSD wrapper behavior while retaining this PR’s measured default-heap errno semantics. I’ve reviewed the current diff; I’ll check the rerun matrix before deciding on approval.

[MarcZX10]

Reviewed the updated allocator wrappers and rechecked the rerun matrix. The default-heap behavior is isolated through ApplicationHeapDefault_nid_no_patch; replacement heaps retain their prior paths. reallocf now delegates to the same realloc error classification and frees the original block only after a nonzero-size failure, with tests for both sides of the measured limit and zero-size retention. The relinker and register-context matrix is green, including Windows; the full Linux/Windows build jobs were skipped by workflow rules. I am relying on the author’s Windows full-suite and PS5 payload measurements for the allocator behavior.

[xXPBSXx]

Linux data point for 55e4819728: WSL2 Ubuntu 24.04 (a virtualised kernel, not bare-metal Linux), GCC 13.3.0, Release, BUILD_TESTING=ON, PR head alone (not merged onto main).

• With the PR: ctest -R '^(guest_reallocalign|guest_allocator_failures)$' passes 3 of 3 (both tests).
• With ApplicationHeap.hpp, ApplicationHeap.cpp, FileAndHeap.cpp and GuestHeap.cpp (libc) restored to the merge-base (new test cases kept): guest_reallocalign fails (Subprocess aborted) and guest_allocator_failures fails.
  Not tested: macOS, Windows, a title run.
  AI-assisted (Claude Code).

[orezraey]

Measured reallocf on a PS5 (firmware 13.60) through libSceLibcInternal.sprx; the source and raw log are in this gist (preallocf). It matches what this branch does on top of realloc: a null block with 0 or 40 bytes allocates, size 0 keeps the block with 0 usable bytes, 2^40 returns null with errno 22 and frees the block (the next malloc(64) reuses it), and SIZE_MAX returns null with errno untouched. The debt entry now says so (latest commit).

[oneandonlydean]

@orezraey Please remove GuestReallocalign.cpp’s final loose-block loop asserting that reallocalign(loose, 0, 4096) throws for the explicitly unmeasured case. The tests should not pin unimplemented behaviour. Keep measured aligned zero-size retention, alignment rounding, errno boundaries and replacement-heap controls, then rerun the focused Linux and Windows tests on this exact head; the earlier Linux report covers 55e4819 rather than this revision.

[orezraey]

Removed the final loose-block loop from GuestReallocalign.cpp in 39d4e73, so the test no longer asserts that the unmeasured reallocalign(loose, 0, 4096) throws; the production refusal and its debt entry stay. The measured aligned zero-size retention, alignment rounding, errno boundaries and replacement-heap checks are unchanged. At 39d4e73, guest_reallocalign, guest_allocator_failures and the application_heap_* tests pass three runs in a row on Windows 11 (MinGW GCC 15.2, 12 tests) and on Linux (WSL2, kernel 6.18, GCC 16.2.1, Release, 14 tests including the Linux-only SDK heap fixtures).