Skip to content

feat(libc): implement freezero - #11848

Open
heditrigui wants to merge 6 commits into
boykopovar:mainfrom
heditrigui:feat/libc-freezero
Open

heditrigui wants to merge 6 commits into
boykopovar:mainfrom
heditrigui:feat/libc-freezero

Conversation

@heditrigui

Copy link
Copy Markdown
Contributor

What

Implement freezero in libc matching FreeBSD/OpenBSD behavior to clear buffer contents before freeing, and add test coverage in ApplicationHeap.cpp.

Tested

Ran python tools/check_conventions.py --base upstream/main cleanly (0 errors).
Ran python core/libs/tests/test_public_headers.py --audit-only cleanly.
Ran python tools/progress.py verifying progress increase (libraries 6567/13449 -> 6568/13450).

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Progress report (5b5786f → bbc686b)

📈 System libraries: 29.15% (+0.0019%, +1 implemented functions)

✅ 1 implemented
Library Name
libc freezero_nid_postfix

Changed libraries (implemented exports)

Library Before After Change
libc 1639/1659 (98.79%) 1640/1660 (98.80%) +0.0007%

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@heditrigui thanks for freezero. One request: the test doesn't yet show that the bytes are actually zeroed before the block is freed, so an implementation that just calls free would pass it too. Could you add a regression that observes the memory at deallocation time (for example through the existing replacement-allocator free callback) and would fail for a plain free? Please cover a partial size (zeroed prefix, untouched rest), size 0 and a null pointer, while still checking that the whole object is freed.

@heditrigui

Copy link
Copy Markdown
Contributor Author

@oneandonlydean Thanks! Added deallocation-time observation via the replacement allocator's release callback: captures storage into lastFreedBytes, asserting that a partial freezero(..., 40) clears bytes 0..39 while leaving the rest untouched, size 0 leaves all bytes intact, and null pointer calls leave the free count unchanged, with the whole object freed in each case.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 11870 docs/dev/debt/libc.md L67-68
PR 12184 core/libs/tests/ApplicationHeap.cpp L568-569

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 docs/dev/debt/libc.md
PR 6819 docs/dev/debt/libc.md
PR 7081 core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 7386 docs/dev/debt/libc.md
PR 7447 core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 docs/dev/debt/libc.md
PR 8965 docs/dev/debt/libc.md
PR 9340 docs/dev/debt/libc.md
PR 9371 docs/dev/debt/libc.md
PR 9447 docs/dev/debt/libc.md
PR 9466 docs/dev/debt/libc.md
PR 9526 core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 10780 core/libs/prx/libc/include/Api.hpp
docs/dev/debt/libc.md
PR 11172 docs/dev/debt/libc.md
PR 11332 docs/dev/debt/libc.md
PR 11724 docs/dev/debt/libc.md
PR 11726 docs/dev/debt/libc.md
PR 11831 core/libs/prx/libc/include/Api.hpp
core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 12518 docs/dev/debt/libc.md
PR 12682 core/libs/prx/libc/include/Api.hpp
core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 12688 core/libs/prx/libc/include/Api.hpp
core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 12700 core/libs/prx/libc/include/Api.hpp
core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 12704 docs/dev/debt/libc.md
PR 12849 docs/dev/debt/libc.md
PR 12965 core/libs/prx/libc/include/Api.hpp
core/libs/prx/libc/src/FileAndHeap.cpp
docs/dev/debt/libc.md
PR 13449 docs/dev/debt/libc.md
PR 13486 core/libs/prx/libc/include/Api.hpp
core/libs/prx/libc/src/FileAndHeap.cpp
core/libs/tests/ApplicationHeap.cpp
docs/dev/debt/libc.md
PR 13794 docs/dev/debt/libc.md
PR 13804 docs/dev/debt/libc.md
PR 13809 docs/dev/debt/libc.md
PR 14212 docs/dev/debt/libc.md
PR 15697 core/libs/prx/libc/include/Api.hpp
PR 15773 docs/dev/debt/libc.md

@maskym

maskym commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Windows result for head 8c25c53a63, merged alone into main 75f3702aed (the PR's Tested section names no Windows run; it conflicts textually with #11846 in ApplicationHeap.cpp, so it was tested in its own tree).

Host: Windows 11 26300, Intel i7-12700K, WinLibs GCC 15.2.0 (x86_64-ucrt-posix-seh, r6), Release, BUILD_TESTING=ON, Ninja. Build clean.

application_heap_replacement fails at ApplicationHeap.cpp:436, the pre-existing require(reallocf(...) == nullptr && lastSize == 48 && frees == 8 && errno == 12). With the conjuncts printed locally: reallocf is null, lastSize 48, errno 12, but frees is 10. The two new freezero_nid_postfix(storage.data(), ...) calls at lines 419 and 426 each go through the counting free, so the hard-coded 8 at line 436 (and the frees == 8 at line 438) no longer match. The other eight application_heap_* tests registered on Windows pass; application_heap_guest_sdk* are Linux-only. Not run on a PS5.

AI-assisted: yes (Claude Code), the runs and the numbers above are from this machine.

@heditrigui

Copy link
Copy Markdown
Contributor Author

@maskym Good catch on the downstream hardcoded frees == 8 count in the reallocf block! Moved the freezero deallocation assertions after the reallocf and scripting test blocks so earlier counter expectations are not perturbed.

@maskym maskym mentioned this pull request Oct 27, 2026
12 tasks done
@maskym

maskym commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Retested head 581e0c0454 on Windows, merged into main 2ee8e9751a together with #11843's 2dc798856c (both merged cleanly).

Host: Windows 11 26300, Intel i7-12700K, WinLibs GCC 15.2.0 (x86_64-ucrt-posix-seh, r6), Release, BUILD_TESTING=ON, Ninja. Build clean.

ctest --test-dir build -R "^application_heap": 9/9 passed, application_heap_replacement in 0.04 s (it aborted at the frees == 8 check on the previous head). Not run on a PS5.

AI-assisted: yes (Claude Code), the runs and the numbers above are from this machine.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 28, 2026
@oneandonlydean oneandonlydean self-assigned this Oct 28, 2026
@TechLevelZero

TechLevelZero commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

✅ AMD GPU test: passed

PR head 3718acb merged onto main b611e63 · tested 2026-11-05 03:05 UTC · 28 min

GPU AMD Radeon RX 6300 (RADV NAVI24), driver Mesa 25.2.8-0ubuntu0.24.04.4, Vulkan 1.4.318
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-42-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 3,822 passed · 14 failed (0 new, 13 also fail on main, 1 unstable on main) · 61 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 3,897 of 3,946 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (1390 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.4 s, ⚠️ public_headers 86.5 s, ⚠️ agc_driver_graphics_maps_fallback 72.7 s, ⚠️ agc_driver_graphics 72.1 s, ⚠️ agc_driver_image_sample_unnormalized 52.1 s
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_pipeline_journal: failed in 9 of the last 10 main runs
Failing on main too (13, not counted against this PR)
agc_driver_alpha_to_coverage_sample_mask_export
agc_driver_guest_memory_lifetime_direct
agc_driver_image_gather_volume
agc_driver_image_table
agc_driver_job_manager_compute_capacity
agc_driver_native_depth_bda_clear
agc_driver_native_depth_buffer_clear
agc_driver_native_depth_clears
agc_driver_native_multisample_depth
agc_driver_submit_progress
agc_submit_progress
guest_mmap_executable
guest_socket_options_multicast
Validation IDs already present on main (4 IDs in 807 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (807 tests): agc_compute_pipeline_replay, agc_driver, agc_driver_address_draw_space_reuse, agc_driver_address_draw_space_reuse_held, agc_driver_address_draw_space_reuse_off, agc_driver_address_draw_space_reuse_transitions, agc_driver_address_draw_space_reuse_verify, agc_driver_address_passes, and 799 more
VUID-vkCmdCopyImage-srcImage-01548 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store_image_copy, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_plane_partially_unmapped_plane, agc_driver_depth_plane_unmapped_plane, and 3 more
VUID-vkCmdCopyImage-srcImage-01551 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store_image_copy, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_plane_partially_unmapped_plane, agc_driver_depth_plane_unmapped_plane, and 3 more
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (2 tests): agc_driver_prepared_exports, agc_driver_srgb8_color_target
Skipped (61)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_copied_writer_label_order: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_copied_writer_label_order_drain: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_float16_unused_nan_payloads: skipped, the device has no native f16 conversions with RTE rounding and kept f16 denormals
agc_driver_gcn_display_present: skipped, no display: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
agc_driver_image_table_vouched_lookups: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a watched imported table cannot be vouched for
agc_driver_linear_target_direct_write_completion: skipped, the device does not keep host imports write-watched, so no completion stamp over the target is tracked
agc_driver_ngg_vertex_wave_reduction: skipped, s3 describes a 32-lane wave only on 32-lane subgroups, and the device has 64 lanes
agc_driver_scattered_runs: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_sparse_dma_buf_import: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_driver_unit_shadow_budget: skipped, the device does not import watched host memory for unit shadows
agc_driver_vertex_elected_atomic_max: skipped, the device runs vertex programs in subgroups, so they keep their cross-lane reductions
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_idle_publish_wait: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a forced watch of the import reads its submissions as CPU stores
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_shadowed_key_stores: skipped, the device imports no watched host memory under a unit shadow
agc_small_tiled_shadow_label: skipped, host imports are unavailable: the test block is not imported and watched
agc_storage_clear_hand_over: skipped, the device imports no watched host memory for the clear-key hand-over
agc_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in 
Reproduce
git fetch origin 3718acbf32431a896716e8cccb7c24a7494b8d02 b611e634f7fc024f0c13e342d96de744448ef1d9
git checkout b611e634f7fc024f0c13e342d96de744448ef1d9 && git merge --no-ff 3718acbf32431a896716e8cccb7c24a7494b8d02
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 12 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 16 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants