Skip to content

test(libc): copy guarded UTF-16 buffers by unit counts - #13311

Open
StoneDC wants to merge 4 commits into
boykopovar:mainfrom
StoneDC:b42/wmemcpy-extent
Open

StoneDC wants to merge 4 commits into
boykopovar:mainfrom
StoneDC:b42/wmemcpy-extent

Conversation

@StoneDC

@StoneDC StoneDC commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What

Adds positive-length wmemcpy tests for 1 through 192 char16_t units, including embedded zeros and units with different high bytes. It checks full-unit copying, exact extents, the returned pointer and source/prefix preservation at protected boundaries.

#13220 merged at 2026-11-01T03:58:51Z. Its original commit e9f09f54fabb6408d3e2711b8df10c10112ab00e remains an ancestor; this independent topic now has no open prerequisite. The earlier shared write-watch configuration fix is already on main and is not a separate feature of this topic.

Refreshed the source branch after GitHub still reported a conflict following the dependency cleanup. The topic fixture is unchanged.

Tested

Exact updated head 24e53d3149bcfcf2b8b5e0abdc18421184170d76 integrates current recorded main ba627802e2210187d417deb64c8f2fed8e33011f. Complete combined validation 52f04e5a6b07cea04982202ab1d9237163391686 now contains all 31 exact topic heads (including the 17 already refreshed). Windows/MinGW and Linux rebuild all affected targets and their real production library dependencies: all 34 registered CTests pass per OS, zero skips, including guest_wmemcpy_extent exactly once and the three related string groups.

The fixture is byte-identical to the preceding published head and the combined tree. No assertion or timeout is weakened. Current-main conventions with this full description, whitespace, exact ancestry and fixture retention pass. This is focused native validation, not a complete default product, local macOS, PS5/title or GPU run. Hosted checks for this new head remain pending until complete.

Earlier rationale and validation (historical revisions and dependency state)

What

Adds positive-length wmemcpy tests for 1 through 192 char16_t units, including embedded zeros and units with different high bytes. It checks full-unit copying, exact extents, the returned pointer and source/prefix preservation at protected boundaries.

Integrate the existing #13220 fix at e9f09f54fabb6408d3e2711b8df10c10112ab00e and current recorded main. The failed macOS write-watch jobs stopped in CMake configuration because execute-fault and read-fault were each registered twice. This head includes the original prerequisite commit, removing the duplicate block while retaining all 18 distinct tests. No parallel fix is introduced, no test assertion is weakened, and no timeout is changed.

This PR's fixture is byte-identical to the previous published head 4e37768823191a9818edb068869a36b3030fbbdc. The extra topic-relative CI change is the existing prerequisite's five-line registration deletion. Proposed landing order is #13220 first, then this independent topic. The submitted head already includes the fix, so its new CI can test it while the prerequisite PR remains open.

Original topic rationale and earlier overlap evidence (historical snapshots)

Adds positive-length wmemcpy tests for 1 through 192 char16_t units, including embedded zeros and units with different high bytes. It checks full-unit copying, exact extents, the returned pointer and source/prefix preservation at protected boundaries.

Implementation basis: Public string/memory contract: FreeBSD wide-character family manual, including wmemcpy. Despite the source filename wmemchr.3, its NAME and SYNOPSIS explicitly document these wide-character interfaces; it is a shared family page, not only the wmemchr function. The fixture calls the actual production function(s) through the existing public Api.hpp and constructs expected results separately from immutable input snapshots, membership tables or standard C++ range algorithms. No production function body or firmware object is copied into the fixture.

Known limits: Uses the repository's canonical char16_t declarations, not host wchar_t. It does not prove console width/layout, test overlap, or replace existing zero-length coverage.

Source/schema: core/libs/prx/libc/src/Strings.cpp core/libs/prx/libc/include/Api.hpp

Exact topic head 4e37768823191a9818edb068869a36b3030fbbdc is one independent commit on main 51c31ea17e9f66cb36fe9655ec31c81610bfcfc9. The current-main repository overlap checker against 51c31ea17e9f66cb36fe9655ec31c81610bfcfc9 reported no confirmed overlap. It could not finish comparisons for these older PRs already conflicting with main: #5021, #6085, #7081, #7447, #7715, #8150, #8965, #11628, #11661, #11689, #11698, #11947, #12688, #12700, #12965. Their shared-file diffs were reviewed separately. 13 scoped file changes projected onto main and then combined with this topic without a new conflict; 2 peer file changes already conflict with main and are not claimed automatically mergeable. They do not provide this command or test matrix. Registration uses a separate target/name and there is no landing-order dependency. Changes to shared registration before landing need a refreshed merge/test check. The original functional-overlap assessment recorded no dependency at that earlier snapshot. The twenty current topic heads are also merged into tested integration cedab5157a32822332671c90812c6780d05157e8.

Tested

Exact submitted head 431e940236ba0cb001a0a5cef248f9f033db4c17. Recorded main 6a3afa4deb18ea58a90efb8040fff8c60e4f2aab; prerequisite #13220 e9f09f54fabb6408d3e2711b8df10c10112ab00e. Both and the previous published topic head are ancestors. The fixture and topic assertions remain unchanged.

Fresh native validation uses combined commit fa9a26172d243ed3bb4258ad0e2ea2dfc27de518, containing all 31 exact submitted heads, on Windows 11 / MinGW GCC 15.2 / CMake 4.3.1 and non-root Ubuntu 26.04 / WSL / GCC 15.2 / CMake 4.2.3. This is explicit combined-commit validation, not 31 separate native product builds. Every individual fixture blob matches that combined tree; pairwise main/prerequisite merges are clean.

  • Full Ninja Release configuration with BUILD_TESTING=ON; build the 31 topic test targets plus guest_string_tests, guest_wide_string_tests and guest_string_compare_tests with their real library dependencies. The existing cached dependency sources/gitlink pins and FFMPEG_PREBUILT_DIR mechanism are reused.
  • Run the 31 registered topic CTests and guest_strings, guest_wide_strings, guest_string_compare: 34/34 pass on each OS, zero skips. This PR's registered guest_wmemcpy_extent is included exactly once. No timeout was raised.
  • Direct standalone write-watch CMake regression on each OS: recorded main's CMakeLists.txt fails configuration with both duplicate-name diagnostics. Replacing only that file with the integrated test(libc): register the macOS write watch fault tests once #13220 version configures successfully and enumerates 18 unique tests, with execute-fault/read-fault exactly once each. This verifies the failing configuration stage and test preservation; it does not claim a local macOS write-watch runtime.
  • Per-head current-main conventions with this complete body and diff whitespace checks pass. Combined public-header, script-registration and CTest skip-policy audits pass. Fixture hashes and prerequisite ancestry are checked before publication.

The combined native target build is not the complete default product build. No local macOS/WASM, PS5/title/GPU or firmware validation is claimed. Old red CI belongs to older heads without the prerequisite; new-head hosted CI starts after publication and remains pending until it actually completes. The prerequisite's external branch is unchanged.

Exact composition of the fresh native validation

#13310 ab8eeea15ea0dcb8aabe95e656a041963f8e4e8a, #13313 1ec676984f08d1ee65496d0d1c487a9fecb8ec49, #13314 1c8549945e9657645da7ce5a98693d50e85b5685, #13317 442aef391ffaec2c9cfe1d098c84dde5f19cd292, #13319 dab492c88b6bd8093c351b1726db02ad91af39ab, #13320 f27454e88a02dd9c9f0510d3afb0aa72378f7e87, #13378 8bb9ed3e152ea1611fe8216f25dd03c7830f2818, #13379 9b39858ef369da4f3eb01e4137e3f932a478ebd2, #13380 6083413b06d03a34e498629da942b8ec40ab7a9f, #13381 cf303f4142403e370f807813b99fc5aa087ad72b, #13391 15fbf68aa97b2617cd03ae94f033020028edeac6, #13394 cb3188c2b1f36700f882311191f86b897c4f01cb, #13395 8165ef480879e5eed57eb8f631abc924aed02ec7, #13303 91bd23b7dd8eb4089321bee7a4475f5fea8fcab4, #13304 fe35f6771a55f9c235fa0641ed5558632c59f314, #13305 7d3995fdbf339d101386dafa4ee0277764a5aba8, #13306 d3f06a729a3145cf1aa8b1dccafd555b0a2bee5a, #13307 9ddb498ae5ea911c04eacad584a59d0e6d66cc9e, #13308 3f77e0c49bf996aeb3e938be6e92f6e0cc139877, #13311 431e940236ba0cb001a0a5cef248f9f033db4c17, #13312 99dc47fb5679db7df08117ea7d0977fa1d875a8e, #13316 8ece7cd1d5d00b16480924bfdf18852a9c4a937c, #13318 2aef950d26736932bd3521fad1a6f9d1afecba3a, #13382 63aa31a6aecf58067ffc8a85ec6fbf870c84215d, #13383 b27ad4917985e1006a168f5c931a6b04e00cdf79, #13385 bc0f43d9fa2ce2180840d86dc3c287989429a994, #13386 de523e20f27b77ca4a85ad7ace1dd89fbc56f25a, #13387 802e6e11cc801feb7baef3946745ba3700f82512, #13389 fc97324e77f079e1225e3c5df1de9071a56b6397, #13392 3ea9223d4970cb702327cec3db96dbd9a07414f5, #13393 631d48778c8f64b57749b5f97c0fa27bfc096c42

Earlier author-reported validation (historical heads; not new results)

Topic head: 4e37768823191a9818edb068869a36b3030fbbdc. Combined integration: cedab5157a32822332671c90812c6780d05157e8, based on main 51c31ea17e9f66cb36fe9655ec31c81610bfcfc9. Ubuntu/WSL GCC 15.2, CMake 4.2.3, Python 3.14.4; Windows MinGW GCC 15.2, CMake 4.3.1, Python 3.14.3.

The registered test calls the actual libc through canonical headers, using OS-protected pages and real input/output buffers. A separate exact-head control compiled the whole relevant production source unit with this fixture: unchanged production code passed, and the fixture rejected an intentional production-function defect. This is a source-unit control, separately from the product build below.

Registered CTest guest_wmemcpy_extent passed on Linux and Windows as part of the filtered twenty-topic run: ctest --test-dir <build> -R <20-topic-pattern> -j 8 -V --output-on-failure, 20/20 passed on each platform, zero skips. cmake --build <linux-full-build> --parallel 8: complete default Linux product build passed. Windows used the full configuration and built the sixteen affected libc test targets plus relinker with --parallel 8; a complete default Windows build is not claimed. Conventions, whitespace, script registration and the public-header declaration/umbrella audit passed. No PS5, title or GPU validation run is claimed. macOS, WASM execution and hosted full-build CI are not reported as passed.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — looks good

The exact-head wmemcpy test checks full-unit copies, return-pointer identity, source preservation, and protected write bounds; no blocking issue was found.

What I ran

Reviewed PR head 4e37768823191a9818edb068869a36b3030fbbdc against current main 58d5af7daa8087d2ffbdb50ae6e0db3cb6a88e48; the merge preview was clean (tree ad12af2140fcb30203e51d54c33f3cbee8fd479c). Environment: Debian 12 x86-64, Python 3.11.2, Git 2.39.5, GCC 12.2.0. I did not build or run C++ tests locally. The author reports this exact test passed on Linux and Windows as part of a 20/20 integration run, and that an exact-head intentional mutation was rejected; not independently run here. Hosted conventions, eligibility, intake, labels, report and portability passed; Linux was skipped and the remaining build jobs were pending at final refresh.

git diff --check 58d5af7daa8087d2ffbdb50ae6e0db3cb6a88e48...4e37768823191a9818edb068869a36b3030fbbdc
python3 tools/check_conventions.py --base 58d5af7daa8087d2ffbdb50ae6e0db3cb6a88e48 --head 4e37768823191a9818edb068869a36b3030fbbdc --title 'test(libc): copy guarded UTF-16 buffers by unit counts' --body /tmp/pr-13311-body.md
python3 tools/pr_test_quality.py --root /tmp/anyps5-review-13311 --json core/libs/prx/libc/CMakeLists.txt core/libs/tests/GuestWmemcpyExtent.cpp
git merge-tree --write-tree 58d5af7daa8087d2ffbdb50ae6e0db3cb6a88e48 refs/review/pr-13311
gh pr checks 13311 -R boykopovar/AnyPS5

Results

Check Result
Diff whitespace Passed; no output from git diff --check.
Conventions and PR template Passed against current main.
Current-main merge preview Passed; clean tree ad12af2140fcb30203e51d54c33f3cbee8fd479c.
Test registration/source review Registered as guest_wmemcpy_extent; tests 1–192 code units, including embedded zeros and high-byte values, with source/output extent assertions and prefix sentinels.
Test-quality heuristic Reported possible missing CMake registration; manual inspection confirmed the target registration through ${LIBS_INCLUDE_DIR}/tests/GuestWmemcpyExtent.cpp.
Author-reported tests Focused test and filtered integration CTests reportedly passed 20/20 on Linux and Windows; intentional production mutation reportedly failed as expected. Not independently run here.
Hosted CI Policy checks and library portability passed; Linux skipped; remaining build jobs pending.
Build/local C++ execution Not run locally.
Unedited local command output

git diff --check produced no output.

Conventions:

All conventions checks passed.

Merge preview:

ad12af2140fcb30203e51d54c33f3cbee8fd479c

Hosted checks:

linux	skipping	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069733423
report	pass	1m0s	https://github.com/boykopovar/AnyPS5/actions/runs/41056128529/job/124063992554
labels	pass	9s	https://github.com/boykopovar/AnyPS5/actions/runs/41056128156/job/124063993460
relinker (macos-latest, clang++)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731241
relinker-asan	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731148
windows	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731121
relinker (ubuntu-latest, g++)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731221
relinker (windows-latest, g++)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731319
library-portability	pass	1m47s	https://github.com/boykopovar/AnyPS5/actions/runs/41056128489/job/124063992442
conventions	pass	18s	https://github.com/boykopovar/AnyPS5/actions/runs/41056128205/job/124063992925
register-context (ubuntu-latest, Debug)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731155
intake	pass	4s	https://github.com/boykopovar/AnyPS5/actions/runs/41056127923/job/124063992052
register-context (ubuntu-latest, Release)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731268
register-context (macos-15-intel, Debug)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731244
register-context (macos-15-intel, Release)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731175
write-watch (macos-15-intel)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731165
write-watch (macos-15)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731208
eligibility	pass	10s	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124063992135
relinker (macos-latest, clang++, x86_64)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056128449/job/124069731385

Negative control

Assertion Mutation reported by author Result
Fixture detects an incorrect wmemcpy implementation Author reports exact-head source-unit mutation Author reports expected failure; I did not run the control locally.

Findings

Blocking

None found in the reviewed source.

Non-blocking

The PR body cites FreeBSD wmemchr(3) as the behavior reference for wmemcpy; that reference is for a different interface. Consider replacing it with a reference for the tested function. This does not block the test change.

Sources

Not verified

No local build or C++ execution, independent Linux/Windows run, macOS or WASM runtime, title, or console test. Author-reported test and mutation results were not independently reproduced; hosted Linux skipped and build jobs remained pending.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/libs/prx/libc/CMakeLists.txt
PR 6819 core/libs/prx/libc/CMakeLists.txt
PR 7081 core/libs/prx/libc/CMakeLists.txt
PR 7447 core/libs/prx/libc/CMakeLists.txt
PR 7715 core/libs/prx/libc/CMakeLists.txt
PR 8150 core/libs/prx/libc/CMakeLists.txt
PR 8965 core/libs/prx/libc/CMakeLists.txt
PR 9929 core/libs/prx/libc/CMakeLists.txt
PR 11831 core/libs/prx/libc/CMakeLists.txt
PR 13390 core/libs/prx/libc/CMakeLists.txt
PR 13486 core/libs/prx/libc/CMakeLists.txt

@adriadam10 adriadam10 self-assigned this Oct 30, 2026
@StoneDC

StoneDC commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 I clarified the reference label and added a direct line anchor for wmemcpy. I checked the exact pinned FreeBSD source through the repository API: wmemchr.3 is the shared wide-character family manual, and its NAME/SYNOPSIS include the tested function(s). The description now makes that explicit rather than relying on the filename. Function declaration.

This is a PR-description clarification only. Head 4e37768823191a9818edb068869a36b3030fbbdc, code and tests are unchanged; the existing validation/approval still refers to that same head. No new runtime result is claimed.

AI-assisted: yes.

@StoneDC

StoneDC commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Integrated the existing shared CI fix at new head 431e940236ba0cb001a0a5cef248f9f033db4c17 on recorded main 6a3afa4deb18ea58a90efb8040fff8c60e4f2aab.

#13220's original commit e9f09f54fabb6408d3e2711b8df10c10112ab00e is now an ancestor, and its duplicate-registration deletion is present in this branch. The previous macOS failure was in configuration, before the fixture ran. The new head can configure with the fix already included; the checklist still declares #13220 first in landing order. The external prerequisite branch is unchanged.

Fresh local verification:

  • Combined commit fa9a26172d243ed3bb4258ad0e2ea2dfc27de518 contains all 31 exact updated topic heads. The fixture in this PR is byte-identical to its previous published head and the combined tree; no test assertion or timeout is changed.
  • Windows/MinGW and Linux: rebuilt the 31 affected native targets and three related string targets, with real library dependencies. All 34 registered CTests pass on each OS, zero skips, including guest_wmemcpy_extent exactly once.
  • Direct configuration regression on both OSes: main's unchanged standalone write-watch CMake file fails with the two duplicate-name errors. Replacing only that file with the integrated prerequisite configures successfully and lists 18 unique tests, with execute-fault/read-fault once each. This verifies configuration/test preservation, not a local macOS runtime.
  • Per-head conventions with the complete PR description, whitespace, prerequisite ancestry, fixture retention, combined public-header/registration and skip-policy audits pass. This is focused native target validation, not a complete default product/PS5/GPU run.

Hosted-check snapshot at 2026-10-30T17:44:30+00:00 (UTC): 5 pending. Current-head checks. Earlier red checks belong to heads without the prerequisite; new-head CI remains pending until it completes. Please re-check this integrated head.

AI-assisted: yes.

@StoneDC

StoneDC commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor Author

Follow-up to the dependency cleanup: GitHub still reported a conflict, so the source branch is now refreshed at 24e53d3149bcfcf2b8b5e0abdc18421184170d76 on recorded current main ba627802e2210187d417deb64c8f2fed8e33011f. Depends on: none remains correct after #13220 merged; its exact original commit is preserved.

The fixture is byte-identical to the preceding published head and the complete combined tree. Combined 52f04e5a6b07cea04982202ab1d9237163391686 contains all 31 exact topic heads. Fresh Windows/MinGW and Linux builds compile the actual affected targets and real production library dependencies; all 34 registered CTests pass on each OS, zero skips, including guest_wmemcpy_extent exactly once plus three related string groups. No assertion or timeout is weakened.

Description conventions, whitespace, exact main/prerequisite ancestry and fixture retention pass. This is focused native validation, not a full default product, local macOS, PS5/title or GPU run. Please recheck this updated head. Its new hosted checks remain pending until complete.

Current-head checks. AI-assisted: yes.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Nov 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants