Skip to content

test(libc): guard UTF-16 string copy and concatenation extents - #13316

Open
StoneDC wants to merge 4 commits into
boykopovar:mainfrom
StoneDC:b42/wide-copy-extent
Open

StoneDC wants to merge 4 commits into
boykopovar:mainfrom
StoneDC:b42/wide-copy-extent

Conversation

@StoneDC

@StoneDC StoneDC commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What

Adds guarded wcscpy/wcscat tests through 128 source units and four destination-prefix lengths, checking full-unit terminators, exact output extent, return pointers, high/surrogate-range units and source/prefix preservation.

#13220 merged at 2026-11-01T03:58:51Z. Its original commit e9f09f54fabb6408d3e2711b8df10c10112ab00e remains an ancestor; this independent topic now has no open prerequisite. The earlier shared write-watch configuration fix is already on main and is not a separate feature of this topic.

Refreshed the source branch after GitHub still reported a conflict following the dependency cleanup. The topic fixture is unchanged.

Tested

Exact updated head f921877548573cbf7aaefde014a19b6ae3da90d8 integrates current recorded main ba627802e2210187d417deb64c8f2fed8e33011f. Complete combined validation 52f04e5a6b07cea04982202ab1d9237163391686 now contains all 31 exact topic heads (including the 17 already refreshed). Windows/MinGW and Linux rebuild all affected targets and their real production library dependencies: all 34 registered CTests pass per OS, zero skips, including guest_wide_copy_extent exactly once and the three related string groups.

The fixture is byte-identical to the preceding published head and the combined tree. No assertion or timeout is weakened. Current-main conventions with this full description, whitespace, exact ancestry and fixture retention pass. This is focused native validation, not a complete default product, local macOS, PS5/title or GPU run. Hosted checks for this new head remain pending until complete.

Earlier rationale and validation (historical revisions and dependency state)

What

Adds guarded wcscpy/wcscat tests through 128 source units and four destination-prefix lengths, checking full-unit terminators, exact output extent, return pointers, high/surrogate-range units and source/prefix preservation.

Integrate the existing #13220 fix at e9f09f54fabb6408d3e2711b8df10c10112ab00e and current recorded main. The failed macOS write-watch jobs stopped in CMake configuration because execute-fault and read-fault were each registered twice. This head includes the original prerequisite commit, removing the duplicate block while retaining all 18 distinct tests. No parallel fix is introduced, no test assertion is weakened, and no timeout is changed.

This PR's fixture is byte-identical to the previous published head 3954ef07cbe613c6694f212aee639823fa993b28. The extra topic-relative CI change is the existing prerequisite's five-line registration deletion. Proposed landing order is #13220 first, then this independent topic. The submitted head already includes the fix, so its new CI can test it while the prerequisite PR remains open.

Original topic rationale and earlier overlap evidence (historical snapshots)

Adds guarded wcscpy/wcscat tests through 128 source units and four destination-prefix lengths, checking full-unit terminators, exact output extent, return pointers, high/surrogate-range units and source/prefix preservation.

Implementation basis: Public string/memory contract: FreeBSD wide-character family manual, including wcscpy and wcscat. Despite the source filename wmemchr.3, its NAME and SYNOPSIS explicitly document these wide-character interfaces; it is a shared family page. wcscat declaration. The fixture calls the actual production function(s) through the existing public Api.hpp and constructs expected results separately from immutable input snapshots, membership tables or standard C++ range algorithms. No production function body or firmware object is copied into the fixture.

Known limits: The buffers use the canonical char16_t API and have sufficient nonoverlapping storage. Surrogate units are copied opaquely; no transcoding, invalid-input or console-layout policy is asserted.

Source/schema: core/libs/prx/libc/src/Strings.cpp core/libs/prx/libc/include/Api.hpp

Exact topic head 3954ef07cbe613c6694f212aee639823fa993b28 is one independent commit on main 51c31ea17e9f66cb36fe9655ec31c81610bfcfc9. The current-main repository overlap checker against 51c31ea17e9f66cb36fe9655ec31c81610bfcfc9 reported no confirmed overlap. It could not finish comparisons for these older PRs already conflicting with main: #5021, #6085, #7081, #7447, #7715, #8150, #8965, #11628, #11661, #11689, #11698, #11947, #12688, #12700, #12965. Their shared-file diffs were reviewed separately. 13 scoped file changes projected onto main and then combined with this topic without a new conflict; 2 peer file changes already conflict with main and are not claimed automatically mergeable. They do not provide this command or test matrix. Registration uses a separate target/name and there is no landing-order dependency. Changes to shared registration before landing need a refreshed merge/test check. The original functional-overlap assessment recorded no dependency at that earlier snapshot. The twenty current topic heads are also merged into tested integration cedab5157a32822332671c90812c6780d05157e8.

Tested

Exact submitted head 8ece7cd1d5d00b16480924bfdf18852a9c4a937c. Recorded main 6a3afa4deb18ea58a90efb8040fff8c60e4f2aab; prerequisite #13220 e9f09f54fabb6408d3e2711b8df10c10112ab00e. Both and the previous published topic head are ancestors. The fixture and topic assertions remain unchanged.

Fresh native validation uses combined commit fa9a26172d243ed3bb4258ad0e2ea2dfc27de518, containing all 31 exact submitted heads, on Windows 11 / MinGW GCC 15.2 / CMake 4.3.1 and non-root Ubuntu 26.04 / WSL / GCC 15.2 / CMake 4.2.3. This is explicit combined-commit validation, not 31 separate native product builds. Every individual fixture blob matches that combined tree; pairwise main/prerequisite merges are clean.

  • Full Ninja Release configuration with BUILD_TESTING=ON; build the 31 topic test targets plus guest_string_tests, guest_wide_string_tests and guest_string_compare_tests with their real library dependencies. The existing cached dependency sources/gitlink pins and FFMPEG_PREBUILT_DIR mechanism are reused.
  • Run the 31 registered topic CTests and guest_strings, guest_wide_strings, guest_string_compare: 34/34 pass on each OS, zero skips. This PR's registered guest_wide_copy_extent is included exactly once. No timeout was raised.
  • Direct standalone write-watch CMake regression on each OS: recorded main's CMakeLists.txt fails configuration with both duplicate-name diagnostics. Replacing only that file with the integrated test(libc): register the macOS write watch fault tests once #13220 version configures successfully and enumerates 18 unique tests, with execute-fault/read-fault exactly once each. This verifies the failing configuration stage and test preservation; it does not claim a local macOS write-watch runtime.
  • Per-head current-main conventions with this complete body and diff whitespace checks pass. Combined public-header, script-registration and CTest skip-policy audits pass. Fixture hashes and prerequisite ancestry are checked before publication.

The combined native target build is not the complete default product build. No local macOS/WASM, PS5/title/GPU or firmware validation is claimed. Old red CI belongs to older heads without the prerequisite; new-head hosted CI starts after publication and remains pending until it actually completes. The prerequisite's external branch is unchanged.

Exact composition of the fresh native validation

#13310 ab8eeea15ea0dcb8aabe95e656a041963f8e4e8a, #13313 1ec676984f08d1ee65496d0d1c487a9fecb8ec49, #13314 1c8549945e9657645da7ce5a98693d50e85b5685, #13317 442aef391ffaec2c9cfe1d098c84dde5f19cd292, #13319 dab492c88b6bd8093c351b1726db02ad91af39ab, #13320 f27454e88a02dd9c9f0510d3afb0aa72378f7e87, #13378 8bb9ed3e152ea1611fe8216f25dd03c7830f2818, #13379 9b39858ef369da4f3eb01e4137e3f932a478ebd2, #13380 6083413b06d03a34e498629da942b8ec40ab7a9f, #13381 cf303f4142403e370f807813b99fc5aa087ad72b, #13391 15fbf68aa97b2617cd03ae94f033020028edeac6, #13394 cb3188c2b1f36700f882311191f86b897c4f01cb, #13395 8165ef480879e5eed57eb8f631abc924aed02ec7, #13303 91bd23b7dd8eb4089321bee7a4475f5fea8fcab4, #13304 fe35f6771a55f9c235fa0641ed5558632c59f314, #13305 7d3995fdbf339d101386dafa4ee0277764a5aba8, #13306 d3f06a729a3145cf1aa8b1dccafd555b0a2bee5a, #13307 9ddb498ae5ea911c04eacad584a59d0e6d66cc9e, #13308 3f77e0c49bf996aeb3e938be6e92f6e0cc139877, #13311 431e940236ba0cb001a0a5cef248f9f033db4c17, #13312 99dc47fb5679db7df08117ea7d0977fa1d875a8e, #13316 8ece7cd1d5d00b16480924bfdf18852a9c4a937c, #13318 2aef950d26736932bd3521fad1a6f9d1afecba3a, #13382 63aa31a6aecf58067ffc8a85ec6fbf870c84215d, #13383 b27ad4917985e1006a168f5c931a6b04e00cdf79, #13385 bc0f43d9fa2ce2180840d86dc3c287989429a994, #13386 de523e20f27b77ca4a85ad7ace1dd89fbc56f25a, #13387 802e6e11cc801feb7baef3946745ba3700f82512, #13389 fc97324e77f079e1225e3c5df1de9071a56b6397, #13392 3ea9223d4970cb702327cec3db96dbd9a07414f5, #13393 631d48778c8f64b57749b5f97c0fa27bfc096c42

Earlier author-reported validation (historical heads; not new results)

Topic head: 3954ef07cbe613c6694f212aee639823fa993b28. Combined integration: cedab5157a32822332671c90812c6780d05157e8, based on main 51c31ea17e9f66cb36fe9655ec31c81610bfcfc9. Ubuntu/WSL GCC 15.2, CMake 4.2.3, Python 3.14.4; Windows MinGW GCC 15.2, CMake 4.3.1, Python 3.14.3.

The registered test calls the actual libc through canonical headers, using OS-protected pages and real input/output buffers. A separate exact-head control compiled the whole relevant production source unit with this fixture: unchanged production code passed, and the fixture rejected an intentional production-function defect. This is a source-unit control, separately from the product build below.

Registered CTest guest_wide_copy_extent passed on Linux and Windows as part of the filtered twenty-topic run: ctest --test-dir <build> -R <20-topic-pattern> -j 8 -V --output-on-failure, 20/20 passed on each platform, zero skips. cmake --build <linux-full-build> --parallel 8: complete default Linux product build passed. Windows used the full configuration and built the sixteen affected libc test targets plus relinker with --parallel 8; a complete default Windows build is not claimed. Conventions, whitespace, script registration and the public-header declaration/umbrella audit passed. No PS5, title or GPU validation run is claimed. macOS, WASM execution and hosted full-build CI are not reported as passed.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — looks good

The exact-head UTF-16 copy/concatenation test checks returned pointers, complete units, source preservation, and protected write extents; no blocking code or test issue was found.

What I ran

Reviewed PR head 3954ef07cbe613c6694f212aee639823fa993b28 against current main 58d5af7daa8087d2ffbdb50ae6e0db3cb6a88e48; the merge preview was clean (tree 08fdacad3d5415194ad21e519b40aa1ddfc96703). Environment: Debian 12 x86-64, Python 3.11.2, Git 2.39.5, GCC 12.2.0. I did not build or run C++ tests locally. The Linux and Windows test runs and source-unit mutation control are author-reported for an integration containing this exact PR head. Hosted eligibility, report and library portability passed; Linux was skipped and other build/policy jobs were pending at final refresh.

git diff --check 58d5af7daa8087d2ffbdb50ae6e0db3cb6a88e48...3954ef07cbe613c6694f212aee639823fa993b28
python3 tools/check_conventions.py --base 58d5af7daa8087d2ffbdb50ae6e0db3cb6a88e48 --head 3954ef07cbe613c6694f212aee639823fa993b28 --title 'test(libc): guard UTF-16 string copy and concatenation extents' --body /tmp/pr-13316-body.md
python3 tools/pr_test_quality.py --root /tmp/anyps5-review-13316 --json core/libs/prx/libc/CMakeLists.txt core/libs/tests/GuestWideCopyExtent.cpp
git merge-tree --write-tree 58d5af7daa8087d2ffbdb50ae6e0db3cb6a88e48 refs/review/pr-13316
gh pr checks 13316 -R boykopovar/AnyPS5

Results

Check Result
Diff whitespace Passed; no output from git diff --check.
Conventions and PR template Passed against current main.
Current-main merge preview Passed; clean tree 08fdacad3d5415194ad21e519b40aa1ddfc96703.
Test registration/source review Registered as guest_wide_copy_extent; source lengths 0–128 and destination prefixes 0, 1, 15, and 33 exercise wcscpy and wcscat. Expected full UTF-16 sequences include the terminator; checks also cover returned destination pointers, source preservation, and eight preceding guard units.
Test-quality heuristic Flagged possible missing registration; manual CMake inspection confirmed registration through ${LIBS_INCLUDE_DIR}/tests/GuestWideCopyExtent.cpp.
Author-reported tests The author reports the focused test and filtered 20-topic CTest set passed 20/20 on Linux and Windows, with an exact-head source-unit mutation rejected by the fixture. A full Linux build and affected-target Windows build are also reported. I did not independently run these.
Hosted CI Eligibility, report, and library portability passed; Linux skipped; remaining jobs were pending at final refresh.
Build/local C++ execution Not run locally.
Unedited local command output

git diff --check produced no output.

Conventions:

All conventions checks passed.

Test-quality heuristic:

{
  "findings": [
    {
      "path": "core/libs/tests/GuestWideCopyExtent.cpp",
      "line": 1,
      "kind": "possible-unregistered-cpp-test",
      "message": "Changed C++ test source path was not found literally in CMake files; inspect registration, generated source lists, and workflow coverage."
    }
  ],
  "negative_control_candidates": [],
  "limits": [
    "Heuristics can miss weak assertions and can flag valid tests; inspect every finding.",
    "A test that passes against base is not by itself proof that the PR test is inadequate.",
    "Negative controls are only candidates until the test actually runs against base and fails by assertion rather than import/setup error."
  ]
}

Merge preview:

08fdacad3d5415194ad21e519b40aa1ddfc96703

Hosted checks:

linux	skipping	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070047808
register-context (macos-15-intel, Debug)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045657
library-portability	pass	1m26s	https://github.com/boykopovar/AnyPS5/actions/runs/41056257546/job/124064428595
intake	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257320/job/124064431591
labels	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257586/job/124064431448
eligibility	pass	9s	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124064429181
conventions	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257583/job/124064431492
register-context (macos-15-intel, Release)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045628
register-context (ubuntu-latest, Debug)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045649
register-context (ubuntu-latest, Release)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045716
relinker (macos-latest, clang++)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070046123
relinker (macos-latest, clang++, x86_64)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045703
relinker (ubuntu-latest, g++)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045635
relinker (windows-latest, g++)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045694
relinker-asan	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045611
report	pass	58s	https://github.com/boykopovar/AnyPS5/actions/runs/41056257460/job/124064429111
windows	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045653
write-watch (macos-15)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045582
write-watch (macos-15-intel)	pending	0	https://github.com/boykopovar/AnyPS5/actions/runs/41056257475/job/124070045610

Negative control

Assertion Mutation reported by author Result
Fixture rejects an incorrect wide-copy implementation Author reports an intentional production-function mutation in the exact-head source-unit control Author reports the fixture rejected it; I did not independently run the control.

Findings

Blocking

None found in the reviewed source.

Non-blocking

The PR body cites FreeBSD wmemchr(3) as the behavior reference for wcscpy/wcscat; that link is for a different interface. Please replace it with a source that documents the functions this test covers. This does not block the test change.

Sources

Not verified

No local build or C++ execution, independent Linux/Windows run, macOS or WASM runtime, title, or console test. The reported test results are author-supplied; hosted Linux was skipped and hosted build jobs remained pending.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/libs/prx/libc/CMakeLists.txt
PR 6819 core/libs/prx/libc/CMakeLists.txt
PR 7081 core/libs/prx/libc/CMakeLists.txt
PR 7447 core/libs/prx/libc/CMakeLists.txt
PR 7715 core/libs/prx/libc/CMakeLists.txt
PR 8150 core/libs/prx/libc/CMakeLists.txt
PR 8965 core/libs/prx/libc/CMakeLists.txt
PR 9929 core/libs/prx/libc/CMakeLists.txt
PR 11831 core/libs/prx/libc/CMakeLists.txt
PR 12682 core/libs/prx/libc/CMakeLists.txt
PR 12688 core/libs/prx/libc/CMakeLists.txt
PR 12700 core/libs/prx/libc/CMakeLists.txt
PR 12965 core/libs/prx/libc/CMakeLists.txt
PR 13390 core/libs/prx/libc/CMakeLists.txt

@adriadam10 adriadam10 self-assigned this Oct 30, 2026
@StoneDC

StoneDC commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 I clarified the reference label and added a direct line anchor for wcscpy and wcscat. I checked the exact pinned FreeBSD source through the repository API: wmemchr.3 is the shared wide-character family manual, and its NAME/SYNOPSIS include the tested function(s). The description now makes that explicit rather than relying on the filename. Function declaration.

This is a PR-description clarification only. Head 3954ef07cbe613c6694f212aee639823fa993b28, code and tests are unchanged; the existing validation/approval still refers to that same head. No new runtime result is claimed.

AI-assisted: yes.

@StoneDC

StoneDC commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Integrated the existing shared CI fix at new head 8ece7cd1d5d00b16480924bfdf18852a9c4a937c on recorded main 6a3afa4deb18ea58a90efb8040fff8c60e4f2aab.

#13220's original commit e9f09f54fabb6408d3e2711b8df10c10112ab00e is now an ancestor, and its duplicate-registration deletion is present in this branch. The previous macOS failure was in configuration, before the fixture ran. The new head can configure with the fix already included; the checklist still declares #13220 first in landing order. The external prerequisite branch is unchanged.

Fresh local verification:

  • Combined commit fa9a26172d243ed3bb4258ad0e2ea2dfc27de518 contains all 31 exact updated topic heads. The fixture in this PR is byte-identical to its previous published head and the combined tree; no test assertion or timeout is changed.
  • Windows/MinGW and Linux: rebuilt the 31 affected native targets and three related string targets, with real library dependencies. All 34 registered CTests pass on each OS, zero skips, including guest_wide_copy_extent exactly once.
  • Direct configuration regression on both OSes: main's unchanged standalone write-watch CMake file fails with the two duplicate-name errors. Replacing only that file with the integrated prerequisite configures successfully and lists 18 unique tests, with execute-fault/read-fault once each. This verifies configuration/test preservation, not a local macOS runtime.
  • Per-head conventions with the complete PR description, whitespace, prerequisite ancestry, fixture retention, combined public-header/registration and skip-policy audits pass. This is focused native target validation, not a complete default product/PS5/GPU run.

Hosted-check snapshot at 2026-10-30T17:44:50+00:00 (UTC): 5 pending. Current-head checks. Earlier red checks belong to heads without the prerequisite; new-head CI remains pending until it completes. Please re-check this integrated head.

AI-assisted: yes.

@StoneDC

StoneDC commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor Author

Follow-up to the dependency cleanup: GitHub still reported a conflict, so the source branch is now refreshed at f921877548573cbf7aaefde014a19b6ae3da90d8 on recorded current main ba627802e2210187d417deb64c8f2fed8e33011f. Depends on: none remains correct after #13220 merged; its exact original commit is preserved.

The fixture is byte-identical to the preceding published head and the complete combined tree. Combined 52f04e5a6b07cea04982202ab1d9237163391686 contains all 31 exact topic heads. Fresh Windows/MinGW and Linux builds compile the actual affected targets and real production library dependencies; all 34 registered CTests pass on each OS, zero skips, including guest_wide_copy_extent exactly once plus three related string groups. No assertion or timeout is weakened.

Description conventions, whitespace, exact main/prerequisite ancestry and fixture retention pass. This is focused native validation, not a full default product, local macOS, PS5/title or GPU run. Please recheck this updated head. Its new hosted checks remain pending until complete.

Current-head checks. AI-assisted: yes.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Nov 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants