Skip to content

fix(libc): ignore non-wide lengths on character scans - #13049

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
JJenkx:fix/libc-character-scan-lengths
Oct 30, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
JJenkx:fix/libc-character-scan-lengths

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What

A character field such as %lls in the title-bundled libc modules stores narrow bytes. Forwarding it to glibc can instead store 32-bit wide characters. The same discrepancy affects ignored length tokens on %c and %[ fields.

Normalize accepted non-wide lengths in host-backed character scans and in the secure scanner's generated directives. Preserve a single l, numeric lengths, widths/suppression, literal %% and scanset members. Regression cases check consumed bytes and destination guards through ordinary, secure and va-list string/file APIs.

Related work: #12428 supplies the Windows ordinary-parser length rule; #12468 supplies wide-field recognition/decoding. This implementation complements those paths. The new Windows cases exercise the secure paths changed here; the new Linux cases exercise all eight entry points.

Tested

Linux x86-64 Release, exact production/test source ce32d855ba3817332ab44e454f95f6257836943d, based on main 7a225bb303c1a452057000f32b70b56486625fd6. Clean merge preview against subsequently fetched main 1189ec3003672bb364dd9f3119ead7b3976e3c83; public branches of related PRs were preserved.

cmake --build build --parallel 2 --target libc guest_formatting_tests guest_wide_formatting_tests guest_safe_error_tests file_streams_tests
ctest --test-dir build -V -R '^(guest_formatting|guest_wide_formatting|guest_safe_errors|file_streams)$'
  • Actual libc builds and 4/4 focused suites pass on both main and the change, zero skips.
  • Both new guest_formatting_tests and file_streams_tests executables fail against coherent unmodified base libraries with the expected character-scan length failures. Restored candidate library bytes match the module used for reference comparison.
  • CPU-only reference calls into two bundled libc modules (PPSA03766 and PPSA07923), using separate processes and private temporary streams. They agree on all 4,968 rows. String cases vary nine lengths, five conversions, widths, suppression and inputs; stream cases cover eight non-wide lengths and three conversions. Observed fields include return, errno, destination prefix and constraint-handler activity.
Reference matrix Rows/library Main mismatches Change mismatches Full rows corrected
sscanf / sscanf_s 3240 1048 698 350
fscanf / fscanf_s 1728 478 128 350

700 full rows corrected; zero regressions in previously correct observed fields. Twenty further destination-prefix fields become correct while an existing short-field return mismatch remains. Existing EOF errno, wide/uppercase-conversion and short-character gaps remain explicit. The short-character gap is recorded in debt. This is not complete scanf coverage.

Authored probes, synthetic reference/main/change rows, positive/negative logs, source pins and hashes, SHA-256 7fceb0465f933407de4eab5d866804a957185ac902f1c8d32c483894d30cf233 (download verified). No SDK binaries, game assets, saves or screenshots are included.

Conventions/whitespace pass. Required overlap scan checked 1431 open PRs with no automatic positive overlap. Shared-file diffs of peers already conflicting with main were reviewed; their classifications are attached. No blanket peer-mergeability claim.

Windows/macOS execution is not locally qualified. Windows ordinary scans use the separate parser covered by #12428; this PR's Windows regression cases cover its four secure entry points. No console, title, graphics, audio or FPS claim.

Independent Windows validation, reported by bunbeans2-arch for topic ce32d855ba3817332ab44e454f95f6257836943d against main f7f03daf52: Windows 11 Pro 22631, i9-9900K, MSYS2 UCRT64 GCC 16.1, Release. file_streams and guest_formatting pass 5/5; keeping the new tests while restoring Formatting.cpp to main makes both fail 5/5; plain main passes 3/3. This is the contributor's run, not a new local Windows execution.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none; library correctness
  • GPUs tested: n/a
  • AI-assisted: yes

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — looks good

The change normalizes ignored narrow-character scan lengths while preserving the single-l wide path, with regression coverage across string, stream, secure, and va-list entry points.

What I ran

Reviewed exact head ce32d855ba3817332ab44e454f95f6257836943d against current main e7d8e353362935a1f00e893d901b1debc671f5ce; the clean merge-preview tree is 6715ec4f2bcc30b5f5733ddbe0aa744f15f1fed1. Environment: Debian Linux x86-64, Python 3.11.2; no compiler or build was run.

git merge-tree --write-tree upstream/main refs/review/pr-13049-current
git diff --check upstream/main...refs/review/pr-13049-current
python3 tools/check_conventions.py --base upstream/main --head refs/review/pr-13049-current
  • The changed source, tests, call paths, and current CI were inspected. Conventions and whitespace checks passed.
  • The PR body records author-run focused validation and its limitations; I did not independently rerun build-dependent tests.

Not verified

I did not build or run the C++ tests locally, and I did not verify physical-console or game behavior. Some hosted platform matrix jobs were still running or queued at review time.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/libs/tests/Formatting.cpp
docs/dev/debt/libc.md
PR 5303 docs/dev/debt/libc.md
PR 7081 docs/dev/debt/libc.md
PR 7386 core/libs/prx/libc/src/Formatting.cpp
core/libs/tests/FileStreams.cpp
core/libs/tests/Formatting.cpp
docs/dev/debt/libc.md
PR 7447 docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 core/libs/tests/FileStreams.cpp
docs/dev/debt/libc.md
PR 8965 docs/dev/debt/libc.md
PR 9340 core/libs/prx/libc/src/Formatting.cpp
core/libs/tests/Formatting.cpp
docs/dev/debt/libc.md
PR 9371 docs/dev/debt/libc.md
PR 9447 docs/dev/debt/libc.md
PR 9526 docs/dev/debt/libc.md
PR 9863 docs/dev/debt/libc.md
PR 10324 core/libs/prx/libc/src/Formatting.cpp
docs/dev/debt/libc.md
PR 10405 docs/dev/debt/libc.md
PR 10620 docs/dev/debt/libc.md
PR 10729 docs/dev/debt/libc.md
PR 11332 core/libs/prx/libc/src/Formatting.cpp
core/libs/tests/Formatting.cpp
docs/dev/debt/libc.md
PR 11661 docs/dev/debt/libc.md
PR 11689 docs/dev/debt/libc.md
PR 11698 docs/dev/debt/libc.md
PR 12057 core/libs/tests/FileStreams.cpp
docs/dev/debt/libc.md
PR 12363 docs/dev/debt/libc.md
PR 12688 docs/dev/debt/libc.md
PR 12700 docs/dev/debt/libc.md
PR 12965 docs/dev/debt/libc.md

@bunbeans2-arch

Copy link
Copy Markdown
Contributor

A Windows run with GCC 16: Windows 11 Pro 10.0.22631, i9-9900K, MSYS2 UCRT64 GCC 16.1.0, Release, main f7f03da with this PR's diff (ce32d855ba) applied.

  • With this PR: file_streams and guest_formatting passed 5 of 5.
  • Same tree with only Formatting.cpp put back to main (this PR's tests kept): both failed 5 of 5 on the new checks, Stream character scan ignored length stored wide data and Character scan ignored length stored wide data.
  • main f7f03da alone (its own tests and source): both passed 3 of 3.

AI-assisted: yes (Claude Code)

@adriadam10
adriadam10 merged commit 91cd8ac into boykopovar:main Oct 30, 2026
20 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the diff against the scan paths: %%, scansets with a leading ]/^], single l and widths/suppression are preserved, and the secure scanner drops non-wide lengths on s, c and [. The new tests pin consumed bytes and destination guards on all entry points and fail without the Formatting.cpp change (confirmed on Windows by @bunbeans2-arch); the merge preview with #12428 and #12468 is clean; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants