Skip to content

fix(libc): isolate byte case comparisons from the host locale - #9642

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
Noxy-HNR:fix/guest-byte-case-locale
Oct 23, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
Noxy-HNR:fix/guest-byte-case-locale

Conversation

@Noxy-HNR

@Noxy-HNR Noxy-HNR commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What

strcasecmp and strncasecmp used the host's tolower, and WebKit's strcasestr did the same. Changing host LC_CTYPE could make bytes compare equal even though the guest still exposes the C locale. On Windows code page 1252, for example, C4 and E4 became equal.

Use the existing guest case conversion for the libc comparisons and the existing ASCII folding helper for the substring search. The result difference and first-match behavior follow FreeBSD 11.4 strcasecmp.c and strcasestr.c. The debt entries retain the unmeasured console locale behavior.

Tested

Windows 11 26H2 (build 10.0.26300), WinLibs GCC 15.2.0 r7, full Release build, head 07bec52 on main d6e7ab4.

  • guest_strings, guest_fnmatch, guest_locale and guest_rune_locale pass, 4/4.
  • All 65,536 byte pairs are checked for comparisons, bounded comparisons, substring search and folded literal fnmatch, under C and an installed Windows code page 1252 locale. Empty strings, bounded nonterminated inputs, repeated prefixes, first match, no match and unchanged inputs are covered. The new regression fails against main on Windows.
  • Linux: stock installs carry only UTF-8 locales, where glibc's tolower leaves bytes above 0x7F unchanged, so the host-locale check reports a skip and passes on main too. jotitadev's run with a Latin-1 locale (localedef -i en_US -f ISO-8859-1) fails on main and passes with this PR, so Linux only catches a regression when a single-byte locale is installed.
  • tools/check_conventions.py --base origin/main and tools/check_test_registration.py pass. tools/pr_overlap.py and merge tests against the 103 open pull requests that touch these files find no new conflict.
  • PS5 hardware and title behavior are untested.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3965 docs/dev/debt/libc.md
PR 5021 core/libs/prx/libc/src/Strings.cpp
docs/dev/debt/libc.md
PR 5303 docs/dev/debt/libc.md
PR 6044 core/libs/tests/GuestFnmatch.cpp
PR 6469 docs/dev/debt/libc.md
PR 7081 docs/dev/debt/libc.md
PR 7201 core/libs/prx/libc/src/Strings.cpp
PR 7648 docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 docs/dev/debt/libc.md
PR 8306 docs/dev/debt/libc.md
PR 8658 core/libs/prx/libc/src/Strings.cpp
PR 8965 core/libs/prx/libc/src/Strings.cpp
docs/dev/debt/libc.md
PR 9133 docs/dev/debt/libc.md
PR 9390 core/libs/prx/libScePosixForWebKit/Export.cpp
docs/dev/debt/libScePosixForWebKit.md
PR 9408 docs/dev/debt/libc.md
PR 9447 docs/dev/debt/libc.md
PR 9526 docs/dev/debt/libc.md
PR 9527 docs/dev/debt/libc.md
PR 9552 docs/dev/debt/libc.md
PR 9669 docs/dev/debt/libc.md
PR 9929 docs/dev/debt/libc.md

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linux run. The fix holds on glibc, but on a stock Linux install the new host-locale check never runs, so guest_fnmatch passes on main too.

Machine: Linux, Ubuntu 26.04, GCC 15.2.0 (glibc), x86-64. Head e52eac7ff merged with git merge --no-commit onto current main d6eafa54d. Full build, BUILD_TESTING=ON, ANYPS5_RELINKER_ONLY=OFF, Ninja Release.

Default locales (C, C.utf8, en_US.utf8): none of the candidate names resolve. CheckCaseInsensitiveHostLocale prints No installed locale changes host byte case folding; host-locale regression skipped. The PR test with main's Strings.cpp and Export.cpp passes. Under a UTF-8 LC_CTYPE, glibc's tolower leaves bytes above 0x7F unchanged, so stock Linux does not hit the bug. On an image with only UTF-8 locales, the test cannot catch a regression on Linux.

With a Latin-1 locale (localedef -i en_US -f ISO-8859-1 <dir>/en_US.ISO8859-1, run with LOCPATH=<dir>, so the test's en_US.ISO8859-1 candidate resolves):

guest_fnmatch
PR test, main sources Checking guest byte comparisons with host LC_CTYPE=en_US.ISO8859-1, then Guest byte case check failed (abort)
PR Passes with the same locale; guest_posix_filesystem, guest_resolver and guest_service_database also pass

So the change also fixes the issue on Linux when the host has a single-byte locale. Today, only a Windows run, or a Linux host with a single-byte locale installed, catches a regression.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@MarcZX10

MarcZX10 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

The core change has useful breadth: all 65,536 byte pairs are exercised for the locale-sensitive operations, with unchanged-input checks and Windows code-page coverage. I noted the report of an earlier full CTest run at a prior base with four failures that reproduce on unchanged main and this branch; please rerun the full suite on the newer base before treating overall suite status as current. The focused tests and current-main conventions are author-reported; the latest hosted conventions, eligibility, intake, labels, report, relinker, registration and write-watch jobs pass, while Linux/Windows full jobs are skipped.

@MarcZX10

MarcZX10 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

I checked the current implementation and the existing Linux report. The strcasecmp paths now use the guest C-locale conversion, while strcasestr uses the existing ASCII-only fold; I found no new code issue. The Windows report covers the locale-sensitive regression, and hosted checks pass, but product Linux/Windows jobs are skipped by path filters. One separate checklist item is stale: the PR targets b27924d, while current main is 47 commits ahead at 8b3a6ba; CONTRIBUTING.md requires a current-main base. The full CTest report is also from a prior base and recorded four failures, so please rebase and rerun the focused locale/string tests (and refresh the suite report) if you decide to move the draft forward. I did not run a project build in this session.

@Noxy-HNR

Copy link
Copy Markdown
Contributor Author

Rebased onto main b7d1d47 and removed the blank lines that split the bullet lists in both debt files. guest_strings, guest_fnmatch, guest_locale and guest_rune_locale pass, 4/4, on Windows 11 with GCC 15.2.0; conventions and overlap are clean. Tested is updated, including @jotitadev's finding that stock Linux only catches a regression with a single-byte locale installed. Thanks for the Latin-1 run.

@Noxy-HNR
Noxy-HNR force-pushed the fix/guest-byte-case-locale branch from 45d0a3b to 07bec52 Compare October 22, 2026 20:11
@Noxy-HNR
Noxy-HNR marked this pull request as ready for review October 22, 2026 20:20
@MarcZX10

Copy link
Copy Markdown
Contributor

I’ve noticed that asking you to hold this PR was unnecessary. It’s fine to keep submitting PRs; maintainers will handle them in due course.

@MaxLikesCode

Copy link
Copy Markdown
Contributor

Data point for the expected values here: on #9163, @JJenkx ran the strcasecmp and strncasecmp of the libc.prx bundled with PPSA07923, PPSA05916 and PPSA13934 (comment). Over 200,000 random pairs they returned only -1, 0 or 1, and they fold ASCII only, so strcasecmp("\xc0", "\xe0") is -1. This PR's tests expect FreeBSD 11.4's folded byte difference (for example -'d'). Which module AnyPS5 follows when a title's libc.prx and libSceLibcInternal differ is the open question in #9691.

AI-assisted: yes (Claude Code)

@oneandonlydean oneandonlydean self-assigned this Oct 23, 2026
@oneandonlydean
oneandonlydean merged commit be686ec into boykopovar:main Oct 23, 2026
19 checks passed
oneandonlydean pushed a commit that referenced this pull request Oct 24, 2026
…tles' libc.prx does

strcasecmp and strncasecmp returned the folded unsigned-byte difference,
following FreeBSD 11.4's strcasecmp.c (#3966, #9642). The libc.prx bundled
with eleven titles returns -1, 0 or 1, the sign of that difference; both
functions now do the same, and the byte-pair tests expect it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
danielradosa added a commit to danielradosa/AnyPS5 that referenced this pull request Oct 24, 2026
…status and reason strings

Only the PR's own commit f916937 (Ocsp.cpp, its test and debt bullet); the upstream merges the PR branch carries (boykopovar#9643, boykopovar#9642, boykopovar#8622, boykopovar#9992, boykopovar#10059) are left out. Ocsp.cpp joins our multi-file source list.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants