Skip to content

feat(libc): implement collate<wchar_t>::_Getcat - #5662

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
Roojool:contrib/g15-libc-pending
Oct 15, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
Roojool:contrib/g15-libc-pending

Conversation

@Roojool

@Roojool Roojool commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

What

collate<wchar_t>::_Getcat is implemented, so the guest wide collation facet is built instead of throwing.

The implementation is the narrow one, copied from _ZNSt7collateIcE7_GetcatEPPKNSt6locale5facetEPKS1_ that sits next to it in the same file: allocate a GuestLocale::CollateFacet (0x20 bytes) through the guest's operator new, leave the two collation-table words at offsets 0x10 and 0x18 null, publish the base facet with a zero reference count, and throw std::invalid_argument("collate::_Getcat: only the C locale is supported") for a null locale, a null implementation, a null name or any name other than "C". Both entry points return size_t and answer 1, which is the facet id the C locale's single category uses.

GuestLocale models one facet layout and one CollateVtable for both element types, so the wide facet carries the narrow vtable and a collate<wchar_t> comparison, transform or hash runs the narrow do_compare, do_transform and do_hash over the guest's 16-bit characters read as bytes. The console's libc.prx exports both _Getcat (NIDs 2wz4rthdiy8 and BSVJqITGCyI) over the same facet, but its wide vtable is not reproduced here; the debt entry records this, the console's null collation-table pointers, and that no module of the PPSA59322, PPSA76677 or PPSA97358 dumps in this tree imports either symbol, so nothing here has been exercised by a title.

std::_Atomic_assert (_ZSt14_Atomic_assertPKcS0_, NID u2tMGOLaqnE) is not part of this PR. The search for its definition came back empty: GCC's libstdc++ has no _Atomic_assert in any release from 4.5.4 to 16.2 or in its ChangeLog from 2007 to 2022, and neither does LLVM's libc++ from release/3.2.x to current. It stays the NotImplemented_nid_no_patch stub rather than gaining an invented message shape.

Tested

  • guest_collate grows CheckWideGetcat, which mirrors the narrow case: a null facet argument returns 1 without allocating, an existing facet is left alone, a non-C name, a null name and a null implementation all throw without allocating, the C locale allocates exactly one 0x20-byte facet whose vtable is the narrow one and whose two collation words are null, compare and hash answer through that vtable, an invalid character range throws, and retain/release/destroy/deleteObject behave as for the narrow facet. On unmodified main (0aa4cb1) with only this test file applied the test fails with std::runtime_error: _ZNSt7collateIwE7_GetcatEPPKNSt6locale5facetEPKS1__nid_postfix not implemented; with the change it passes.
  • Windows 11, WinLibs GCC 15.2 MinGW: guest_collate passes 3 runs, and the neighbour suites pass 10/10 (guest_locale, guest_rune_locale, guest_strftime, guest_time_conversion, guest_wcsrtombs, guest_wcsrtombs_s, guest_wide_conversion, guest_generic_category, guest_character). The exception ABI suites pass 22/22; guest_exception_handler reports "Not Run" on this branch and on the base alike because the focused Windows runs built only the named targets, not its test binary; it is not a regression.
  • WSL Ubuntu, g++ 15.2: guest_collate passes 1/1 and the same ten neighbour suites pass 10/10 twice.
  • python tools/check_conventions.py --base refs/remotes/upstream/main passes; python tools/check_test_registration.py build-g15 reports 786 test executables with none unregistered.
  • No console run and no title run.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw; silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository
  • No new third-party code
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

Copilot AI balanced review requested due to automatic review settings October 14, 2026 05:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Progress report (d92a8da → 3dffc8c)

📈 System libraries: 88.75% (+0.04%, +4 implemented functions)

✅ 4 implemented
Library Name
libc WideCollateCompare
libc WideCollateHash
libc WideCollateTransform
libc _ZNSt7collateIwE7_GetcatEPPKNSt6locale5facetEPKS1__nid_postfix

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. One blocking point:

  • core/libs/prx/libc/src/LocaleSupport.cpp:278: the wide facet is published with the narrow g_collateVtable, so any collate<wchar_t> compare/transform/hash runs the narrow code over 16-bit characters read as bytes. That returns plausible but wrong results instead of throwing (e.g. little-endian byte order gives a different ordering for characters above 0xFF, hash differs from a wide hash, and do_transform throws on the zero high byte of every ASCII wide character). Exact-or-throw applies (CONTRIBUTING): either add a wide vtable with 16-bit do_compare/do_transform/do_hash, or make the wide vtable entries throw until they are implemented, and document that in the debt entry.
  • core/libs/tests/GuestCollate.cpp:101-108: the wide test only compares narrow ASCII strings through the narrow vtable, so it pins the behaviour above. Once the wide vtable exists, add rows with wchar_t-width data (a character >0xFF, an embedded high byte).

Welcome to push again with that addressed.

@Roojool
Roojool force-pushed the contrib/g15-libc-pending branch from 1867b58 to 661b167 Compare October 14, 2026 18:31
Copilot AI balanced review requested due to automatic review settings October 14, 2026 18:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 6017 docs/dev/debt/libc.md L62-63
PR 6018 docs/dev/debt/libc.md L62-63
PR 6087 core/libs/prx/libc/src/LocaleSupport.cpp L273-275

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3965 docs/dev/debt/libc.md
PR 5303 docs/dev/debt/libc.md
PR 5835 docs/dev/debt/libc.md
PR 6044 core/libs/tests/GuestCollate.cpp
PR 6107 docs/dev/debt/libc.md
PR 6222 docs/dev/debt/libc.md
PR 6226 docs/dev/debt/libc.md

Copilot AI balanced review requested due to automatic review settings October 15, 2026 05:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Roojool

Roojool commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Addressed with the real wide vtable (option 1), pushed as a second commit.

collate<wchar_t> now publishes g_wideCollateVtable with 16-bit do_compare (lexicographic wchar ordering, then length), do_transform (character copy) and do_hash (one FNV-1a round per character, so wide text below 0x100 hashes like its narrow spelling). The dispatch convention was verified first: ranges arriving at the vtable are guest 16-bit wchar_t with character counts — the old narrow-vtable behavior proves it (do_transform threw on every ASCII wide string via the null high byte; do_compare(u"\u0100", u"\u0001") returned -1 on bytes 0x00 vs 0x01).

The new test rows pin exactly your cases: a character above 0xFF (ordering now differs from byte-wise), an embedded high byte, and wide hash/transform. They fail on the previous head and pass with the fix (Windows + WSL, plus 9 locale neighbours on both). The debt entry keeps the derived-not-measured honesty, including the hash-widening judgement call and the byte-sized wide-string limitation.

@screamm

screamm commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Windows run of head 3dffc8ca9 alone, merged onto main d92a8dad3 (clean merge; it also merges cleanly onto the newer db81992c5, not built).

  • Host: Windows 11 Pro 26220, Ryzen 7 5700X, RTX 4070 (driver 596.21), WinLibs MinGW-w64 GCC 15.2.0 r7, Ninja, Release.
  • Build: the full build and libs pass with 0 errors. No warnings in the files this PR touches; the 5 warnings in the build are the same as on main.
  • Tests, --repeat until-fail:3:
    • guest_collate: 3/3.
    • the 94 tests registered in core/libs/prx/libc/CMakeLists.txt (locale, wide-string, strftime, exception-vtable and the rest): 282/282 runs.
    • Negative control: this PR's guest_collate_tests.exe with main's libc.prx aborts with what(): _ZNSt7collateIwE7_GetcatEPPKNSt6locale5facetEPKS1__nid_postfix not implemented; with this PR's libc.prx it passes.
  • Non-GPU suite (ctest -j8 --timeout 120 -E '^agc_driver_|^video_out_flip'): 813/814. hardware_oracle_compare hit its 20 s timeout while the host was busy and passed when rerun alone.
  • tools/check_conventions.py and git diff --check pass.

Titles: catalog homebrew, each relinked with that side's own relinker and libs, 60 s in a window (ANYPS5_REQUIRE_DISPLAY=1) with no input.

main d92a8dad3 this PR merged onto it
ccleste PPSA59322 running at 60 s, ccleste | FPS: 79.14 (3586), no stop line running at 60 s, ccleste | FPS: 65.85 (4394), no stop line
RommPS PPSA76677 running at 60 s, RommPS | FPS: 43.63 (1909), no stop line running at 60 s, RommPS | FPS: 27.82 (1876), no stop line
XPSemu PPSA97358 exits 0xC0000409, no frame exits 0xC0000409, no frame

XPSemu stops the same way on both sides. Its own log (data/xemu/xemu.log) ends with:

[gpu] driver failure: AGC graphics: VGT_SHADER_STAGES_EN=0x12030: unsupported geometry VGPR allocation
radv/ps5: the GPU has not reached submission 2 (at 1) in 10 s
vk_result = -4
Assertion failed: (vk_result == VK_SUCCESS && "vk check failed"), function render_geom_shader_triangles, file ../hw/xbox/nv2a/pgraph/vk/gpuprops.c, line 456.
[libc] abort() called from 0000000000f98c1c

That is the geometry-VGPR stop of #5604, not a collate stop. None of the three titles imports 2wz4rthdiy8 or BSVJqITGCyI (each loads its own libc.prx), as the PR body says, so the title runs only show that nothing regressed.

The second commit addresses both points of @adriadam10's review: the wide facet now has its own vtable (the test checks it differs from the narrow one), and there are rows with characters above 0xFF, an embedded high byte, and wide hash and transform.

One thing I would check before merging. WideCollateTransform sizes the result in bytes: for u"abc" the test expects size == 6 and capacity == 14. The narrow transform next to it follows the Dinkumware/MSVC basic_string layout exactly (inline capacity 15, length | 15, the 0x1000 threshold with 0x27 padding). In that layout, size and capacity count elements. For a 2-byte element the inline buffer holds 8 elements, the small-string capacity is 7 and the round-up mask is 7, and the string counts as heap-allocated once capacity exceeds 7 (MSVC STL _String_val::_BUF_SIZE, _Alloc_mask and _Large_mode_engaged). A guest std::wstring laid out like that would treat capacity 14 as a heap string, read the first 8 bytes of abc as its pointer, and report size() 6. If the wide string uses the same library as the narrow one, the values would be size = 3, capacity 7 inline, and length | 7 with (capacity + 1) * 2 bytes on the heap. I have not checked this on a console, and no local title reaches this code.

Minor points:

  • The PR description still says the wide facet carries the narrow vtable.
  • The 12 added comment lines are explanations. CONVENTIONS.md allows comments only for technical debt; the checker passes only because the PR also edits the debt file.

Verdict: on Windows the build, the tests (including the negative control) and the non-GPU suite pass, and the three titles behave as on main. The review points are addressed. The remaining question is the byte-sized wide transform result.

AI-assisted (Claude Code).

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the new 16-bit g_wideCollateVtable (LocaleSupport.cpp:160-220): compare by value then length, transform sized in bytes with a 16-bit terminator, hash one round per character. The wide rows in guest_collate (characters above 0xFF, embedded high byte) fail on main without the change and pass here; local build + ctest 105/105 green, and the Windows run by @screamm covers MinGW. Merging.

@adriadam10
adriadam10 merged commit f5a55a1 into boykopovar:main Oct 15, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants