[rooklz]

What

The GB 18030 tables followed ICU, and the sources disagree on 26 sequences, so the debt entry called them unmeasured. I measured both profiles on a PS5 with homebrew payloads that load the console's libSceCesCs.sprx: sceCesMbcToUtf32 over every one- and two-byte sequence, every four-byte sequence below 0x8431A530, every 0x81 0x30-led sequence and the supplementary range (sampled, plus its edges), and sceCesUtf32ToMbc over every value from U+0000 to U+110001. On the console:

• GB18030-2000 matches ICU's gb-18030-2000.ucm in the BMP, as here, but maps no supplementary character: 0x90308130-0xE3329A35 answer 0x805C0020 with length 4, and U+10000 and up do not encode.
• GB18030-2005 is the 2000 table with 0xA8BC and 0x8135F437 swapped (U+1E3F and U+E7C7), the one difference between the two profiles' sweeps. It keeps the 24 PUA cells that gb-18030-2005.ucm remaps to U+FE10-U+FE19, U+9FB4-U+9FBB and six supplementary ideographs, so those code points take their four-byte forms as in 2000. Above the BMP it maps plane 2 only, U+20000-U+2FFFF (0x95328236-0x9A348431), both ways.

The 2005 profile now uses the 2000 tables with the swap, and Gb18030.hpp drops the 2005 tables. Both profiles' encode and decode sweeps match the console apart from two topics covered elsewhere: the surrogate error code (#9175) and the lengths of rejected characters.

Tested

• guest_ces_chinese: the per-profile two-byte, four-byte and encode hashes and the supplementary sample hashes are re-pinned (the 2000 BMP hashes are unchanged), the boundary table now expects no supplementary character in 2000 and plane 2 only in 2005, the difference table keeps the swap only, and the scalar, UTF-16 and string checks use U+20000 in 2005 and refusals in 2000. The other guest_ces* tests pass.
• macOS (Apple clang 21, x86-64): all seven guest_ces* sources pass against this Export.cpp.
• Hosted Linux runner (verify run on my fork; Ubuntu 24.04, GCC, Release, lavapipe), same tree as head cb534c3d82: every guest_ces* test passes; its one failure, pr_overlap_encoding, fails the same way on main on that runner.
• check_conventions.py passes. pr_overlap.py reports feat(libSceCesCs): encode UCS-2, UTF-8, UTF-16 and UTF-32 characters into any MBCS profile #9175, feat(libSceCesCs): add the Windows-1253, 1255 and 1257 profiles #9912, feat(ces): add eight sparse single-byte profiles #10056, fix(libSceCesCs): convert single-byte characters as measured on a PS5 #10183, fix(libSceCesCs): terminate and count MBCS string conversions as measured on a PS5 #10185, fix(libSceCesCs): encode EUC-JP and Shift-JIS as measured on a PS5 #10186 and refactor(libs): add full export headers for libs #10706 on nearby lines of Export.cpp, the guest_ces* tests or the debt file; none changes the GB 18030 tables (refactor(libs): add full export headers for libs #10706 moves declarations into export headers). Merge order: feat(ces): add eight sparse single-byte profiles #10056 carries the commits of fix(libSceCesCs): terminate and count MBCS string conversions as measured on a PS5 #10185 and fix(libSceCesCs): encode EUC-JP and Shift-JIS as measured on a PS5 #10186, which list this PR as a same-line overlap, and it changes the same Chinese test; whichever of this PR and feat(ces): add eight sparse single-byte profiles #10056 lands second rebases onto the other and reruns guest_ces_chinese.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Fixes: none
• Unblocks: none
• GPUs tested: n/a
• AI-assisted: yes

[MarcZX10]

The console-measured table consolidation and per-profile tests are substantial. The required conventions check is failing because the PR title is the placeholder fix(libSceCesCs). Please update it to identify the behavior (for example, fix(libSceCesCs): map the GB 18030 profiles as the console does) and let the check rerun. I have not approved while that required check is red.

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 10056	core/libs/tests/GuestCesChinese.cpp L224-225, L227-228, L230-234, L236-239

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 9912	core/libs/prx/libSceCesCs/Export.cpp docs/dev/debt/libSceCesCs.md
PR 11614	core/libs/prx/libSceCesCs/Export.cpp docs/dev/debt/libSceCesCs.md

[MarcZX10]

I reviewed exact head cb534c3d82ea5d970f6b5ecd00d9a354ce7253c3. The checkout passes tools/check_conventions.py against current main using the PR title/body, and git diff --check is clean. The hosted conventions job fails for one concrete reason: the title fix(libSceCesCs) is not Conventional Commits syntax. Please rename it, e.g. fix(libSceCesCs): map GB 18030 profiles as measured on PS5, then rerun the check.

There is also a real merge-order dependency with #10056. That PR explicitly depends on #10186, which in turn depends on #10185; these are not all independent updates. I compared the exact trees: #10056 and this PR both change Export.cpp, Gb18030.hpp, GuestCesChinese.cpp, and the CES debt entry, and the Chinese test changes are substantial and overlapping. Please add #10056 to the dependency/merge-order list and confirm the 2005/2000 profile expectations still hold after composing those commits. #10183/#10185/#10186 already list #10739 as same-line merge-order overlap, so this is partly acknowledged; the #10056 chain is the missing dependency.

Current checks: eligibility, intake, labels and report pass; the six host/product jobs are skipped. I did not run the guest CES test locally or independently verify the console measurements. The PR reports a current-host CES test run on its fork, which is useful evidence once the branches are composed.

[MarcZX10]

The hosted conventions check fails on this exact head because the title fix(libSceCesCs) is not Conventional Commits syntax, as reported in my earlier comment. Please rename the PR with a typed scope and description (for example, fix(libSceCesCs): map GB 18030 profiles as measured on PS5) and rerun the conventions check. The merge-order dependency with #10056 also remains to be recorded: it overlaps this PR in the GB18030 tables and Chinese test, and it carries the #10186/#10185 stack. Eligibility, intake, labels and report pass; product test jobs are skipped. I have not run the guest CES test locally.

[rooklz]

@MarcZX10 thanks. The title is fixed (fix(libSceCesCs): map the GB 18030 profiles as the console does). Rebased onto current main after #10184, #10185 and #10186 landed (head 0641e0355a). The only overlap was the short-buffer check in GuestCesChinese.cpp, which now uses #10185's separate output array. On that head, merged onto main, all 14 guest_ces* tests pass, including the 2000/2005 expectations. #10056 still conflicts with main and carries those three PRs' commits. The description now lists it in the merge order: whichever of #10056 and this PR lands second rebases onto the other and reruns guest_ces_chinese.

[MarcZX10]

Reviewed updated head 0641e0355a67aac313c30563ffa9fdc569c6caaa. The title now passes conventions, the description records the real overlap/merge order with #10056, and the code and tests reflect the revised 2000/2005 supplementary mappings. The author reports all 14 guest_ces* tests passed on this head after applying it to main; the current hosted run confirms conventions, eligibility, labels, and report, while all product build jobs are skipped. I have not independently run the CES suite or verified the PS5 sweep. I found no new code blocker in the diff, but am leaving this as a comment review rather than treating the author-reported console/test evidence as independently verified.

[rooklz]

@MarcZX10 thanks for the re-review. Both items from your change request are done (the title and the #10056 merge order), so if nothing else is outstanding, could that request be dismissed or replaced with your current review? It's the only thing the merge queue is still waiting on.

[MarcZX10]

Reviewed exact head 0641e0355a67aac313c30563ffa9fdc569c6caaa. The title now meets the repository convention, and the description records the overlap/order with #10056 and its dependent stack; since either PR can land first and the second rebases, Depends on: none is consistent. The GB18030 table change and revised boundary expectations match the reported console behavior. The author reports all 14 guest_ces* tests passing on this head and provides the PS5 sweep details; I did not independently rerun the CES suite or console measurement. Hosted conventions, eligibility, labels, and report checks pass; product jobs are skipped by workflow policy. No remaining blocker found.

[rooklz]

Rebased onto current main at head 9a914b76aa. Independently re-ran the supplementary-plane sweep this PR documents (sceCesMbcToUtf32/sceCesUtf32ToGb on U+10000, U+20000, U+2FFFF, U+10FFFF against GB18030-2000, GB18030-2005, GBK and EUC-CN on the real console) and it corroborates exactly what's in the What section: 2000 maps no supplementary character, 2005 maps plane 2 only (U+20000-U+2FFFF).

Main grew two things since this PR's last head that needed reconciling on rebase, both now folded in and covered by guest_ces_chinese (79/79):

• The rejected-length convention from fix(libSceCesCs): report the length of a rejected MBCS character as the console does #10722: a 4-byte unassigned GB18030 sequence now reports consumed=4 (was unchanged) to match that measured PR.
• A couple of test assertions that assumed consumed/produced stay byte-for-byte unchanged on error actually get overwritten in specific paths (rejected<>'s length!=0 branch, and scalarToProfiled's *targetSize=size on DST_BUFFER_END); fixed to match what the implementation actually does.

No change to the 2000/2005 mapping tables or the core "what" of this PR.

[babyhuey]

Windows run of this head 9a914b76aa7, merged onto main 204c6747d0a (clean merge, local only), with the selection CI's skipped Windows product job uses: Windows 11, Ryzen 7 5800X, RTX 3080, WinLibs GCC 15.2, Release, BUILD_TESTING=ON. The tree was configured fresh after the merge, then select_tests.py select / ninja -f select.ninja selected / --target libs / select_tests.py ctest --parallel 4 --timeout 120, with check_skips.py --job windows (all skips expected).

• Selected 398 tests: 391 pass, 1 fails, pr_overlap_encoding, which also fails on 204c6747d0a itself on this machine, 6 skip. No failures from this PR.

AI-assisted: yes (Claude Code)