[rooklz]

What

sceCesJisGetLevel threw. It now reports the JIS X 0213 level of a plane, row and cell, as a PS5 does: int sceCesJisGetLevel(uint8_t plane, uint8_t row, uint8_t cell, uint32_t* level). The signature comes from the console module's disassembly (libSceCesCs.sprx, NID hQeUMTVgPHU).

I measured it on a PS5 with a homebrew payload that loads the console's libSceCesCs.sprx and calls it for every plane 0-3, row 0-255 and cell 0-255. On the console:

• level 1 and 2 are the JIS X 0208 cells of rows 1-47 and 48-84, 3 is the rest of plane 1, and 4 is plane 2;
• a cell JIS X 0213 leaves empty answers 0x805C0020 with its level; the 25 cells that need two code points count as assigned;
• a plane other than 1 or 2, or a row or cell outside 1-94, answers 0x805C0015 and stores level 0;
• a null level pointer answers 0x805C0001.

The function uses the JIS X 0213 tables of #10426. A native build of it matches the console for all 262144 inputs.

Tested

• guest_ces_jis2004 sweeps every plane, row and cell, hashing result and level, and checks the hash of the console's sweep (0xD41EFACB, 11233 assigned cells), plus spot cases for each level, empty cells, the out-of-range answer and a null pointer. The other guest_ces* tests pass.
• macOS (Apple clang 21, x86-64): all eight guest_ces* sources pass against this Export.cpp.
• Hosted Linux runner, head 27ef055d64 (verify run on my fork; Ubuntu 24.04, GCC, Release, lavapipe): every guest_ces* test passes; its one failure, agc_driver_sgpr_pattern_fill_keys, is main's on that runner.
• check_conventions.py passes.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: feat(libSceCesCs): add the EUC-JIS-2004 and Shift_JIS-2004 profiles #10426
• Fixes: none
• Unblocks: none
• GPUs tested: n/a
• AI-assisted: yes

[github-actions]

Progress report (88e804b → 64f05fe)

📈 System libraries: 26.42% (+0.0029%, +1 implemented functions)

✅ 1 implemented

Library	Name
libSceCesCs	sceCesJisGetLevel

Changed libraries (implemented exports)

Library	Before	After	Change
libSceCesCs	269/287 (93.73%)	270/287 (94.08%)	+0.35%

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 11614	docs/dev/debt/libSceCesCs.md L28
PR 14085	docs/dev/debt/libSceCesCs.md L28-29
PR 15409	docs/dev/debt/libSceCesCs.md L24-25, L28

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 9912	core/libs/prx/libSceCesCs/Export.cpp core/libs/prx/libSceCesCs/PendingExports.cpp docs/dev/debt/libSceCesCs.md
PR 12946	core/libs/prx/libSceCesCs/Export.cpp core/libs/prx/libSceCesCs/PendingExports.cpp core/libs/prx/libSceCesCs/include/Api.hpp docs/dev/debt/libSceCesCs.md
PR 13486	core/libs/prx/libSceCesCs/Export.cpp core/libs/prx/libSceCesCs/PendingExports.cpp core/libs/prx/libSceCesCs/include/Api.hpp docs/dev/debt/libSceCesCs.md
PR 13573	core/libs/prx/libSceCesCs/Export.cpp core/libs/prx/libSceCesCs/PendingExports.cpp core/libs/prx/libSceCesCs/include/Api.hpp docs/dev/debt/libSceCesCs.md
PR 14212	docs/dev/debt/libSceCesCs.md

[MarcZX10]

Reviewed exact head 27ef055 and its incremental 62-line change over declared prerequisite #10426. The level calculation distinguishes the JIS X 0208 ranges from the remaining plane-1 cells and plane 2; assignment status includes the 25 multi-code-point cells, and invalid plane/row/cell handling writes level zero when the output pointer is available. The exhaustive 0..3 by 0..255 by 0..255 hash and targeted cases match the reported PS5 sweep. I verified the JIS X 0213 source table permits use/modification/distribution, checked conventions and diff whitespace, and syntax-compiled the exact PR Export.cpp and GuestCesJis2004.cpp successfully. The current hosted product jobs are skipped; I did not run the exhaustive runtime sweep locally. Merge remains dependent on #10426.

[MarcZX10]

Reviewed the new sceCesJisGetLevel implementation and its exhaustive test against the prerequisite EUC-JIS-2004/Shift_JIS-2004 profile work in #10426. The bounds checks happen before table indexing, invalid coordinates zero the output when present, and valid coordinates return the level independently of whether the cell is assigned. The level classification and assignment test cover plane 2, JIS X 0208 rows, the remaining plane-1 rows, and the 25 two-code-point cells.

I independently compiled this head's Export.cpp and GuestCesJis2004.cpp with GCC 12/C++20, linked the focused test directly, and ran it successfully (exit 0). That exercises the full 4 × 256 × 256 coordinate sweep and expected hash/count, along with the profile round trips. git diff --check upstream/main...head is clean.

I could not run the full CMake target in this checkout because the other third-party git submodules are not initialized; the hosted product-build jobs are also path-skipped. The PR records the author's Linux and Windows test runs and the PS5 sweep used for the expected results. I did not independently reproduce the console measurement. With that limitation recorded, I found no blocking issue in this change.

[MarcZX10]

Requesting a fix for the assigned two-code-point JIS X 0213 cells described in the inline comment. The current tests explicitly expect valid standard mappings to fail in both directions.

[MarcZX10]

The new profile tests currently assert that EUC-JIS-2004 A4 F7 and the corresponding Unicode sequence U+304B U+309A are unassigned. The JIS X 0213 mapping table lists A4F7 as U+304B+309A (https://www.x0213.org/codetable/euc-jis-2004-std.txt, entry 0xA4F7); the corresponding Shift_JIS-2004 cell is also a two-code-point mapping. I independently confirmed Python's euc_jis_2004 codec decodes A4F7 to that pair and encodes the pair back to A4F7.

As written, decodeJis2004 looks up one scalar in PLANE1, and encodeJis2004 searches only single-scalar reverse entries, so these assigned cells cannot round-trip. Please represent the multi-scalar mappings in the string conversion paths and add decode/encode round-trip coverage for EUC and Shift_JIS. If the console intentionally rejects them despite the standard profile mapping, please document the measured console result and scope the behavior explicitly; the current test alone treats a known mapping as unassigned without evidence.

[rooklz]

@MarcZX10 thanks, that needed evidence. Measured on a PS5 (the console's sceCesMbcToUtf32, sceCesMbcsStrToUtf32Str and sceCesUtf32StrToMbcsStr with its own EUC-JIS-2004 and Shift_JIS-2004 profiles):

input	scalar decode	string decode
EUC A4F7, A5F7, A6F8, ABC4	0x805C0020, 2 bytes used	0x805C0020, nothing consumed or written
Shift_JIS 82F5, 8397, 8663	0x805C0020	0x805C0020
EUC A4AB (control)	U+304B	U+304B

UTF-32 input	EUC-JIS-2004	Shift_JIS-2004
U+304B U+309A	0x805C0020 after 1 consumed, A4 AB written	0x805C0020 after 1, 82 A9 written
U+00E6 U+0300	0, A9DC ABDC (each separately, not ABC4)

So the console's profiles don't implement the standard's two-code-point mappings in either direction, and the PR's "unassigned" is the console's behaviour, not an omission. I've made that explicit:

• feat(libSceCesCs): add the EUC-JIS-2004 and Shift_JIS-2004 profiles #10426 (7657ea7785): pins all of the cases above, including the partial encodes with their consumed and produced counts. The debt entry now says the behaviour is measured.
• This PR (cbc227c5c1): restacked onto that head.

sceCesJisGetLevel still counts those 25 cells as assigned, as measured for it. All 16 guest_ces* tests pass in a Linux x86-64 container.

[oneandonlydean]

@rooklz At cbc227c, remove the local export declarations, including sceCesJisGetLevel at line 20, and include the canonical libSceCesCs public header. The exact-head audit fails. Take the #10426 header correction first and restack. Preserve the newly measured two-code-point-cell rejection and assignment-level distinction.

Please run ctest -R public_headers, including the applicable public_headers_compile_* tests, after the header changes; rerun the affected guest tests.

[rooklz]

@oneandonlydean done at 599ed106e7. The test-local APS5_VABI declarations are removed and the test includes prx/libSceCesCs/include/Api.hpp, which already declares every function it uses. Restacked on #10426's corrected head; the two-code-point-cell checks are unchanged. In a Linux x86-64 container (Ubuntu 24.04, GCC 13.3, Release), ctest -R '^guest_ces|^public_headers' passes: public_headers, every public_headers_compile_* batch, and all guest_ces* tests.

[rooklz]

Rebased onto current `main` (head `7fae6a5547`); the only conflicts were debt-file churn, resolved with the existing merge tool, all `guest_ces*` tests still pass.

On the combining cells: measured on a real PS5 (fw 13.40) with a homebrew payload calling `libSceCesCs.sprx` directly — `sceCesMbcToUtf32`/`sceCesMbcsStrToUtf32Str` on EUC-JIS-2004 `A4F7`, `A5F7`, `A6F8`, `ABC4` and the Shift_JIS-2004 equivalents all return `0x805C0020` (unassigned), and `sceCesUtf32StrToMbcsStr` on U+304B+U+309A returns the same for both profiles. This isn't a blanket failure of multi-scalar combining: U+00E6+U+0300 on the same EUC context round-trips correctly (`consumed=2 produced=4`, encodes back to `A9DCABDC`). So the console itself rejects these four specific cells for decode/encode despite the written standard mapping them, while accepting other two-code-point cells — the existing test's expectations match the console exactly, and no code change is needed here.

This is consistent with (not a contradiction of) this PR's own `sceCesJisGetLevel` measurement, which counts these same 25 two-code-point cells as assigned for level-reporting purposes — `sceCesJisGetLevel` is a table/level lookup, a different code path from the actual decode/encode functions, and the two were measured independently on the same console.

Probe source and raw output, for the record:

```c
// probes/libSceCesCs/jis2004_combining.c (dlopens libSceCesCs.sprx on-console)
dec("euc A4F7", eucCtx, euc, "\xA4\xF7", 2); // -> scalar ret=805c0020 used=2 u=0
enc("euc 304B+309A", eucCtx, 0x304B, 0x309A); // -> ret=805c0020 consumed=1 produced=2 out=a4ab
enc("euc 00E6+0300", eucCtx, 0x00E6, 0x0300); // -> ret=0 consumed=2 produced=4 out=a9dcabdc
```

Full output: `A4F7`/`A5F7`/`A6F8`/`ABC4` (EUC) and `82F5`/`8397`/`8663` (Shift_JIS) all unassigned on decode; `304B+309A` unassigned on encode for both profiles; `00E6+0300` and `304B+0041` (plain kana, no combining) both succeed. Happy to attach the full run if useful.

[rooklz]

@MarcZX10 your change request allowed one resolution for the two-code-point cells: documenting the measured console result if the console intentionally rejects the standard mapping. That measurement is on record since 2026-10-26: on a PS5 (fw 13.40), EUC-JIS-2004 A4F7/A5F7/A6F8/ABC4 and the Shift_JIS-2004 equivalents return 0x805C0020 for both decode and encode, while other two-code-point cells (e.g. U+00E6+U+0300) round-trip on the same context, so the pinned expectations match the console and no code change was made. babyhuey's independent Windows run of the head merged onto main 4b1a9a3b2f5 found no PR-attributable failures; both guest_ces_jis2004* tests pass there. Since the measurement the head moved only by rebases and the SPIRV gitlink rollback (117c232b2c; gitlinks match main). If this satisfies the condition, could you take another look and resolve the thread?

[maskym]

Windows result for head 117c232b2c, merged alone into main 2ee8e9751a (the PR's Tested section names no Windows run; it conflicts textually with #11614 in libSceCesCs/Export.cpp, so the two were tested in separate trees).

Host: Windows 11 26300, Intel i7-12700K, WinLibs GCC 15.2.0 (x86_64-ucrt-posix-seh, r6), Release, BUILD_TESTING=ON, Ninja. Build clean.

ctest --test-dir build -R "^guest_ces|^guest_error_dialog$|import_audit": 18/18 passed, the new guest_ces_jis2004 and guest_ces_jis2004_module in 0.04 s each, import_audit in 8.55 s. Not run on a PS5.

AI-assisted: yes (Claude Code), the runs and the numbers above are from this machine.

[rooklz]

I rebased onto current main (319621ba2e). The commits from #10426 and the tools/ErrorDialog commits are already in main, so they dropped out. The branch is now the single sceCesJisGetLevel commit, at head 0647217ecf, and no submodule pointers changed. The only conflict was the pending-exports count in the debt file, which now gives the merged count (46).

At the new head: check_conventions.py passes with the PR title and body, git diff --check is clean, pr_overlap.py --branch reports only the CesCs PRs it reported before, and a fresh relinker-only build passes ctest 391/391.

AI-assisted: yes (Claude Code).

[MarcZX10]

Re-checked — approved

The console measurements you supplied resolve my earlier concern about the assigned two-code-point JIS X 0213 cells. The evidence distinguishes these specific EUC-JIS-2004/Shift_JIS-2004 mappings from other valid two-code-point sequences that do round-trip, so the measured rejection is consistent with the implementation and pinned expectations. The PR description now records those console results; the current head contains no behavior change that would invalidate the prior implementation review.

I also confirmed prerequisite #10426 is merged, this PR records it as a dependency, and the current branch is mergeable. The author reports Windows tests on a merge with current main and the prior exhaustive sweep plus macOS test results; hosted Windows and policy checks pass, while Linux hosted CI is skipped. I did not rerun the exhaustive sweep locally.

[rooklz]

Rebased onto current main (f776fddcff), head 04112df6ff. The sceCesJisGetLevel commit is unchanged; the only merge reconciliation is the pending-export count in the debt entry, which now matches main's 26 minus this export. No submodule pointers changed.

Re-verified at this head in a Linux x86-64 container (Ubuntu 24.04, GCC 13.3, Release): check_conventions.py passes, guest_ces_jis2004 and guest_ces_jis2004_module pass (2/2), and pr_overlap.py --branch exits 0. AI-assisted: yes.

[MarcZX10]

Re-checked — new head, looks good

The sceCesJisGetLevel implementation and its exhaustive test are unchanged from the previously approved head; the rebase only reconciles the pending-export count in the debt entry.

What I ran

Reviewed head 04112df6ff52bd2e12cc6eb14806daa1d87f1dc9 against base 1bc97b9527eaf6b56681cd3465bca67b75329161; the local merge preview against current main was clean and dependency #10426 is already in the base. Environment: Debian Linux x86_64, Python 3.11.2. I compared the exact implementation and test blocks with the previously approved head 0647217ecf16797ad496d6c3fe8e3a01e61cd636 and ran the no-build precheck.

python3 /marc/AnyPS5/tools/pr_precheck.py --repo boykopovar/AnyPS5 --pr 10751
gh api repos/boykopovar/AnyPS5/contents/core/libs/prx/libSceCesCs/Export.cpp?ref=04112df6ff52bd2e12cc6eb14806daa1d87f1dc9
gh api repos/boykopovar/AnyPS5/contents/core/libs/tests/GuestCesJis2004.cpp?ref=04112df6ff52bd2e12cc6eb14806daa1d87f1dc9

Results

Check	Result
Diff check	Passed.
Repository conventions	Passed.
Dependency and merge preview	Passed; #10426 is satisfied by base and the preview is clean.
Change since approved head	The implementation and complete test file are unchanged; only rebase context and the debt-file pending-export count differ.
Hosted eligibility, library-portability, and report	Passed on this head.
Hosted Linux and specialized product jobs	Skipped.
Author-reported tests	The author reports guest_ces_jis2004 and guest_ces_jis2004_module passed 2/2 on this head.
Precheck ABI/test-quality helpers	Not executable from the extracted PR checkout because these workspace-only helper scripts are absent there; their status is not evidence of a PR defect.

Unedited output — conventions

All conventions checks passed.

Unedited output — merge preview

# PR stack check — fresh

Target: https://github.com/boykopovar/AnyPS5/pull/10751
Target branch tip: `1bc97b9527eaf6b56681cd3465bca67b75329161`
All heads unchanged: **True**
Dependency order: #10751

## Individual merge previews

- PR #10751: **clean** against current base; head `04112df6ff52bd2e12cc6eb14806daa1d87f1dc9`; changed paths: none reported

## Dependency relationships

- #10426 → #10751: **satisfied_by_base**; overlapping changed paths: none

## Limits

Merge previews apply each exact PR head independently to the current base. A dependency head that is absent from its dependent is reported separately from merge conflicts; file overlap is only a signal and does not prove a conflict. This tool does not build, resolve conflicts, publish comments, or update branches.

Findings

Blocking

None.

Non-blocking

None.

Sources

• Implementation at the rechecked head
• Export validation and result handling at the rechecked head
• Exhaustive test and targeted boundary assertions at the rechecked head

Not verified

• I did not compile or run the C++ tests on this rebase. The implementation and test contents are identical to the previously approved head, and the author reports both focused tests passed on this head.
• Hosted Windows was still queued; hosted Linux and specialized product jobs were skipped. I did not repeat the PS5 measurement.

[rooklz]

Rebased onto current main (26fa1212b8), head 492a0e305f. The sceCesJisGetLevel commit is unchanged; the only merge reconciliation is the debt file (kept the new Cp1254/Cp1258 unassigned bytes entry and the pending-export count at 25). check_conventions.py passes; relinker-only build passes.

[oneandonlydean]

@rooklz a small one from the rebase: in docs/dev/debt/libSceCesCs.md the new sceCesJisGetLevel entry, and the existing "Single-byte profiles" and "Cp1254 and Cp1258 unassigned bytes" entries next to it, now start with a space ( - [). That rewrites the two existing entries for whitespace only. Could you drop the leading space so the diff is just your new entry and the count change? The rest of the rebase looks good to us.

[rooklz]

@oneandonlydean done at 5954984266 — the leading space is dropped from all three lines. The diff of docs/dev/debt/libSceCesCs.md is now just the new sceCesJisGetLevel entry and the 26→25 pending-export count; the "Single-byte profiles" and "Cp1254/Cp1258" entries no longer change. check_conventions.py --base origin/main passes.

[maskym]

Retested head 64f05fe297 on Windows (its own commits changed since my earlier run), merged alone into main a4eca2a056.

Host: Windows 11 26300, Intel i7-12700K, WinLibs GCC 15.2.0 (x86_64-ucrt-posix-seh, r6), Release, BUILD_TESTING=ON, Ninja. Build clean.

guest_ces_jis2004, guest_ces_jis2004_module: 2/2 pass, three runs. Not run on a PS5.

AI-assisted: yes (Claude Code), the runs and the numbers above are from this machine.

[TechLevelZero]

✅ AMD GPU test: passed

PR head 64f05fe merged onto main a4eca2a · tested 2026-11-02 21:20 UTC · 14 min

GPU	AMD Radeon RX 6300 (RADV NAVI24), driver Mesa 25.2.8-0ubuntu0.24.04.4, Vulkan 1.4.318
Host	Ubuntu 26.04.1 LTS, kernel 7.0.0-42-generic, container ubuntu:24.04
Logs	build and test logs for this run
Build	✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests	✅ 969 passed · 4 failed (0 new, 3 also fail on main, 1 unstable on main) · 3 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup	CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection	976 of 3,591 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation	✅ no new validation IDs vs main (452 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers)	⚠️ agc_driver_pipeline_journal 120.3 s, ⚠️ agc_driver_image_sample_unnormalized 57.7 s, ⚠️ agc_driver_lazy_resolved_abi 42.1 s, ⚠️ unwrap_self 37.2 s, ⚠️ public_headers 35.8 s

Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)

agc_driver_pipeline_journal: failed in 4 of the last 6 main runs

Failing on main too (3, not counted against this PR)

agc_driver_image_table
agc_driver_native_depth_bda_clear
agc_driver_native_multisample_depth

Validation IDs already present on main (4 IDs in 380 tests)

VUID-VkDeviceCreateInfo-pNext-pNext (380 tests): agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, agc_driver_bda_span_reads, agc_driver_bindless_images, and 372 more
VUID-vkCmdCopyImage-srcImage-01548 (6 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (6 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (1 test): agc_driver_prepared_exports

Skipped (3)

agc_driver_copied_writer_label_order: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_float16_unused_nan_payloads: skipped, the device has no native f16 conversions with RTE rounding and kept f16 denormals
agc_driver_ngg_vertex_wave_reduction: skipped, s3 describes a 32-lane wave only on 32-lane subgroups, and the device has 64 lanes

Reproduce

git fetch origin 64f05fe297fbb175288e7baf606f31211f9ebeff a4eca2a056bf3bbbec98ab42e9d28163e4ea9cb8
git checkout a4eca2a056bf3bbbec98ab42e9d28163e4ea9cb8 && git merge --no-ff 64f05fe297fbb175288e7baf606f31211f9ebeff
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 12 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 16 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

_{Automated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.}