Skip to content

fix(tools): resolve Windows system library suffixes in audits - #8719

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
ArjenS-LA:fix/audit-windows-system-suffixes
Oct 23, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
ArjenS-LA:fix/audit-windows-system-suffixes

Conversation

@ArjenS-LA

@ArjenS-LA ArjenS-LA commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What

Mari0's registry names SDK .sprx dependencies while Windows loads their .prx providers, producing ten false missing-library reports. Normalize .sprx/.suprx/.debug_prx only when the corresponding actual Windows PE system provider exists. Preserve guest-module precedence, literal ELF host names, and physical host-to-host dependency checks.

Review update: Windows suffix lookup now preserves exact-case matches, resolves a unique case-insensitive PE provider to its actual filename, and leaves ambiguous matches unresolved. Current head is 037ad5dd967f0f2e8e1ada34482973e607de6efb, the original AI-assisted commit authored by MarcZX10, applied by fast-forward without rewriting it. Its mixed-case regression fails on the previous head and passes here. Windows x64 / Python 3.13.7: 123 audit tests ran, 120 passed and 3 platform cases skipped; 2 commit-producing Git fixtures were deliberately excluded. Conventions and diff whitespace checks pass. Fresh Mari0 audit runs retain the original results; a separate library layout exposing the unchanged libSceAudioOut bytes as LIBSCEAUDIOOUT.prx reproduces one false missing dependency on the previous head and removes it here. Original package, registry and library hashes remain unchanged. This is an accommodated diagnostic workflow, not guest execution. The earlier validation below remains tied to its stated commits.

Tested

Commit cf17f057303109c05ad5957d359b857a8f5c7b0d; branch fix/audit-windows-system-suffixes. Full audit suite: 124 tests run in 5.368 s, 3 platform skips, passed. Command: python -m unittest discover -s tools/tests -p test_import_audit.py.

Exactly 10 false missing-library reports disappear; complete import records and counts remain identical. Baseline fails all three SDK suffix cases; fixed passes, including controls for literal ELF names and guest precedence.

Windows 11 x64; Python 3.13.7. Fresh baseline d97361bd0df0a54b11570602391aa003bff31047. The same focused regression fails on baseline for the intended behavior and passes with this commit. Full audit suites ran after rebasing; no tracked source edits were needed during validation.

Homebrew/title evidence — indirect diagnostic workflow: Mari0 1.6.2 from websrv v0.34. Archive SHA-256 e0331ed5a22e7fe72843d268cd2c8b297d761a75b7c83d828f5cc58a82a72ce3; original love.elf 54bcc91670fd759bbd3488284dce351b6421cb83016e9ad8da6b5186da067a39; shipped libOSMesa.so.8 128d2ab20b980b96decb1f20a9ed57d10cc922605259666b63efa52963920f0a.

Fresh baseline/fixed audits use the unchanged registry generated earlier from that original package, its original shipped modules, and the same 126 hashed system-library artifacts from af691b717ad068bb93ac50bfc428af63ab259613. Historical conversion used relinker baseline 11ee778434dacf31f2ce0ecff8229e0839e36b34 with --windows --trap-sdk-syscall-stubs --registry --sce-module-path <original-Mari0> --module-dir . <original-Mari0>/love.elf <evidence>/mari0/love.exe. The trap flag, explicit module directory, external output directory, and older frozen system libraries are disclosed accommodations. This publication check does not claim reconversion with new main, game startup, guest execution, gameplay, GPU testing, or physical-PS5 behavior.

The fresh command on each tool is python <tool>/import_audit.py <pinned>/love.registry.json --libs <frozen-libraries> --source <rebased-source>/core/libs/prx --modules <original-Mari0> --json <report>. All audit runs retain exit 1 for unrelated unresolved imports. Package, registry, and library hashes remain unchanged. Every run retains 882 references and class counts {'implemented': 848, 'stub': 12, 'absent': 7, 'module': 0}; assertions check the report improvement directly.

The tree-only merge of both exact rebased commits (f92844fb00837d6d4ac66d5e7942358a0ffd782e) also passes 125 tests, with 4 skips (3 platform skips and 1 Git-fixture skip outside a checkout). Its Mari0 audit jointly preserves all 12 corrected weak flags and removes exactly 10 false missing-library reports.

Historical native validation only: before rebasing, the relinker-only build passed 303 targets under MinGW-w64 GCC 15.2.0. Broad CTest reported 211 passed, 1 failed, 1 skipped of 213. The unrelated shader_cache_runtime_sources fixture could not find Ninja; isolated runs with explicit toolchain PATH passed on baseline (65.67 s) and fixed (61.76 s). That existing fixture exceeds 30 s locally; no timeout or unrelated source was changed. long_paths remained skipped. The previous suffix branch's registered import_audit CTest passed in 18.64 s. These results are not fresh native builds of the rebased commits; this change affects Python audit tooling only.

Fresh overlap: complete 800-PR inventory against 339217ebc90f6fe91556525425aa18292d5f563b, exact peer heads, implementation review, and tree-only merge comparisons. Tested base is d97361b; the subsequent main changes affect unrelated audio libraries. Both topics merge cleanly with checked main and with each other, including newly opened #8718 at 700be18. #8572 (missing-directory hints) and #7460 (PE optional-header bounds) merge cleanly and implement different behavior. Both topics have a text conflict with #6171 (35bf15c510cb50ee8152d911c686e5eccdb95c15); the resolution must retain its declared aliases and physical host-module set together with this change. #7454 (11840ad87c4065873cb2923b2367232d33bbe5ee) already conflicts with main in audit_host_dependencies, so its pairwise result remains inconclusive. No semantic duplicate identified. #8124's guest ModuleNames support is already on main and is preserved.

Recorded report assertions: 882 references; implemented 848, stub 12, absent 7, module 0. Weak-only: exactly 12 binding flags corrected and every other import field unchanged. Suffix-only: exactly 10 false missing-library reports removed and complete import records unchanged. Combined: both improvements preserved. Registry SHA-256 d9e9cf8cf2639221e61ba5600298130c452f1e53f8572e8b363ba43c40cbb053. All input hashes were checked before and after.

The #6171 conflict is near main module setup: retain its host_modules filename set for host checks and add windows_libraries=windows to audit().

Checked overlapping heads: #8572 6736aa663d3234cf024af69162f40db455586375, #7460 70986ca71f0d44aec7d9e06b3ae7e47c7c08bcc2, #7454 11840ad87c4065873cb2923b2367232d33bbe5ee, #6171 35bf15c510cb50ee8152d911c686e5eccdb95c15. No merge ordering has been agreed with #6171.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none (diagnostic improvement only)
  • GPUs tested: n/a
  • AI-assisted: yes

@MarcZX10

Copy link
Copy Markdown
Contributor

One Windows-specific edge case I think is still uncovered: system_library() checks normalized in windows_libraries case-sensitively. Windows resolves library filenames without case sensitivity, and this tool already folds names for title modules and host dependency lookup. If the built PE provider is LibSceA.prx while the registry names libscea.sprx, this normalization will miss it and keep a false missing-library/mismatch report. Could the lookup resolve the normalized name against windows_libraries case-insensitively (while returning the actual provider filename), with a regression test?

@MarcZX10

Copy link
Copy Markdown
Contributor

Independent Linux test result on the published head cf17f057303109c05ad5957d359b857a8f5c7b0d: Debian 12 x86_64, Python 3.11.2; python3 -m unittest discover -s tools/tests -p test_import_audit.py passed 124 tests, 2 skipped. I ran the exact tool, helper, and test files from this head in an isolated temporary extraction. This verifies the focused audit suite here; I did not run a full project build.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 6171 tools/import_audit.py
tools/tests/test_import_audit.py
PR 7454 tools/import_audit.py
tools/tests/test_import_audit.py

@MarcZX10

Copy link
Copy Markdown
Contributor

I reproduced the case-sensitivity edge case against this exact head in an isolated Linux fixture: LibSceA.prx provides AAAAAAAAAAA, while the module and registry refer to libscea.sprx. The audit lists the actual provider as LibSceA.prx, but reports missing: {"libscea.sprx": 2}, library_mismatch: 1, and exits 1 under --strict-providers. This confirms the Windows filename case-folding gap from my earlier comment. A regression test for mixed-case provider resolution should catch it.

@MarcZX10

Copy link
Copy Markdown
Contributor

I prepared a follow-up for the case-sensitive Windows provider lookup. The direct push to this branch was denied (403), so I put the commit on my fork for you to cherry-pick if useful: 037ad5dd96 — fix(tools): resolve Windows providers case-insensitively. It builds a case-folded provider index, returns the provider’s actual filename, preserves an exact-case match, and leaves ambiguous case-folded matches unresolved. The new regression failed against the current PR head and now passes. Exact-head test_import_audit.py: 125 tests passed, 1 skipped; conventions and git diff --check passed. The commit is authored by MarcZX10 and was AI-assisted.

@MarcZX10

Copy link
Copy Markdown
Contributor

One Windows case remains in this suffix resolution: the provider membership check compares the normalized .prx name with windows_libraries using exact spelling. A guest can import libscea.sprx while the indexed provider is LibSceA.prx, so the Windows audit can still report it missing. I tested a follow-up locally: prefer an exact match, otherwise accept a case-folded match only when it is unique, and add a mixed-case strict-provider regression. python3 -m unittest tools.tests.test_import_audit passes (125 tests, 1 skipped) and the conventions checker passes. I couldn't push the commit to this fork (GitHub returned 403), so the tested patch is not on this PR yet.

@ArjenS-LA

Copy link
Copy Markdown
Contributor Author

Applied your offered commit 037ad5dd967f0f2e8e1ada34482973e607de6efb unchanged by fast-forward, preserving your authorship. The mixed-case regression fails on cf17f057303109c05ad5957d359b857a8f5c7b0d and passes on the new head. On Windows x64 / Python 3.13.7 the audit suite ran 123 tests: 120 passed, 3 platform skips; 2 Git fixtures that create disposable commits were excluded to respect the daily quota. Conventions and git diff --check pass.

I also reran the actual Mari0 diagnostic inputs. With one frozen provider exposed under LIBSCEAUDIOOUT.prx, the previous head falsely lists libSceAudioOut.sprx; the fixed audit removes it. Package, registry and library bytes are unchanged. Both retain exit 1 for the same unrelated absent imports. This is a deliberately case-renamed host-library layout and audit execution only, not gameplay evidence.

The refreshed 806-PR comparison against 5b59fbc5539b6c6db7a26972f6069f1c6cc179ac keeps the known #6171 text conflict in tools/import_audit.py; #7454 remains inconclusive because it conflicts with main. The other shared-file peers, including #8718, merge cleanly. The description now records this fix and its validation.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 037ad5dd967f0f2e8e1ada34482973e607de6efb. The lookup keeps exact-case matches first, accepts a case-folded match only when unique, and returns the actual provider filename, avoiding both false missing-library reports and arbitrary selection when names collide. The added strict-provider regression covers libscea.sprx resolving to LibSceA.prx.

Local validation on this head: python3 -m unittest tools.tests.test_import_audit — 125 tests passed, 1 skipped; conventions passed; git diff --check passed. The author also reports Windows audit tests and validation against the Mari0 diagnostic inputs. I did not run a full project build.

@xXPBSXx

xXPBSXx commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Linux data point for 037ad5dd96: WSL2 Ubuntu 24.04 (a virtualised kernel, not bare-metal Linux), Python 3.12, PR head alone (not merged onto main), python3 -I -m unittest discover -s tools/tests -p test_import_audit.py.

  • With the PR: 125 tests OK (1 skipped).
  • With tools/import_audit.py restored to the merge-base (the PR's tests kept): 4 failures (test_windows_guest_dependencies_use_host_prx_suffixes, test_windows_suffix_lookup_matches_provider_case_insensitively).
    Not tested: macOS, Windows, a merge onto current main.
    AI-assisted (Claude Code).

@JJenkx

JJenkx commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Independent Windows run, as data only (Windows 11 Enterprise 10.0.22621, Python 3.15). Head 037ad5d.

  • tools/tests/test_import_audit.py: OK (3 skipped) with this PR's import_audit.py. With main's import_audit.py swapped in, 4 failures.
  • No-regression check: I ran main's and this PR's import_audit.py on 22 titles' relinker registries, with the same Windows-built libraries (main 8b3a6ba) and the same --source. All 22 titles give byte-identical output. None of these registries names an .sprx, .suprx or .debug_prx library, though, so they show that ordinary titles are unaffected; they don't exercise the new suffix resolution. I don't have a Mari0-style registry to show the fixed case.

AI-assisted (Claude Code).

@oneandonlydean oneandonlydean self-assigned this Oct 22, 2026
@oneandonlydean
oneandonlydean merged commit b271388 into boykopovar:main Oct 23, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants