Skip to content

fix(tools): recognize guest module identities in import audit - #6171

Open
johnohhh1 wants to merge 1 commit into
boykopovar:mainfrom
johnohhh1:fix/import-audit-module-identities
Open

johnohhh1 wants to merge 1 commit into
boykopovar:mainfrom
johnohhh1:fix/import-audit-module-identities

Conversation

@johnohhh1

@johnohhh1 johnohhh1 commented Oct 15, 2026 •

Copy link
Copy Markdown

What

Correct a false failing import audit for Middle Ages Ping Pong, PPSA28319 01.000.000. Its Il2CppUserAssemblies.prx declares the SCE identity Il2cppUserAssemblies. The relinker resolves that identity, but the audit reports four absent imports and a missing module.

Read ELF SONAMEs and SCE module identities using the relinker's exact-name precedence and suffix rules. Preserve ambiguity checks, import counts and physical filename requirements for host dependencies. This corrects preflight diagnostics; the title already runs.

Merge order: #7454, then #6171, agreed with #7454's author. The prepared identity-side normalization will be applied with the rebase after #7454 lands; its implementation is not bundled here.

Tested

Head 35bf15c51, rebased cleanly onto synced main 8727ddb29. The tool, tests and documentation are byte-for-byte identical to the earlier October 20 head 98596149e; fresh Linux checks and the saved title audit pass again. The contribution combines declared identities with main's new ModuleNames filename-suffix and Windows case matching. It preserves unscoped guest-import lookup, NID disambiguation normalization, PE support and the Windows host-importer set. Host lookup still receives physical module filenames rather than guest identity aliases.

  • Real title audit: with identical saved main-2626e5ee7 registries/libraries and current source classification, main's tool exits 1 with four absent imports; the PR exits 0 with none. Both retain 2,355 references, 1,052 unique imports and eight stub classifications. No missing host dependencies. This reruns the audit, not the game or native build.
  • Linux: 126 tests pass, one Windows-only skip; registered CTest passes. Conventions against current upstream and whitespace checks pass.
  • Earlier October 20 author-run CI: Windows Server 2025/Python 3.12 and 3.14 each passes 125 tests with two POSIX-only skips; Ubuntu passes 126 with one Windows-only skip. Those runs used 98596149e, whose audit tool/test blobs exactly match this head; Windows was not rerun for this rebase. Fresh local Linux validation also passes all six subprocess CLI checks. The seven regression methods produce 83 failed subcases and zero errors against main's tool. No upstream native CI is claimed.
  • Earlier combination evidence: October 19's prepared resolution with fix(tools): resolve $ORIGIN-prefixed converted module dependencies in import_audit #7454 11840ad87 and our 74c5e7029 passed 122 Linux tests with one skip, plus four cross-feature methods. The author and another contributor report compatible combinations at their recorded pins. These are historical results; the combination will be revalidated after fix(tools): resolve $ORIGIN-prefixed converted module dependencies in import_audit #7454 lands on current main.

Pinned evidence, CI results and replay patch. Historical runtime evidence remains main 2626e5ee7: rendered match and exit 0 after SIGINT at 45.47 s, RTX 5070/NVIDIA 595.99.02, SDL dummy audio. No full-playability, performance, console result or second affected title is claimed. Earlier independent Windows validation applies to the original revision.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: fix(tools): resolve $ORIGIN-prefixed converted module dependencies in import_audit #7454
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes, Codex; author responsible for the change

@ideabsassistenza

Copy link
Copy Markdown
Contributor

Independent Windows check of 657f16ea286035a8615e2d56614a354cd4559e31, against baseline ab9a4768407d862d1879d7d0ee66bd9c3d9d67a8, using CPython 3.12.14 on Windows 11 x64/NTFS.

  • Complete import-audit module: 82 passed, 3 skipped (85 tests, 10.704 s). The skips are the two POSIX permission tests and source-commit stamping because this check uses extracted files without a Git checkout. The Windows read-only replacement test and symlink-report test both execute and pass.
  • The same six newly added test methods against the baseline tool produce 55 failed assertions/subcases, no errors; all pass with the PR. These include the 33 valid declared-alias combinations, ambiguous/invalid-name refusals and import-count preservation.
  • Six additional subprocess CLI checks from paths containing spaces pass with the PR: section-based SONAME, sectionless SysV/SCE identity, GNU-hash/SCE identity, undeclared filename-case mismatch, ambiguous identity, and multiple aliases. The valid alias cases change exit 1 to 0 with no absent imports or missing modules. Multiple aliases retain four references rather than duplicating the module's own import. An undeclared case mismatch still reports absent/missing on Windows. Ambiguity now exits 2 and preserves the previous JSON report instead of replacing it with an incomplete diagnosis.

Exact tool/test blobs were extracted from Git; the workspace checkout was unchanged. Python/synthetic files only: no title replay, native build, CMake/CTest, GPU or Linux/macOS run. Composition with #5571 or current main was not tested; the overlap still needs handling separately.

AI-assisted testing and reporting: OpenAI Codex.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 15, 2026
This was referenced Oct 15, 2026
@adriadam10 adriadam10 self-assigned this Oct 18, 2026
@github-actions github-actions Bot added the unblocks title Names a title ID on the Unblocks line label Oct 18, 2026
@johnohhh1
johnohhh1 force-pushed the fix/import-audit-module-identities branch from 79b5bcd to a12828c Compare October 18, 2026 23:03
@github-actions github-actions Bot removed the unblocks title Names a title ID on the Unblocks line label Oct 18, 2026
@johnohhh1

Copy link
Copy Markdown
Author

@MarcZX10 @ggkouujere-debug I checked #6171 rebased to a12828cf6 on main fc623324a, together with #7159 at abea0949a and #7454 at 0df919176.

  • 7159 + 6171: combine the PE reader with declared-name collection and preserve the three-value read_modules return. Adapting the companion PE test's unpacking gives 113 passing tests and one Windows-only skip on Linux.
  • 7454 + 6171: preserve both test insertions. The existing suite passes, but new cross-feature checks find eight failures: $ORIGIN/Declared.prx is not normalized before the SCE identity lookup, so six valid-resolution cases fail and two ambiguous identities are missed. Applying strip_origin to the requested names before that lookup resolves all eight; the missing-host-file guard still passes.
  • All three: the candidate resolution passes 114 existing tests with one Windows-only skip, plus all four additional cross-feature methods. The original patches remain separate; I have not added either of your implementations to this PR.

Pinned heads, candidate patches, replay commands and results.

Which merge order would you prefer? I can handle the rebase and the identity-side resolution here after either of yours lands. The PR currently declares no semantic dependency; I will update that once we agree on an order.

There is also fresh author-run Windows and Linux validation of the rebased tool/test files. The actual PPSA28319 audit still changes exit 1 to 0 and four absent imports to zero. This fixes a false preflight failure; the game already runs. If anyone has the same declared-name mismatch on another title, an audit report would help establish the broader impact. PPSA28319 remains our only confirmed affected title.

AI-assisted testing and reporting: OpenAI Codex.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 19, 2026
@ggkouujere-debug

Copy link
Copy Markdown

Thank you for the thorough analysis! I've verified the combination locally: merged both PRs, applied strip_origin before the identity lookup as you suggested, and all 114 tests pass.

I agree with your proposed merge order: #7454 first, then #6171 with your rebase. Please proceed with the identity-side resolution on your end.

Appreciate the collaboration!

@joedeangelis533-bot

Copy link
Copy Markdown
Contributor

@johnohhh1 @ggkouujere-debug I'll queue our Windows host-library capitalization fix after both of these PRs, preserving the order recorded above: #7454 -> #6171 -> our follow-up. The follow-up will declare Depends on: #7454, #6171; its eventual rebase will retain host_modules for host-file lookup and add the PE importer set alongside it.

Our branch changes only Windows host dependency lookup. Its seven regression tests cover capitalization, transitive gaps, ELF and converted PE module providers, mixed ELF/PE graphs, ambiguous variants/report preservation, and cycles. ELF name lookup stays case sensitive.

Validation on Windows/Python 3.12.14:

I'll open the standalone follow-up with these dependencies and take its rebase after both land. The combined resolution is compatibility evidence, not extra implementation in that follow-up. No game runtime, full native build, Linux/macOS, or performance claim.

AI-assisted implementation, testing and communication: OpenAI Codex.

@johnohhh1

Copy link
Copy Markdown
Author

Thanks @ggkouujere-debug — recorded the agreed order #7454 → #6171 in Depends on and the description.

Rebased this PR onto main b6583dd86 as 74c5e7029, resolving the new conflict after #7159 merged. The resolution retains PE guest modules, main's Windows host-importer set and Unicode diagnostics, and passes physical host_modules to the host checker. The existing PE reader regression now checks its three-value result.

Fresh author-run CI: Windows/Python 3.12 and 3.14 each passes 120 tests with two POSIX-only skips; Linux passes 121 with one Windows-only skip. Six CLI cases pass on each runner. The baseline control still gives 83 failed subcases and zero errors. No new native build or title run.

The prepared combination with #7454 11840ad87 retains the Windows lookup changes, passes 122 tests with one Windows-only skip on Linux, and passes all four cross-feature methods. The identity-side strip_origin normalization is ready for the rebase after #7454 lands; your implementation stays separate from this PR.

AI-assisted implementation, testing and reporting: OpenAI Codex.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 19, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 20, 2026
@johnohhh1
johnohhh1 force-pushed the fix/import-audit-module-identities branch from 74c5e70 to 9859614 Compare October 20, 2026 17:55
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 20, 2026
Resolve module dependencies through ELF SONAMEs and declared SCE module identities, preserving exact-name precedence and rejecting ambiguity. Keep each module import counted once across aliases and repeated inputs.

Middle Ages Ping Pong (PPSA28319) reaches a rendered match on current main, but its Il2CppUserAssemblies.prx declares Il2cppUserAssemblies. The audit falsely reported four missing imports and a missing module. Add section-based and sectionless regression coverage and document the matching rules.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs needs rebase Conflicts with main, rebase needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants