Skip to content

fix(tools): audit ELF modules with out-of-file section tables - #5571

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
Dripl1x:codex/fix-tools-out-of-file-section-tables
Oct 15, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
Dripl1x:codex/fix-tools-out-of-file-section-tables

Conversation

@Dripl1x

@Dripl1x Dripl1x commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What

tools/import_audit.py --modules, the audit docs/user/USAGE.md describes for a title's own modules, stops with exit code 2 before it reports anything when it reads the libc.prx that homebrew titles ship. An fSELF container carries the module's original ELF header but not its section table. Unwrapping copies each segment to its program-header offset and keeps that header, so e_shoff (0x194f18, 38 sections) points past the end of the 1335962-byte file. The relinker reads program headers and converts the same file without complaint.

In elf_exports and elf_imports, use the program-header reader added by #4559 when e_shoff is zero or at or beyond the end of the file. This also avoids reading section 0 for an extended count (e_shnum == 0) outside the file. A section table that starts inside the file but runs past its end still raises AuditError.

What stops on this

No guest path is involved: this stops the documented audit of a title. On main 90d0cb481, RetroArch (PPSA99169):

python tools/import_audit.py app.registry.json app.libc.prx.guest.prx.registry.json --libs <libs> --modules <PPSA99169>/sce_module
FAIL: <PPSA99169>\sce_module\libc.prx: section header table at 0x194f18 with size 2432 lies outside the file (1335962 bytes)

Exit code 2. With this change on the same main, the audit exits 0:

682 references, 658 unique imports
  implemented    650 imports     671 references
  stub             8 imports      11 references
  absent           0 imports       0 references
  module           0 imports       0 references
...
Current stop (first unresolved import, in registry order)
  <out>\app.registry.json: none
  <out>\app.libc.prx.guest.prx.registry.json: none

libc.prx is the only module shipped by each of the 63 executable titles in ps5-homebrew-catalog. Four builds of it occur, and 58 titles share one of them. Every copy carries the same e_shoff and section count. Each title was unwrapped as described above and audited with the command above, using the tool from main and then from this change. With the tool from main, the audit stops for all 63 with the line above (one title's copy is 1336250 bytes). With this change it completes for all 63: 58 exit with 0, and 5 exit with 1 because the title itself has absent imports. <libs> are libraries from an earlier build of main; they decide the classification, not whether the module can be read.

Tested

Windows 11, Python 3.14. Head 7ff22e76d, on main 90d0cb481.

  • python tools/tests/test_import_audit.py: 64 tests pass. Three are skipped on Windows: two POSIX permission tests, and one symbolic-link test that needs a privilege this account lacks.
  • Negative control: with tools/import_audit.py from main and the new tests, 4 tests fail and 8 raise errors, all in the out-of-file section-table cases.
  • The real-title runs above.
  • The conventions checker, including the title and description, and git diff --check pass.

Not tested: Linux and macOS (the change is pure Python with no platform branch), and any title run; the audit result describes imports, not whether a title runs.

Checklist

  • Based on current main; no other open PR implements the same fallback
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw; no silent stubs added
  • No notes, investigation .md files or images added to the repository
  • No new third-party code or dependencies
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: --modules audits of the catalog titles
  • GPUs tested: n/a
  • AI-assisted: yes (written with ChatGPT; verified and opened with Claude Code)

🤖 Generated with Claude Code

Use program headers when the section table starts at or beyond EOF. Keep rejecting tables that start inside the file but are truncated.

Cover imports, exports, full module audits, and extended section counts.
@Dripl1x
Dripl1x force-pushed the codex/fix-tools-out-of-file-section-tables branch from c0ad701 to 7ff22e7 Compare October 14, 2026 05:23
@Dripl1x

Dripl1x commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Re-checked on current main 90d0cb481, because #4887, #4966 and similar PRs were closed with "no title is known to reach this".

This is a tool fix, so what stops on it is the audit of a title, not the title itself. Here is the title ID and the log line:

  • What: docs/user/USAGE.md describes tools/import_audit.py ... --modules <title>/sce_module as the way to audit a title together with its own modules. The modules have to be unwrapped from fSELF first, as the usage guide says.
  • Why it stops: an fSELF container keeps the module's original ELF header but not its section table. After unwrapping, e_shoff (0x194f18, 38 sections) points past the end of the file. The relinker reads program headers and converts the same file. import_audit.py reads the section table first and exits.
  • RetroArch (PPSA99169) on main 90d0cb481:
    FAIL: <PPSA99169>\sce_module\libc.prx: section header table at 0x194f18 with size 2432 lies outside the file (1335962 bytes)
    
    The exit code is 2 and nothing else is reported.
  • The same with this PR (rebased onto 90d0cb481, head 7ff22e76d): the audit exits 0 and reports 682 references, 658 unique imports, 0 absent, and no current stop.
  • Scope: libc.prx is the only module of each of the 63 executable titles in ps5-homebrew-catalog, and every copy has this header. With main's tool the audit stops on all 63 with the line above. With this PR it completes on all 63: 58 exit 0, and 5 exit 1 because the title itself has absent imports.
  • Tests: tools/tests/test_import_audit.py passes 64 tests (3 skipped on Windows: POSIX permissions and a symbolic-link privilege). With main's import_audit.py, the new cases give 4 failures and 8 errors.

The description is updated with the same evidence.

AI-assisted (Claude Code).

@ideabsassistenza

Copy link
Copy Markdown
Contributor

Independent Windows validation of head 7ff22e76dc54f096f724381aef1a0c936b86cf9b, using CPython 3.12.14 on Windows 11 Pro (i7-8700).

  • Complete pinned test_import_audit.py: 64 discovered cases, 62 passed and two POSIX-permission skips, in 5.318 s. The Windows symlink case ran successfully on this account.
  • Negative control: the same three new regression methods, using the unmodified reader from main 31bfc4e8693d86318824d83c47cfa418703a7cc8 in the same fixture environment, produce four failed subcases and eight errors. With the PR reader, all three methods pass, including rejection of a section table that starts inside the file but is truncated.
  • Additional generated ELF probe sets e_shoff=0x194f18 and e_shnum=38, matching the header fields reported for the homebrew module. The baseline audit entry point exits 2; the PR completes with exit 0 and resolves the fixture's imported symbol and dependency through program headers.

This probe uses a small synthetic ELF, not the real RetroArch/module files. I did not independently reproduce the catalog runs or test any title, native build or GPU path. These are pinned Python branch-head checks, not a composition with current main. No source changes made.

AI-assisted: yes (Codex); results checked against the saved logs.

@github-actions

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
#6171 tools/import_audit.py L191-192

@adriadam10 adriadam10 self-assigned this Oct 15, 2026
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the two-line change in elf_exports/elf_imports and the new tests: import_audit passes on the merged tree and fails with tools/import_audit.py reverted, and a section table that starts inside the file but is truncated is still rejected. Windows run by @ideabsassistenza also green; merging once the Windows build is green.

Note: #6171 touches the same lines (191-192), so whichever lands second needs a rebase.

@adriadam10
adriadam10 merged commit beb723a into boykopovar:main Oct 15, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants