[StarMrGamer]

What

Part of #2136: the syscall check now names the syscall the guest tried to reach, instead of only the code offset.

Seven catalog homebrew titles stop on the raw FreeBSD syscall stubs that ps5-payload-dev's libc carries (libc/syscalls.c), and the current message gives nothing to act on:

FAIL: Forbidden syscall instruction at code offset 0xb03d47

The fixed stubs set the number with a constant right before the instruction (48 C7 C0 09 00 00 00 is mov rax, 9, FreeBSD link; B8 39 00 00 00 is mov eax, 57, symlink). SyscallScanner now looks back for a constant load into rax and adds it:

FAIL: Forbidden syscall instruction at code offset 0xa: syscall number 9 (offset 0xa)

The number is only reported when it can be established. The constant load must be the instruction directly before the syscall, or one mov r10, rcx (the SDK's first-argument move, which does not write rax) in front of it. Any other intervening instruction stops the inference rather than returning a stale value, so mov rax, 9; add rax, 1; syscall reports no number instead of 9. A stub that takes the number from a register — the SDK's generic syscall() / __syscall() wrapper reads rax from rdi — also reports no number and keeps today's text. The check still throws and nothing else changes; --skip-syscall-check is unaffected.

This does not pick one of the three directions in #2136, and the relinker still refuses the title. It makes the failure say which call it hit, which all three directions need, and it is the number #2136's author had to recover by hand with a disassembler.

Tested

Linux x86-64, relinker-only build (-DANYPS5_RELINKER_ONLY=ON -DBUILD_TESTING=ON), GCC 16.2.1: ctest 45/45. syscall_scanner covers all four forbidden opcodes — the 64-bit constant stub → number 9, the 32-bit constant stub → number 57, int 0x80 → number 11, sysenter → no number, sysret with a constant load in front of it → no number, a stub that loads rax from a register → no number, and a stub whose intervening instruction writes rax → no number (the regression for the review). test_syscall_number covers the same contract through the Python harness, so it also runs on Windows and macOS, where the C++ fixture helpers do not build. tools/check_conventions.py --base origin/main and tools/check_test_registration.py both pass.

Windows: not run here. @rmz-24 built 529476c7 (the previous head, unchanged in this respect) onto main d70b8998 with MinGW-w64 GCC 15.2.0 and reported 510/510 on the full ctest.

Not tested: a real title; no PS5 or homebrew title was run.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• AI-assisted: yes

[MarcZX10]

The syscall-number lookback can report a stale value after an intervening instruction writes RAX. I reproduced this on the PR head by adding 48 83 C0 01 (add rax, 1) between the mov rax, 9 and syscall in kConstantStub, and expecting 10. The test fails: the scanner still reports syscall number 9.

ConstantSyscallNumber currently checks only FlowKind::Sequential before continuing backward; it does not check whether the intervening instruction changes RAX/EAX. Please stop the inference when an intervening instruction may write RAX (or accurately propagate the value for supported instructions), and add a regression for this case. The original test suite passes; the added local repro exposes the diagnostic error.

[rmz-24]

Windows run: Windows 11 Pro 25H2 (10.0.26200.9457), RTX 5060 Ti (NVIDIA 617.42), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON, full build plus libs. 529476c cherry-picked onto current main d70b899 (applies cleanly).

• Full ctest: 510/510.
• syscall_scanner itself isn't built on Windows: it is registered inside the Linux x86-64 block, and the shared ElfFixture.hpp includes sys/mman.h, so it can't be moved out as is. The rest of the relinker suite, including its Windows PE execution tests, passes with this change.

AI-assisted: yes

[oneandonlydean]

#1935 is on main now and conflicts in SyscallScanner.cpp (and core/relinker/CMakeLists.txt); please rebase, keeping its opcode-offset classification as you described.

[adriadam10]

Checked on Linux: syscall_scanner and syscall_number pass, and both fail with SyscallScanner.cpp reverted to main, so they pin the change. The lookback logic (constant load directly before the instruction, or one mov r10, rcx in between, otherwise no number) looks right, and the rax-clobber case from the earlier review is covered.

One blocker, same as @oneandonlydean asked above:

• #1935 is on main now and conflicts with this PR in core/relinker/relinker/src/analysis/SyscallScanner.cpp and core/relinker/CMakeLists.txt (git merge-tree reports content conflicts in both). Please rebase and make the lookback work with the prefixed-opcode offset #1935 introduces, then re-run syscall_scanner and syscall_number.

Not covered by the Windows run linked above: it ran the previous head, and syscall_number.py was added afterwards.

[MarcZX10]

I re-reviewed current head 5fc98348b71bedb1b2f92a2ee9437f84965b93ab. The earlier issues I raised are addressed: an unrecognized intervening instruction stops syscall-number inference, and the lookback uses decoded opcode offsets so prefixed instructions work with #1935's scanner change.

Validation	Result
Linux Release relinker-only build	Passed
Full relinker CTest suite	68/68 passed
Conventions against current upstream/main	Passed
git diff --check	Passed

My earlier CHANGES_REQUESTED review no longer reflects this head, so I approve the code changes. This does not replace the repository-required manual Linux/Windows Build and Linux/Windows/macOS relinker workflow; those full Build checks are still skipped on the PR and remain required before merge.

[DavidSamir]

Independent run on macOS (Apple silicon M2, macOS 26.7.1, Apple clang 21.0.0, CMake 4.2.3, Release, relinker-only build).

5fc9834 merged onto main b5f5362 (clean merge):

• ctest 91/91, including syscall_number.
• With SyscallScanner.cpp restored to main's version, syscall_number fails and the other 90 pass, so the test pins the change.
• Not covered here: syscall_scanner is not registered on macOS.

AI-assisted: yes

[klosixgaming-arch]

Independent validation on Windows

Built the relinker and ran the PR head 5fc98348's test on my own box:

• Windows 11 Pro (26200), WinLibs MinGW-w64 GCC 15.2.0 (r7), Release, BUILD_TESTING=ON.

Results:

• test_syscall_number.py PASSES against the built relinker. All nine cases behave as documented:
  • constant stub → syscall number 9; 32-bit stub → 57; int 0x80 → 11; prefixed constant → 9;
  • register-loaded, sysenter, sysret (with a const load in front), clobbered-rax, and prefixed-clobbered → no number (kept the plain offset text);
  • in every case the relinker still refuses (non-zero exit, and the test asserts no output file is produced).

Rigor check (is the test real?): I disabled the number-reporting branch in SyscallScanner.cpp (forced the Invocation guard false), rebuilt, and the test failed — AssertionError: ('constant', 'did not report syscall number 9'). Restoring the line → PASS. So the harness genuinely verifies the new diagnostic rather than passing unconditionally.

Note on the C++ fixture: I confirmed in core/relinker/CMakeLists.txt that syscall_scanner_tests (C++) is gated to Linux x86_64 only, so on Windows/macOS the Python harness is the complete path — consistent with your note that the C++ fixture helpers don't build there. I validated via the Python harness accordingly.

Honest limits: I only verified the syscall-number diagnostic (a message improvement). I did not run a real title (you note none was run either), and I did not test Linux/macOS.

Written with AI assistance.

[adriadam10]

Rebased onto current main (only conflict: the add_test lines in core/relinker/CMakeLists.txt next to to_rosetta, both kept). Relinker build, syscall_scanner and syscall_number pass, and the scoped ctest is 92/92. The failed write-watch (macos-15-intel) job is a write_watch_threads timeout unrelated to this relinker-only change. Also confirmed by @MarcZX100 (Linux), @rmz-24 and @klosixgaming-arch (Windows), @DavidSamir (macOS). Merging.