[gguerra26]

What

Balatro (PPSA21402, discussion #1142) stops in its bundled libc initializer on andnq %rsi, %rcx, %rax under Rosetta on an Apple M1 running macOS 15.3.1. A standalone x86-64 ANDN program reproduces SIGILL without game files. --to-rosetta previously left ANDN unchanged.

Lower 32- and 64-bit ANDN register and memory forms to scalar NOT/AND sequences. Preserve source and scratch registers, the caller's red zone, stack-relative addressing, RIP-relative operands, and the defined status flags. Plain conversion and --to-intel keep ANDN unchanged. Invalid VEX forms and a stack displacement that cannot accommodate the spill frame fail explicitly.

Semantics follow the ANDN entry in Intel SDM volume 2A. The documentation now distinguishes the earlier M5 Max observations from the instruction support observed on this M1.

Tested

• Apple M1, macOS 15.3.1 (24D70), Apple clang 17: all 150 relinker-only tests pass, including ELF/Mach-O conversion coverage for ANDN.
• A separate x86-64 build executes the generated lowering through Rosetta: 32/64-bit results, defined flags, register aliases, RSP source/destination, indirect/SIB/stack/RIP-relative memory, scratch registers and the red zone. Decoder coverage checks every destination/source register combination. Execution test completes in about 1.5 seconds.
• Balatro 01.015.000 reconverts successfully and progresses past the original ANDN stop. Its next stop is shlxq %rax, %rdx, %r8, another unsupported instruction. This PR does not establish playability or FPS.
• The game test used runtime libraries from mugurc/AnyPS5 macos-port at b7eb6b6b515c6163b6a0a6dd09b630a0414a0149, with the local Apple SDK input-number parsing build fix. The relinker is built from this upstream-based branch.
• Repository conventions and git diff --check pass. Linux/Windows execution has not been run locally; their existing CI can exercise the portable conversion checks, and Linux x86-64 also executes the lowering test.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw; no silent stubs added
• No notes, investigation files, images, game files or logs added to the repository
• No new third-party dependencies
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: PPSA21402 Balatro (PPSA21402) #1142 (the ANDN startup stop only)
• AI-assisted: yes

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 5990	docs/dev/debt/relinker.md L22

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 3891	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4156	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4396	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4397	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4422	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4423	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4424	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4520	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4521	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4522	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4523	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4566	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4599	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4718	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4740	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4749	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4760	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4813	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 4891	docs/user/USAGE.md
PR 5021	core/relinker/CMakeLists.txt docs/dev/debt/libkernel.md docs/dev/debt/relinker.md docs/user/USAGE.md
PR 5835	docs/dev/debt/libkernel.md docs/user/USAGE.md
PR 5925	docs/user/USAGE.md
PR 5993	docs/dev/debt/libkernel.md
PR 6044	core/relinker/CMakeLists.txt
PR 6111	core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp docs/dev/debt/relinker.md docs/user/USAGE.md
PR 6112	core/relinker/CMakeLists.txt docs/dev/debt/relinker.md
PR 6222	docs/dev/debt/libkernel.md

[jotitadev]

Measured on a different Apple silicon machine: M2 Max, macOS 27.2 (26B5091g).

Relinker-only build of this branch (9acd5dcc0)

• arm64 build: 150/150 tests pass.
• x86_64 build (-DCMAKE_OSX_ARCHITECTURES=x86_64) under Rosetta: 150/150 tests pass with --timeout 30, including andn_lowering.

ANDN and the next stop (shlxq) on this machine

Neither stop reproduces here. A standalone x86-64 program runs each instruction in a child process and compares the result with a reference computed natively on arm64. The following run without SIGILL and return the reference result, with and without ROSETTA_ADVERTISE_AVX=1:

• BMI1: ANDN, BEXTR, BLSI, BLSR, BLSMSK, TZCNT
• BMI2: SHLX, SHRX, SARX (32/64-bit, register and memory sources), RORX, BZHI, PDEP, PEXT, MULX
• LZCNT, POPCNT, MOVBE, FMA3, F16C

On this machine ADCX/ADOX, RDSEED, RDPID, CLWB, SHA, SSE4a, XSAVEOPT and XSAVEC raise SIGILL.

CPUID leaf 7 under Rosetta reports BMI1=0, BMI2=0 and AVX2=0 by default, and all three as 1 with ROSETTA_ADVERTISE_AVX=1. Whether these instructions execute therefore seems to depend on the macOS / Rosetta version rather than on what CPUID advertises: ANDN faults on M1 with macOS 15.3.1 and runs on M2 Max with macOS 27.2. The lowering stays correct on hosts that do execute ANDN.

Written with AI assistance (Claude Code). I ran the builds, tests and measurements above on my machine.

[011010]

Verified on macOS 27.0.1 (26A434), Apple M4, x86-64 under Rosetta 2, Apple clang 21: the relinker-only build passes ctest 150/150, including andn_lowering, which mmaps and executes the lowered sequences. The ANDN lowering holds on macOS 27 as well as the 15.3.1 you reported.

[simonbbby]

Independent Linux x86-64 execution coverage at exact head 9acd5dcc07d8149020bf50fbd5e09f7633a2613f, standalone on its recorded parent, no other PRs merged.

Ubuntu 24.04, Ryzen 9 9950X, GCC 13.3, Ninja Release, ANYPS5_RELINKER_ONLY=ON, BUILD_TESTING=ON:

• All 159/159 CTests pass (25.19 s), with no skips reported. andn_lowering, amd64_only_converter and relinker_conversion also pass 3/3 each.
• The stock ANDN fixture really executes its generated sequences on this Linux x86-64 build; it does not take the non-x86 execution fallback.
• I extended the fixture locally from its six selected register-pair patterns to all 16 destinations × 16 inverted sources × 16 second sources, both widths, three input patterns: 24,576 register execution cases, plus the existing 12 memory cases (indirect, displacement, stack, SIB and RIP-relative). All pass the existing result, defined-flags, source/scratch preservation and red-zone assertions.
• HAL supports native ANDN, so I also ran that same expanded fixture with the original ANDN bytes in place of the generated lowering, relocating the RIP-relative instruction to its actual test site. Native ANDN passes the same assertions for all 24,576 register and 12 memory cases. The extended test, exercising both modes, passes three CTest repetitions (0.05–0.06 s each), plus one direct run. These are checks against the fixture's expected semantics in each mode, not a comparison of undefined flags.
• Matched negative control: restoring only Amd64OnlyInstructionMatcher.cpp from this head's parent, retaining the PR fixture and lowering implementation, makes andn_lowering_tests exit 1 with Rosetta mode did not require an ANDN trampoline. Restoring the PR matcher and the unmodified stock test, rebuilding, then rerunning all three focused fixtures passes; the checkout is clean.

This fills Linux host execution/conversion coverage. No new title run or Rosetta execution claim; the reported title's SIGILL is not reproduced on this AMD host.

[JoaoPauloCMarra]

M4 Pro, macOS 27.0.1 (26A434), Apple clang 21, PR head 9acd5dc, relinker-only build
ctest 150/150 on arm64 and on an x86_64 build under Rosetta (--timeout 30), including andn_lowering

like the M2 Max run, plain ANDN doesnt fault here. a small x86_64 program that runs the 32 and 64-bit register forms gets the right result under Rosetta, no SIGILL (memory forms not tested)
so far only the M1 on macOS 15.3.1 faults. not sure if its the chip or the OS version, so it might be worth saying in the debt entry that macOS 27 doesnt need the lowering

AI-assisted (Claude Code), I ran all of the above on my machine