Skip to content

feat(libkernel): thread-local storage for Mach-O guests - #11047

Open
imellag wants to merge 1 commit into
boykopovar:mainfrom
imellag:feat/libkernel-macho-tls
Open

imellag wants to merge 1 commit into
boykopovar:mainfrom
imellag:feat/libkernel-macho-tls

Conversation

@imellag

@imellag imellag commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What

This is mugurc's 8b52faaba5 from his macOS series (ci/macos-series-2, listed on #1). mugurc is co-author of the commit.

main's macOS relinker turns each %fs access of a guest image into a stub that calls Aps5GuestTlsAllocate_nid_no_patch on a thread's first access, registers each image's TLS descriptor with Aps5GuestTlsRegister_nid_no_patch (MacOsImage.cpp), and binds guest modules' __tls_get_addr to Aps5GuestTlsGetAddr_nid_no_patch (MacOsGuestModuleWriter.cpp). No library defines them, so no relinked executable starts on macOS: dyld stops at symbol not found in flat namespace '_Aps5GuestTlsAllocate_nid_no_patch'.

GuestTlsMacOs.cpp (libkernel, whole file under __APPLE__) defines them:

  • Aps5GuestTlsRegister checks the descriptor the relinker writes (version 1, layout at offsets 0 to 0x28 as MacOsImage.cpp writes it) and creates the pthread key; the key's destructor frees a thread's block.
  • Aps5GuestTlsAllocateThreadPointer allocates the block: the TLS template, zeros, then a 0x100-byte thread control block whose first word points at itself, as on FreeBSD. It stores the thread pointer under the key.
  • Aps5GuestTlsAllocate is the entry the stubs call: rax holds the descriptor and returns the thread pointer, and every other register is preserved, the upper halves of the ymm registers only where AVX exists.
  • Aps5GuestTlsGetAddr returns the thread's block plus the offset.

The libkernel debt file gets an entry for the macOS TLS design (one block and thread pointer per image, 0 at the stack guard word), and the __tls_get_addr entry no longer says the macOS resolver is undefined. Linux and Windows build an empty file.

Tested

Apple M4 Pro, macOS 27.0.1, Apple clang 21, x86-64 Release under Rosetta 2 (-DCMAKE_OSX_ARCHITECTURES=x86_64, Ninja, BUILD_TESTING=ON). main alone doesn't build the libraries on macOS, so the tree is main 61fc572cee with #10423, the open macOS build PRs (#10366, #10419, #10552, #7631, #7804, #4630, #10540, #10659, #10660, #10669, #10676), #10889, #10898 and the parts of mugurc's series that #7201 also carries. Planet of Lana (PPSA19450) is relinked with that tree's relinker (--macos --to-rosetta).

  • Without this PR: the relinked eboot stops in dyld at symbol not found in flat namespace '_Aps5GuestTlsAllocate_nid_no_patch'.
  • With it: the guest gets past its TLS setup and runs until its first window, which needs the main thread (that part is feat(libkernel): start the guest off the main thread on macOS #11048). With both, Lana boots, loads the save and plays in the village: 27 to 30 fps from 85 s in each of 10 runs of 120 s.

Linux: GuestTlsMacOs.cpp passes g++ -std=gnu++20 -fsyntax-only with GCC 14 (gcc:14 image, x86-64). Not built on Windows; the file is empty there too.

check_conventions.py --base upstream/main passes. pr_overlap.py reports #9461, which edits the line next to the __tls_get_addr debt entry, and #5021 adding a file with the same name; neither touches the same code.

#7201 (the whole macOS port) and #5021 (the native ARM64 AOT draft) carry their own versions of these functions. #7201's head (6687cbac29, 2026-10-16) conflicts with main, needs a rebase anyway, and is being split into smaller PRs such as #10552 and #10419. #5021 is a draft of more than 43,000 lines that conflicts with main and says no merge into main is implied. If either lands first with these functions, this PR is not needed and I will close it.

Checklist

  • Based on current main; no other open PR implements the same functions (except Macos port #7201 and feat(aot): consolidate native ARM64 compiler, runtime and consumers #5021, see the end of Tested)
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head fd8ac6f. The descriptor checks match the relinker TLS layout. Allocation aligns the block, copies the initialized template and zeros the remainder, stores one thread pointer per image in a pthread key, and frees each block through the key destructor. The first-access assembly preserves the guest caller-saved GPRs and XMM/YMM state around allocation while returning the thread pointer in RAX; the relinker-generated site restores the saved flags. The author reports the unresolved-symbol failure before the change and successful macOS title execution with #11048. Platform product jobs are skipped by path policy; I could not execute the Apple-specific assembly on this Linux runner.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 5021 core/libs/prx/libkernel/Pthread/src/GuestTlsMacOs.cpp

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4563 core/libs/prx/libkernel/CMakeLists.txt
PR 4630 docs/dev/debt/libkernel.md
PR 5021 core/libs/prx/libkernel/CMakeLists.txt
core/libs/prx/libkernel/Pthread/src/GuestTlsMacOs.cpp
docs/dev/debt/libkernel.md
PR 6149 docs/dev/debt/libkernel.md
PR 6936 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 7088 docs/dev/debt/libkernel.md
PR 7105 docs/dev/debt/libkernel.md
PR 7121 docs/dev/debt/libkernel.md
PR 7149 docs/dev/debt/libkernel.md
PR 7192 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 7201 core/libs/prx/libkernel/CMakeLists.txt
PR 7447 docs/dev/debt/libkernel.md
PR 7509 docs/dev/debt/libkernel.md
PR 7804 docs/dev/debt/libkernel.md
PR 8010 docs/dev/debt/libkernel.md
PR 8520 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 8605 docs/dev/debt/libkernel.md
PR 9156 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9199 docs/dev/debt/libkernel.md
PR 9284 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9293 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9297 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9302 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9383 core/libs/prx/libkernel/CMakeLists.txt
PR 9389 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9509 docs/dev/debt/libkernel.md
PR 9557 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9566 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9575 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9669 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9693 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9732 docs/dev/debt/libkernel.md
PR 9947 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 10319 docs/dev/debt/libkernel.md
PR 10442 docs/dev/debt/libkernel.md
PR 10621 docs/dev/debt/libkernel.md
PR 10666 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 10755 core/libs/prx/libkernel/CMakeLists.txt
PR 11264 docs/dev/debt/libkernel.md

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rechecked the exact current heads of #11047 (fd8ac6fcb95d0d4838aecb3bafe12d876f0103a2) and #9461 (de5306dd29180871c2010d434fdfcfa71c1f9056). Their implementation changes are independent, but git merge-tree --write-tree reports a content conflict in docs/dev/debt/libkernel.md: #9461 inserts its TSC-frequency entry next to the __tls_get_addr entry that this PR replaces and follows with the new macOS TLS entry. Both entries can be retained, but the current overlap report does not merge automatically.

CONTRIBUTING.md requires the authors to agree which PR goes first when the overlap report identifies shared lines and record the order with Depends on. Please coordinate with #9461’s author and update the checklist/rebase as agreed. This finding concerns the debt-file composition only; I found no same-function implementation in #9461. I’m leaving changes requested until the required ordering is recorded.

@imellag

imellag commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

@milescoviello #9461 and this PR both touch docs/dev/debt/libkernel.md next to the __tls_get_addr entry: this PR updates that entry and adds a macOS TLS entry below it, and #9461 adds its TSC entry below it too. The code doesn't overlap, and both entries should stay. I propose #9461 first, since it was opened earlier: once it merges I rebase this PR and keep both entries. If you'd rather have this one first, that works for me too, and then #9461 only needs the same one-line rebase. Does #9461 first work for you?

@milescoviello

Copy link
Copy Markdown
Contributor

@imellag yes, #9461 first was fine with me, sorry for the slow answer. It merged at 18:49 UTC, so your rebase keeps both entries.

AI-assisted: yes (Claude Code)

FreeBSD code reaches its thread pointer through %fs and macOS cannot set
the fs base. The macOS relinker turns each %fs access of a guest image
into a stub that reads the thread pointer from a pthread key and calls
Aps5GuestTlsAllocate on a thread's first access, registers the TLS
descriptor of an image with Aps5GuestTlsRegister, and routes
__tls_get_addr to Aps5GuestTlsGetAddr. No library provided them, so a
relinked executable could not start.

- Aps5GuestTlsRegister validates the descriptor the relinker wrote and
  creates the key; its destructor frees the block of a thread.
- Aps5GuestTlsAllocateThreadPointer allocates the block (the TLS template
  followed by zeros, then a 0x100-byte thread control block whose first
  word points at itself, as on FreeBSD) and stores the thread pointer
  under the key.
- Aps5GuestTlsAllocate is the entry the stubs call: rax holds the
  descriptor and returns the thread pointer, every other register is
  preserved, the upper halves of the ymm registers only where AVX exists.
- Aps5GuestTlsGetAddr returns the block of the thread plus the offset.

Co-authored-by: mugurc <mceker@gmail.com>
@imellag
imellag force-pushed the feat/libkernel-macho-tls branch from fd8ac6f to 337a1fc Compare October 26, 2026 08:03
@imellag

imellag commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

@milescoviello Thanks. Rebased onto main 14c99b52ac with both entries kept: #9461's sceKernelGetTscFrequency and this PR's Guest TLS on macOS.

@milescoviello

Copy link
Copy Markdown
Contributor

@imellag thanks. Both entries are there in the rebased debt file.

AI-assisted: yes (Claude Code)

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 26, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants