Skip to content

feat(libkernel): implement shm_open for anonymous objects - #1860

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
overlordxrz-source:feat/libkernel-shm-open
Oct 10, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
overlordxrz-source:feat/libkernel-shm-open

Conversation

@overlordxrz-source

Copy link
Copy Markdown
Contributor

What

libkernel does not export shm_open (QuJYZ2KVGGQ). ccleste (PPSA59322) stops on it at load on main.

Its bundled Mesa calls it in two places, both with SHM_ANON:

  • At 0x480750: shm_open(SHM_ANON, O_RDWR | O_CREAT | O_CLOEXEC, 0600), then ftruncate, then close on failure. This is Mesa's FreeBSD os_create_anonymous_file. Its caller then mmaps the descriptor MAP_SHARED.
  • At 0xb853ce: shm_open(SHM_ANON, O_RDWR [| O_CLOEXEC], 0600).

shm_open is added in the new libkernel/File/src/SharedMemory.cpp, following FreeBSD 11's kern_shm_open (sys/kern/uipc_shm.c):

  • The access mode must be O_RDONLY or O_RDWR. Flags other than O_ACCMODE | O_CREAT | O_EXCL | O_TRUNC | O_CLOEXEC are rejected. Both return EINVAL, before the path is looked at.
  • SHM_ANON with O_RDONLY returns EINVAL.
  • SHM_ANON with O_RDWR returns a new descriptor for a zero-length object. Each object is a host anonymous file: a memfd on Linux, and on Windows a delete-on-close temporary file opened with _O_TEMPORARY | _O_SHORT_LIVED. Because the guest descriptor is the host CRT descriptor, the existing ftruncate, read/write, pread/pwrite, fstat and close work on it unchanged.
  • EMFILE, ENFILE and ENOMEM from the host are returned as the guest errors; any other host failure throws. The guest errno is kept on success.
  • A named object throws (NotImplemented_nid_no_patch). No title is known to open one.

mmap is unchanged. It accepts only private anonymous mappings and fails a mapping of a descriptor with EOPNOTSUPP. So os_create_anonymous_file now succeeds, and the caller's MAP_SHARED mapping fails on its error path (it checks for MAP_FAILED) instead of the title failing to load. Shared file mappings are a separate change. docs/dev/TechnicalDebt.md records this, the ignored mode and the named-object gap.

Log

I don't have a console, and I couldn't run the title here: the host is aarch64 and the project needs x86-64. The stop is from @SleglJan's run of the homebrew catalog on main 4b6ab6b in #1578 (comment). Each title was relinked with --registry and run for 8 s on Linux. In the table of first stops, shm_open is the stop for PPSA59322 ccleste, with no open PR. That run reports the stop as app.elf: symbol lookup error: app.elf: undefined symbol: QuJYZ2KVGGQ.

The call sites and arguments come from its unwrapped eboot.bin: the relocation for QuJYZ2KVGGQ, then the call/jmp qword ptr [rip+…] through that slot. The ftruncate (ih4CD9-gghM), close and mmap (BPE9s9vQQXo) slots were resolved the same way.

If someone with the title can post the next stop with this PR applied, I'll add it here.

Tested

aarch64 Debian 13 host. The full project cannot be built here because it needs x86-64.

  • New core/libs/tests/GuestShmOpen.cpp (guest_shm_open) checks:
    • three anonymous objects (Mesa's flags, plain O_RDWR, and O_CREAT | O_EXCL | O_TRUNC | O_CLOEXEC) give distinct descriptors and leave errno unchanged
    • after ftruncate to 16 KiB, data written with pwrite reads back, the last byte is zero and a read at the end returns 0
    • a second object is still empty, so objects are not shared
    • all three close
    • EINVAL for O_RDONLY, O_CREAT alone, O_WRONLY, access mode 3, O_APPEND and O_NONBLOCK, and for bad flags on a named path
    • a valid named open throws
  • It passes natively on aarch64 (Zig 0.17 clang, glibc 2.36), and on x86_64-linux-musl under qemu-user. Both builds use SharedMemory.cpp with host-backed stand-ins for the guest ftruncate/pread/pwrite/close, which the CMake test takes from libkernel.
  • Negative check: without the change the test does not link. Three mutants each make it abort: anonymous O_RDONLY accepted, no flag check, and a named path accepted.
  • SharedMemory.cpp and the test compile for x86_64-linux-gnu and x86_64-windows-gnu with -Wall -Wextra. The only warnings are the existing -Wreturn-type-c-linkage ones from General.hpp. The Windows _wsopen_s path is compiled but not run.
  • python3 tools/check_conventions.py --base origin/main passes.
  • Not run with a title.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Progress report (c963432 → e607b1c)

📉 System libraries: 85.67% (-0.03%, +1 declared functions)

🆕 1 declared as stubs
Library Name
libkernel shm_open_nid_postfix

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 9, 2026
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 10, 2026
@overlordxrz-source

Copy link
Copy Markdown
Contributor Author

Rig results for this PR (Linux, homebrew catalog). Plain main at 7a7eb369 vs main + #1860 (plus our #2028 and #2140) at e908ea61:

Title main with #1860
ccleste (PPSA59322) stops at QuJYZ2KVGGQ (shm_open) gets past it, next stop 1nTKA7pN1jw (utime)
RommPS (PPSA76677) stops at QuJYZ2KVGGQ (shm_open) gets past it, next stop 1nTKA7pN1jw (utime)
vkQuake (PPSA99010) stops at QuJYZ2KVGGQ (shm_open) gets past it, next stop 1nTKA7pN1jw (utime)
./app.elf: symbol lookup error: ./app.elf: undefined symbol: QuJYZ2KVGGQ
./app.elf: symbol lookup error: ./app.elf: undefined symbol: QuJYZ2KVGGQ
./app.elf: symbol lookup error: ./app.elf: undefined symbol: QuJYZ2KVGGQ

No regressions in the other titles between the two builds.

AI-assisted: yes.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 10, 2026
@babyhuey

Copy link
Copy Markdown
Contributor

Checked on current main ee50132 with this PR (0e19deb) merged, Linux (Ubuntu 24.04, GCC 13) and Windows 11 (MinGW-w64 GCC from WinLibs), Release.

The branch conflicts with main only in docs/dev/TechnicalDebt.md: main rewrote the Windows host-pointer import entry right above the new shm_open entry. Keeping main's version of that entry and adding the shm_open line resolves it.

  • guest_shm_open passes 5/5 on both (--repeat until-fail:5); no CRT assertion dialog on Windows.
  • vkQuake (PPSA99010) stops at shm_open on main and gets past it on both:
    • Linux: app.elf: symbol lookup error: app.elf: undefined symbol: 1nTKA7pN1jw (utime)
    • Windows: FAIL: unresolved ELF import 1nTKA7pN1jw (utime) from libSceLibcInternal.prx

@overlordxrz-source

Copy link
Copy Markdown
Contributor Author

Thanks @babyhuey for testing on both platforms. Rebased onto current main. The only conflict was the libkernel source list in CMakeLists.txt (main added OpenFileLimit.cpp, so both are kept). TechnicalDebt.md now keeps main's Windows host-pointer entry with the shm_open line added, as you described. No code changes.

AI-assisted: yes.

@babyhuey babyhuey mentioned this pull request Oct 10, 2026
9 tasks done
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 10, 2026
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the flag validation against FreeBSD stable/11 kern_shm_open (access mode, accepted flag mask, SHM_ANON + O_RDONLY → EINVAL): matches. Built on Linux and guest_shm_open passes; it does not link without the change. Named objects throw, the ignored mode and the missing shared mmap are in TechnicalDebt, and @babyhuey's Linux and Windows runs (test 5/5, vkQuake past shm_open) cover the Windows path. Merging.

@adriadam10
adriadam10 merged commit 473e64b into boykopovar:main Oct 10, 2026
10 checks passed
SleglJan pushed a commit to SleglJan/AnyPS5 that referenced this pull request Oct 10, 2026
…rnel-shm-open

feat(libkernel): implement shm_open for anonymous objects
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants