Skip to content

feat(libkernel): support macOS pipe write readiness - #13560

Open
SHOnnay wants to merge 1 commit into
boykopovar:mainfrom
SHOnnay:codex/feat-macos-pipe-write
Open

SHOnnay wants to merge 1 commit into
boykopovar:mainfrom
SHOnnay:codex/feat-macos-pipe-write

Conversation

@SHOnnay

@SHOnnay SHOnnay commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

What

On macOS, GuestSockets::PipeWriteState threw for the write end of an anonymous pipe, so guest pipe-write events could not query readiness. The macOS branch now uses a private host kqueue with EVFILT_WRITE, a zero timeout and close-on-exec. It reports available host-buffer space when writable and zero with EOF when the reader closes. Each snapshot creates a queue, which is closed on success and exception paths; interrupted queries retry and other host errors throw with the host errno.

The Linux and Windows branches of PipeWriteState are byte-for-byte unchanged. Host buffer capacity remains different from FreeBSD's 16 KiB buffer, as recorded in the existing debt entry. Reverse-direction pipe events remain unsupported. Named FIFOs also remain unsupported: on this Mac, the host reports a FIFO writable after its reader closes without EOF, although a write fails with EPIPE. F_GETPATH distinguishes vnode-backed FIFOs from anonymous pipes; XNU returns EBADF for a non-vnode descriptor before attempting path lookup.

Contract sources:

The isolated fixture compiles production Udp.cpp directly, using dead stripping to discard unrelated runtime functions. A dedicated, file-gated CI job builds it on macos-15-intel and macos-15, following the write-watch job's checkout/merge pattern.

Related:

Tested

macOS 27.0.1 (26A434), Apple M1. Native ARM64 and x86-64 under Rosetta were tested. Physical Intel macOS, Linux, Windows, a GPU and full guest titles were not run locally. The isolated fixture does not establish full guest-runtime kevent behavior.

Clean upstream/main baseline (4bc9e779aa7abb2109c65ee3bca19e1b85df3be8):

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_RELINKER_ONLY=ON
cmake --build build
ctest --test-dir build --output-on-failure

Result: 100% tests passed out of 522 (268.00 seconds).

Regression fixture:

cmake -S core/libs/tests/pipe_write -B build/pipe-write -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build/pipe-write
ctest --test-dir build/pipe-write --output-on-failure

With upstream/main's unchanged Udp.cpp, 7 of 9 cases fail (ready, full, threshold, EOF, duplicate, cleanup and descriptor exhaustion); invalid descriptors and the unsupported FIFO boundary pass. With the fix, 9/9 pass. The threshold case drains 511 bytes from a full pipe (not writable), then one more (writable, data 512), then another 512 (data 1024). Other cases cover repeated level snapshots, empty/full reader closure, duplicated descriptors, linked/unlinked FIFOs, descriptor leaks, and recovery after EMFILE.

ctest --test-dir build/pipe-write --output-on-failure --repeat until-fail:20
cmake -S core/libs/tests/pipe_write -B build/pipe-write-x86 -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_OSX_ARCHITECTURES=x86_64
cmake --build build/pipe-write-x86
ctest --test-dir build/pipe-write-x86 --output-on-failure
ctest --test-dir build/pipe-write-x86 --output-on-failure --repeat until-fail:20

Result: 180/180 successful executions on ARM64; 9/9 and 180/180 under Rosetta.

cmake --build build
ctest --test-dir build -j4 --output-on-failure
python3 tools/tests/test_build_eligibility.py
python3 tools/check_conventions.py --base upstream/main
actionlint -oneline .github/workflows/build.yml

Results: relinker-only 522/522 (72.21 seconds); eligibility 10/10; conventions zero errors with the existing debt-length warning on the edited pipe entry (base 1230 characters, branch 1329); actionlint no new findings compared with upstream/main. Additional harness checks verified the new job's affected/unaffected paths for PR and manual runs, with full=true and full=false. The new output mirrors write-watch selection on push runs.

Overlap check: python3 tools/pr_overlap.py --branch codex/feat-macos-pipe-write checked 1597 open PRs. It reported 42 uncompared PRs because those branches conflict with main, ending with overlap check failed closed; manually compare every reported PR branch before opening this one. This is not a clean automated result. Every reported shared-file diff was inspected from its true merge base. None implements this macOS readiness helper; the shared Udp.cpp heads retain the macOS throw or predate the helper. None changes this PR's pipe-debt bullet. #12741 adds distinct edgeWatch/edge-watch workflow settings while this PR adds pipeWrite/pipe-write; keeping both is an integration/rebase concern, not a build dependency. Other shared debt-file topics are independent.

Uncompared PRs: #4630, #5021, #6936, #7088, #7105, #7121, #7135, #7149, #7192, #7447, #7464, #7509, #7804, #8605, #9156, #9242, #9293, #9297, #9302, #9326, #9389, #9557, #9566, #9843, #9938, #10025, #10442, #10552, #10645, #11215, #11628, #11661, #11689, #11698, #11939, #12682, #12688, #12700, #12741, #12925, #12965, #12970.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes (Codex)

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 13997 .github/workflows/build.yml L141

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4630 docs/dev/debt/libkernel.md
PR 5021 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 6936 docs/dev/debt/libkernel.md
PR 7088 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 7105 docs/dev/debt/libkernel.md
PR 7121 docs/dev/debt/libkernel.md
PR 7135 docs/dev/debt/libkernel.md
PR 7149 docs/dev/debt/libkernel.md
PR 7192 docs/dev/debt/libkernel.md
PR 7447 docs/dev/debt/libkernel.md
PR 7464 docs/dev/debt/libkernel.md
PR 7507 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 7509 docs/dev/debt/libkernel.md
PR 7804 docs/dev/debt/libkernel.md
PR 8605 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 9156 docs/dev/debt/libkernel.md
PR 9199 docs/dev/debt/libkernel.md
PR 9242 docs/dev/debt/libkernel.md
PR 9293 docs/dev/debt/libkernel.md
PR 9297 docs/dev/debt/libkernel.md
PR 9302 docs/dev/debt/libkernel.md
PR 9326 docs/dev/debt/libkernel.md
PR 9389 docs/dev/debt/libkernel.md
PR 9557 docs/dev/debt/libkernel.md
PR 9566 docs/dev/debt/libkernel.md
PR 9843 docs/dev/debt/libkernel.md
PR 9938 docs/dev/debt/libkernel.md
PR 10025 docs/dev/debt/libkernel.md
PR 10319 docs/dev/debt/libkernel.md
PR 10442 docs/dev/debt/libkernel.md
PR 10645 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 11002 docs/dev/debt/libkernel.md
PR 11215 docs/dev/debt/libkernel.md
PR 11624 docs/dev/debt/libkernel.md
PR 11625 docs/dev/debt/libkernel.md
PR 11628 docs/dev/debt/libkernel.md
PR 11661 docs/dev/debt/libkernel.md
PR 11689 docs/dev/debt/libkernel.md
PR 11698 docs/dev/debt/libkernel.md
PR 11896 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 11939 docs/dev/debt/libkernel.md
PR 12560 docs/dev/debt/libkernel.md
PR 12682 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 12688 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 12700 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 12826 docs/dev/debt/libkernel.md
PR 12925 docs/dev/debt/libkernel.md
PR 12965 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 14108 docs/dev/debt/libkernel.md
PR 14111 docs/dev/debt/libkernel.md
PR 14128 docs/dev/debt/libkernel.md
PR 14212 docs/dev/debt/libkernel.md

@adriadam10 adriadam10 self-assigned this Oct 31, 2026
@SHOnnay

SHOnnay commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Both pipe-write jobs pass (macos-15 and macos-15-intel). The write-watch jobs (macos-15, macos-15-intel) fail at configure on main as well, not from this change: core/libs/tests/write_watch/CMakeLists.txt:42 registers write_watch_execute-fault and write_watch_read-fault a second time ("add_test given test NAME ... already exists"). #13220 removes the duplicate loop and is still open.

@imellag

imellag commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

A run on Apple Silicon with the x86-64 build under Rosetta 2, which is how AnyPS5 runs on these Macs (the CI jobs cover arm64 and Intel natively).

macOS 27.2 (Apple M4 Pro), Apple clang 21, head a4d86e641c applied on main 21a556f567 plus the support commits needed to build on macOS:

  • The full build of libkernel with this change compiles and links (the full macOS build is not in CI).
  • pipe_write fixture built with -DCMAKE_OSX_ARCHITECTURES=x86_64: all 9 cases pass with --repeat until-fail:3.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants