Skip to content

feat(libkernel): cancel blocked POSIX writes safely - #11969

Closed
Felipebmaitan wants to merge 66 commits into
boykopovar:mainfrom
Felipebmaitan:feat/posix-io-cancellation-writes
Closed

Felipebmaitan wants to merge 66 commits into
boykopovar:mainfrom
Felipebmaitan:feat/posix-io-cancellation-writes

Conversation

@Felipebmaitan

@Felipebmaitan Felipebmaitan commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What

Implement milestone 2: a blocked POSIX write can be cancelled while retaining descriptor ownership until native operations drain. Completed positive counts, including partial counts, win; zero/error returns check pending cancellation after guards release. Disabled cancellation stays pending, and cleanup/TSD run exactly once after operation ownership ends.

APIs: write, _write, writev, _writev, pwrite, pwritev, send, sendto, sendmsg, _sendto, _sendmsg. Paths include ordinary files, shared-memory objects, pipes, stream and datagram sockets. Positioned pipe/socket writes retain their existing errors. SCE and libSceNet gain no cancellation points. Accept/connect and multiplexed waits remain later milestones.

The write layer reuses #11872's descriptor admission, signals and cancellation/close machinery. Native transports preserve known partial progress; Windows pipe vector transfers preserve small-write atomicity. Linux preflight metadata and synchronous flushes now participate in native interruption. An interrupted two-phase O_FSYNC write can leave buffered bytes visible without durable completion; it reports success only after flushing succeeds and never replays those bytes after interrupted flushing. Direct SCE flush behavior is preserved. Restricted-right Windows pipe handles fall back to their existing native write capability when attribute queries are denied.

Stack: read 8c68efe; transport 3c360d5; write dc7772e. Includes pinned main 228cd64 and declared read prerequisites #10817/#10755 with original commits/authorship preserved. The transport branch preserves all six exact #10943 candidate commits through 6adaa5a. The standalone owning-TLS repair eebaac5 and descriptor export repair 4527224 belong to the read prerequisite.

Transport-only comparison; write-only comparison (17 files, seven commits after transport).

Publication constraint: CONTRIBUTING allows 20 open PRs while an author has 1-19 merged; this author has 20 open/17 merged. The separate transport branch is published but cannot have an upstream PR yet. This account lacks upstream push access and upstream has no read/transport base branch. This PR still targets main, so its Files changed includes prerequisites; the linked incremental comparison is write-only. A maintainer-provided upstream base or merged prerequisites is needed for a write-only upstream Files changed view.

Overlap coordination: #11872 tracker explicitly covers #11896, #10645, #10552 and #11625. #11896 confirms reads first; external #10645/#10552 replies remain pending. #11625 source head is preserved. Write/carrier seam requests remain linked for #8463, #9019, #10645 and #10943. No agreement or combined runtime validation of those external PRs is claimed. The overlap report includes inconclusive comparisons for older base-conflicting PRs. Its live base d13b3d7 has no libkernel changes after the pinned main; its libc drift is an unrelated 15-line vector-algorithm addition. recvfrom/sendto are moved existing wrappers, not new duplicate exports (the public-header audit verifies ownership). The report's #11969 row compares this PR's previous head before publication, not another implementation.

Tested

Current-main refresh (2026-10-28)

Merged upstream main 957a22d51f02ccacff218cdf9a27c36af7cc9aeb without rewriting published commits; resulting head 560f498196e7db59ef8b4f41463105dadb96e15a. The refreshed #11872 head 21d5a977f9 is an ancestor. Code merged automatically; the manual debt resolution retains main's #8916 sleep-cancellation behavior and this stack's read/write/flush/signal limitations. Original transport commits and authorship remain intact.

Linux x86-64 Ubuntu 24.04 container translated on Apple Silicon, GCC 13.3, Release, bundled dependencies built from source: production libc/libkernel and focused test targets build; 536/536 tests pass, zero skips. Selection consists of all registrations using guest_io_write_cancel_tests, guest_io_write_sync_cancel_tests, guest_io_read_cancel_tests, guest_io_descriptor_access_tests and guest_pthread_cancel_tests, run with ctest --test-dir /tmp/anyps5-build -R <exact selected names> --output-on-failure -j 4. This is translated-container evidence, not native Linux/Windows or console evidence; no full-suite or fresh title run. Signal/context tests are outside this selection.

Public-header audit: 368 libraries / 12,990 exports, unique declarations and umbrella coverage verified. Whitespace and conventions against actual upstream main pass (one inherited SSL debt-length warning). The required --base origin/main check also ran, but the fork's origin/main is still 0c6a5bd5938f; it reports upstream main's pre-existing non-Conventional revert subject 97f1d41a2c, rather than an error introduced by this refresh. The live tools/pr_overlap.py --branch refresh/posix-io-cancellation-writes --repo boykopovar/AnyPS5 --jobs 4 scan completed; existing unresolved overlaps/order and transport-publication constraints remain, with no new agreement claimed. Earlier evidence below remains attributed to its original revisions.

Windows 11 / WinLibs GCC 15.2 Release; Linux/WSL2 imav-tao / GCC 13.3 Release. Full production libs, relinker and focused-target builds pass on both systems.

  • 668/668 Windows and 712/712 Linux focused runtime tests pass, including patched-library loading. Public headers: Windows 65/65; Linux audit verifies 368 libraries / 12,811 exports, plus 64/64 compiler batches.
  • Real full pipes and loopback sockets cover backpressure, scalar/vector/positioned APIs, partial-progress byte accounting through EOF, peer drain/close, cancellation/completion races, disabled requests, aliases, close/reuse, cleanup/TSD, empty datagrams, append/shared offsets and admission locks. Existing reads, signals/contexts/nonlocal exits, borrowed libSceNet descriptors, AIO, SSL and transport mode/timeout tests are included.
  • Synchronous-write cancellation/zero-progress negative controls fail before their fixes on both OSes. A truly restricted Windows pipe duplicate reproduces guest EIO before the fallback repair; ordinary CreateFile handles silently retained attributes access and did not reproduce it. Two-byte/one-free-byte write/writev regressions return EAGAIN with unchanged queued bytes; the suggested partial-write failure was not reproduced on this backend.
  • 40/40 Linux preflight cases cover both metadata queries across write/writev/pwrite/pwritev, cancellation, disabled cancellation, completion, close and signals. Real FUSE GETATTR held before ordinary write requires external release at e87a337 (zero bytes, no FUSE interrupt); the final head issues FUSE_INTERRUPT, cancels promptly and leaves zero bytes. Holding post-write synchronous classification also cancels promptly without external release and retains two buffered bytes. The private probe calls built production libraries; host libstdc++ preloading avoids unpatched libc guest-unwinder interposition in this host fixture.
  • Human Fall Flat PPSA02429 v01.017.000: same immutable relinked executable/assets, cold startup/no input on read, transport and final write heads, both OSes. Reaches rendering/save/network setup and actual blocking UDP recvfrom (flags 0), sends and title timeout/nonblocking-mode changes. The evidence comment gives all counts and capture limits. This is startup socket compatibility evidence, with no multiplayer/TCP-title, title cancellation/vector-write, complete playthrough, console, performance or title-unblock claim.

Fresh independent read-only MICRO/MACRO pairs reviewed the read repairs, transport composition and successive write candidates. Confirmed synchronous flush, restricted pipe and preflight metadata findings were fixed with regressions. The final pair reviewed dc7772e and found no remaining confirmed defect; both independently ran the 40 preflight cases. Earlier Windows address-waking fixture flakiness reproduces on the prerequisite baseline; this final serial suite passes.

Git diff --check and python3 tools/check_conventions.py --base origin/main pass with zero errors and one inherited SSL debt-length warning. tools/pr_overlap.py completes at the exact final head. No tracked notes/logs, new third-party dependency or source comments were added.

Limits remain in library debt: deferred cancellation only; non-interruptible drivers can delay cancellation/close, descriptor replacement and stdio closure can wait, Windows pipe quota readiness retries use 1 ms, and Windows blocking PEEK|WAITALL remains unsupported. macOS runtime and console behavior are unverified.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: feat(libkernel): cancel blocked POSIX reads safely #11872; transport branch feat/posix-socket-transport (upstream PR blocked by author limit)
  • Fixes: none (the separate transport layer owns libkernel: MSG_DONTWAIT can block on Windows after a readiness check #10943)
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

lgqdaxia and others added 30 commits October 25, 2026 22:47
Keep native signals arriving beyond the receive timeout and bound the retained-owner wait. The same fixture rejects a source variant that incorrectly restarts its deadline after EINTR.

AI-assisted: Codex
Read O_NONBLOCK from the native socket for every retained transfer so descriptor aliases and passed sockets do not turn nonblocking I/O into a blocking wait. Keep guest timeout accounting and native query errors intact.

AI-assisted: Codex
Exercise real native descriptor aliases switching both blocking directions and a saturated nonblocking send after the guest descriptor closes, independently of SCM_RIGHTS.

AI-assisted: Codex
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 28, 2026
@oneandonlydean oneandonlydean self-assigned this Oct 28, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

@Felipebmaitan thanks for the write-only compare link and the stack notes, they made the increment readable. Two questions after reading the write and transport layers:

  1. An interrupted O_FSYNC flush turns a successful write into -1/EINTR. SyncWrites returns false when the flush after the write is interrupted, and WriteSync::Complete then reports -1 with EINTR even though the bytes are already in the file and the offset has moved (GuestIoWriteSyncCancel pins this). IoInterrupted() is also true for a deferred guest signal, not only cancellation, so a guest that retries write on EINTR writes the same bytes twice. That goes against the "completed positive counts win" rule the rest of the PR follows, and FreeBSD returns the count once a synchronous write's data is copied. Could Complete return the positive count when the flush is interrupted (data written but not durable), or finish the flush without interruption?

  2. On Windows, WaitTransferReady builds a new Waker on every POSIX would-block wait: socket, FIONBIO, bind, getsockname, connect and close, plus one ephemeral loopback port, and it throws if setup fails. A guest that does blocking recv/send in a loop pays that on every wait. Could it reuse a per-thread waker, or an event like the earlier read path did?

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@Felipebmaitan thanks for splitting this one up. The write-only increment on top of #10817, #10755 and #11872 reads well. The one piece still missing a PR of its own is the transport step (the #10943 commits this branch carries). Please open it as its own PR even though it takes you over the 20-open-PR limit; that's fine for this one. Once it's open we'll review the transport on its own, and #11969 can stay the write-only increment stacked on it.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 28, 2026
@Queuse

Queuse commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

I am fine with separating the #10943 transport commits into the dedicated PR the maintainer requested, leaving this PR as the write-only increment on top. Please retain the six-commit transport history and the attribution of my Unix SSL correction. My earlier agreement to carry those commits here does not require keeping them bundled after that split. I have no newer standalone correction; the SCM_RIGHTS diagnostic stack remains a separate compatibility step.

@Felipebmaitan

Copy link
Copy Markdown
Contributor Author

Superseded by the smaller reviewed stack: #12682 reads, #12688 native blocking transport, #12700 write cancellation. The write-only comparison is linked in #12700, head 7a2c024. Original branches, commits/authorship and review history remain preserved. The interrupted-flush retry issue is fixed in #12700 (positive counts retained); the per-wait waker concern is fixed in #12688 (per-thread reuse). Final write validation passes 699 Windows / 777 Linux cases, both header checks and fresh MICRO/MACRO review. Nothing is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants