Repository navigation
feat(libkernel): support deferred cancellation in POSIX accept - #12965
Felipebmaitan wants to merge 171 commits into
Conversation
Keep native signals arriving beyond the receive timeout and bound the retained-owner wait. The same fixture rejects a source variant that incorrectly restarts its deadline after EINTR. AI-assisted: Codex
Read O_NONBLOCK from the native socket for every retained transfer so descriptor aliases and passed sockets do not turn nonblocking I/O into a blocking wait. Keep guest timeout accounting and native query errors intact. AI-assisted: Codex
Exercise real native descriptor aliases switching both blocking directions and a saturated nonblocking send after the guest descriptor closes, independently of SCM_RIGHTS. AI-assisted: Codex
… close the descriptor
…calls on a released thread handle
|
@CnRJay Responding to #12965 (review); @JJenkx this also updates the Windows hang investigation. Head Exact accept-only comparison uses actual ancestor The shared layer uses 64 descriptor ownership shards; final state release erases only the matching entry, with no process-wide mutex/full-map sweep. Per-descriptor admission locking remains necessary to serialize close/replacement and is unmeasured against main in a title scene. Close retries back off from 1 to 16 ms and wake immediately on reader completion notifications. A single interrupt is insufficient across the pre-native-call gap; the deterministic control below demonstrates it. A non-interruptible host call still has no total drain-time bound. Signal-registry retries sleep with the same backoff instead of yielding in a tight registry scan. No speed, CPU-use or title performance improvement is claimed. No paired same-scene trace_digest evidence exists. Main’s receive-buffer validation and Unix SCM_RIGHTS are retained with cancellation. All four macOS empty-iovec guards remain alongside read/write cancellation and synchronous-write handling. Existing deterministic stream-close hooks and the underscore context repair are retained. Windows callback draining now defers once when another I/O owner is active, preventing pop/reinsert cycling through WaitingEntry. A Windows-only injected active-owner drain case checks deferred, exactly-once delivery. This candidate repairs a concrete queue cycle; it has not established the cause or resolution of the reported Windows 22621 sigjump hang. Linux repetitions do not settle it. Accept injects failures before descriptor reservation and after reservation/before map insertion. Tests compare native descriptor counts before accept, at cleanup and at TSD; cancellation and longjmp handoff tests also check disposal before those callbacks. Native Windows execution remains outstanding. TestedFresh Ubuntu 24.04.5 x86-64 container through Apple Silicon translation, GCC 13.3, Release: production libraries and focused executables rebuild; 74/74 accept/descriptor/context/jump/signal/stream/socket CTests pass. The four installation/disposal cases each also pass 20 consecutive repetitions. Both fault injections are exercised: before descriptor reservation returns EMFILE (24), and after reservation/before map insertion returns ENOMEM (12). Cleanup and TSD verify native resource counts against the warmed baseline. Cancellation and longjmp handoff cases check the same disposal order; the peer EOF checks remain. Negative controls use production ownership paths: omit ReleaseNumber after map-insertion failure → install_map_failure fails (CTest total 0.29 s); move RetireIoStack after cleanup → handoff_cancel fails (0.25 s). Both mutations are restored, the production sources are explicitly rebuilt, source hashes match this commit, and the complete 74-test selection passes again. The first reserve-failure fixture run did not actually inject its intended failure, and correctly rejected the successful accepted descriptor at cleanup. The missing test-switch wiring was corrected before this final run; that initial result is not represented as a production leak. Reads passes 50 focused checks, transport 59, writes 397. Native macOS 27.0.1 arm64 standalone write-watch passes 25/25; execute-fault/read-fault each register once and match main exactly. The four macOS vector guards remain with cancellation by source inspection; no full macOS libkernel runtime is claimed. Conventions pass with zero errors and the inherited SSL debt-length warning. The fork's origin/main points to an older main; to avoid checking unrelated upstream changes or modifying shared refs, the command uses an isolated Git context with origin/main pinned to the actual upstream target 66e1f7f. The overlap scan covered 1,848 open PRs and failed closed for uncomparable branches; no clean-overlap or new owner agreement is claimed. The active-owner callback-drain regression is Windows-only and has not run natively here. Its guard prevents a concrete requeue cycle, but the Windows 22621 intermittent sigjump hang is still not claimed explained or fixed. No Linux repetitions are offered as Windows evidence. This PR remains draft. Fresh full-build dispatch for #12965 with select=true was attempted and denied HTTP 403 (admin rights required). Exact-head CI and the maintainer bench remain external gates. No fresh native Windows, title/console, GPU, full public-header audit or performance validation. Immediate-path admission locking and socket-policy costs are unmeasured; no speed claim or paired trace_digest result. Historical title captures and broad test results remain in older conversation comments at their original heads and do not establish title unblock, full playthrough or multiplayer behavior. The shared close retry is retained for correctness, with backoff, rather than asserted unnecessary: the reads negative control acknowledges and drains the first interrupt before entering the native read, then a single Interrupt + predicate wait fails in 2.26 s. Reader completion notifications still release close immediately; non-interruptible native calls have no total drain-time bound. Per-descriptor admission locks remain, and their cost is unmeasured. The callback guard is owned in #12682 and propagated bottom-up. DeliverIoCallbacks previously popped a node, then WaitingEntry could hand it back to a currently active I/O owner and reinsert it at the head, indefinitely. The injected Windows test queues under an active IoPoint and invokes the drain there, then checks that delivery happens exactly once only after ownership ends. It is a concrete source-level cycle and candidate repair; without native execution and a Windows 22621 reproduction it does not close your hang report. Unblocks: none remains correct after the review correction. Main-based Files changed is cumulative; the linked actual-ancestor comparison isolates accept. The PR is draft and its upstream base stays main. #10552 goes first as the maintainer requested; all declared open prerequisites and original published heads remain ancestors. No PR merged or replaced. AI-assisted: yes. |
Progress report (66e1f7f → 93e4d25)📉 System libraries: 26.28% (-0.01%, -2 reverted functions)
|
| Library | Name |
|---|---|
| libkernel | shm_open_nid_postfix |
| libkernel | shm_unlink_nid_postfix |
Changed libraries (implemented exports)
| Library | Before | After | Change |
|---|---|---|---|
| libkernel | 702/751 (93.48%) | 700/751 (93.21%) | -0.27% |
|
Windows CI follow-up at Follow-up on the failed Windows job at transport parent The inherited UDP test reproduces locally before this fix: with assertion diagnostics, its unreadable-source/malformed-address case aborts at line 421 with guest errno 22 instead of 14 (0.14 s test time). Restoring main's source-fault precedence makes the complete UDP test pass. Fresh Ubuntu 24.04.5 x86-64 container through Apple Silicon translation, GCC 13.3, Release: production libraries and relevant executables rebuild; 25/25 focused CTests pass at Conventions pass against an isolated origin/main pinned to upstream 66e1f7f (the fork's main is stale), with the inherited SSL debt-length warning only. Whitespace checks pass. Refreshed overlap scanning remains inconclusive for unmergeable branches, not a clean-overlap claim. New-head Windows results are pending. The failed parent Windows job did pass The transport composition had replaced main's Address-failure return with an unconditional -1, dropping EFAULT precedence for an invalid datagram source. This change restores the general source check and leaves valid-source address errors intact; it adds line/errno diagnostics without weakening the existing assertion. No performance claim, PR merge, base retarget or force push. AI-assisted: yes. |
What
Restores main's datagram source-fault precedence when address validation also fails: an unreadable payload returns EFAULT (14), while a readable payload retains the address error. The transport composition had dropped this check, so GuestUdp's existing malformed-address/unreadable-payload assertion received EINVAL (22). Adds assertion-line/guest-errno diagnostics without changing any test expectation. The owning fix is in #12688 and is propagated through #12700 and #12965.
Head
93e4d25ef55af1d3943bfcace4edd33d6c41bd67; upstream main66e1f7f993c9c1339c2de11264b76bab03218cd2. Adds cancellable accept above writes, retaining accepted sockets across deferred delivery and disposing unpublished results on cancellation or context abandonment. This PR is draft while native Windows validation of the reported 22621 context hang remains outstanding.Exact accept-only comparison uses actual ancestor
c2b279e81e0f11ae4aee2353a80bfba976e26121. Main-based Files changed remains cumulative until prerequisites land. The comparison base for reads includes main, #105523dffebe9ecadfe21e3fc58f934515e168cc5c158, #10817e78214475e96dc2165f5da7bb48399a6c14258e9, #1075592f4276f28e9643fe7e336e1cc40ee02f30277daand #1214903c2195878367e871c28a258cb080f78896cc372. Published topic heads and maintainer commits remain ancestors; no force push, replacement PR or PR merge. #10552-first order is declared as requested, not represented as already merged.The shared layer uses 64 descriptor ownership shards; final state release erases only the matching entry, with no process-wide mutex/full-map sweep. Per-descriptor admission locking remains necessary to serialize close/replacement and is unmeasured against main in a title scene. Close retries back off from 1 to 16 ms and wake immediately on reader completion notifications. A single interrupt is insufficient across the pre-native-call gap; the deterministic control below demonstrates it. A non-interruptible host call still has no total drain-time bound. Signal-registry retries sleep with the same backoff instead of yielding in a tight registry scan. No speed, CPU-use or title performance improvement is claimed. No paired same-scene trace_digest evidence exists.
Main’s receive-buffer validation and Unix SCM_RIGHTS are retained with cancellation. All four macOS empty-iovec guards remain alongside read/write cancellation and synchronous-write handling. Existing deterministic stream-close hooks and the underscore context repair are retained.
Windows callback draining now defers once when another I/O owner is active, preventing pop/reinsert cycling through WaitingEntry. A Windows-only injected active-owner drain case checks deferred, exactly-once delivery. This candidate repairs a concrete queue cycle; it has not established the cause or resolution of the reported Windows 22621 sigjump hang. Linux repetitions do not settle it.
Accept injects failures before descriptor reservation and after reservation/before map insertion. Tests compare native descriptor counts before accept, at cleanup and at TSD; cancellation and longjmp handoff tests also check disposal before those callbacks. Native Windows execution remains outstanding.
Tested
Follow-up on the failed Windows job at transport parent
f0d497242036b6c24620e8d192449fdb3c53ceaf: that Server 2025 / 10.0.26100 run built successfully, then recorded 1,963 passes, 904 explicit skips and one failure (guest_udp). Skipped entries are not passes. The failed assertion had no line diagnostic in that run, so the new native rerun must confirm there is no additional Windows-only failure.The inherited UDP test reproduces locally before this fix: with assertion diagnostics, its unreadable-source/malformed-address case aborts at line 421 with guest errno 22 instead of 14 (0.14 s test time). Restoring main's source-fault precedence makes the complete UDP test pass. Fresh Ubuntu 24.04.5 x86-64 container through Apple Silicon translation, GCC 13.3, Release: production libraries and relevant executables rebuild; 25/25 focused CTests pass at
93e4d25ef55af1d3943bfcace4edd33d6c41bd67. Selection includesguest_udp, socket options/native modes/timeouts/events/port-zero and Unix SCM_RIGHTS. Accept also includes all 15 accept/cancellation/installation/disposal scenarios.Conventions pass against an isolated origin/main pinned to upstream 66e1f7f (the fork's main is stale), with the inherited SSL debt-length warning only. Whitespace checks pass. Refreshed overlap scanning remains inconclusive for unmergeable branches, not a clean-overlap claim. New-head Windows results are pending.
The failed parent Windows job did pass
guest_io_context_deferred_drain,guest_io_descriptor_close_before_nativeand oneguest_io_context_sigjumpsample, each in about 0.02 s. This is native evidence for that parent on Server 2025, not proof that the separate intermittent Windows 22621 hang is fixed. Accept remains draft; maintainer bench and native 22621 validation remain external.The remaining validation paragraphs below describe the earlier published head
d682ea71365098a3a0410cc544b1337ab8064b1band retain its historical limits.Fresh Ubuntu 24.04.5 x86-64 container through Apple Silicon translation, GCC 13.3, Release: production libraries and focused executables rebuild; 74/74 accept/descriptor/context/jump/signal/stream/socket CTests pass. The four installation/disposal cases each also pass 20 consecutive repetitions.
Both fault injections are exercised: before descriptor reservation returns EMFILE (24), and after reservation/before map insertion returns ENOMEM (12). Cleanup and TSD verify native resource counts against the warmed baseline. Cancellation and longjmp handoff cases check the same disposal order; the peer EOF checks remain.
Negative controls use production ownership paths: omit ReleaseNumber after map-insertion failure → install_map_failure fails (CTest total 0.29 s); move RetireIoStack after cleanup → handoff_cancel fails (0.25 s). Both mutations are restored, the production sources are explicitly rebuilt, source hashes match this commit, and the complete 74-test selection passes again. The first reserve-failure fixture run did not actually inject its intended failure, and correctly rejected the successful accepted descriptor at cleanup. The missing test-switch wiring was corrected before this final run; that initial result is not represented as a production leak.
Reads passes 50 focused checks, transport 59, writes 397. Native macOS 27.0.1 arm64 standalone write-watch passes 25/25; execute-fault/read-fault each register once and match main exactly. The four macOS vector guards remain with cancellation by source inspection; no full macOS libkernel runtime is claimed.
Conventions pass with zero errors and the inherited SSL debt-length warning. The fork's origin/main points to an older main; to avoid checking unrelated upstream changes or modifying shared refs, the command uses an isolated Git context with origin/main pinned to the actual upstream target 66e1f7f. The overlap scan covered 1,848 open PRs and failed closed for uncomparable branches; no clean-overlap or new owner agreement is claimed.
The active-owner callback-drain regression is Windows-only and has not run natively here. Its guard prevents a concrete requeue cycle, but the Windows 22621 intermittent sigjump hang is still not claimed explained or fixed. No Linux repetitions are offered as Windows evidence. This PR remains draft. Fresh full-build dispatch for #12965 with select=true was attempted and denied HTTP 403 (admin rights required). Exact-head CI and the maintainer bench remain external gates.
No fresh native Windows, title/console, GPU, full public-header audit or performance validation. Immediate-path admission locking and socket-policy costs are unmeasured; no speed claim or paired trace_digest result. Historical title captures and broad test results remain in older conversation comments at their original heads and do not establish title unblock, full playthrough or multiplayer behavior.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)