Repository navigation
feat(libkernel): cancel blocked POSIX reads safely - #11872
Felipebmaitan wants to merge 46 commits into
Conversation
… close the descriptor
…calls on a released thread handle
|
@Felipebmaitan thanks for the close fix. Close now wakes admitted readers and waits for them to drain, which resolves the earlier hang. Five points from two reads of the current head (21d5a97), none blocking:
|
1 similar comment
|
@Felipebmaitan thanks for the close fix. Close now wakes admitted readers and waits for them to drain, which resolves the earlier hang. Five points from two reads of the current head (21d5a97), none blocking:
|
|
Superseded by #12682 (read cancellation layer, head 3b0d2c6). Original branch, commits, review history and order tracker are preserved. The replacement composes current main and passes fresh MICRO/MACRO review, full production builds, 345 Windows / 405 Linux focused tests, both header checks and Human Fall Flat startup smoke runs. Transport and writes remain separate following layers; nothing is merged. |
What
Add deferred cancellation for POSIX
read,_read,readv,_readv,pread,preadv,recv,recvfrom,recvmsg,_recvfromand_recvmsg. This includes read/readv on guest sockets and the existing accepted native-file, pipe, standard-descriptor, shared-memory, archive and random-device paths. Existing directory/driver/flag rejection and positioned socket/pipe errors remain. SCE read and libSceNet entry points acquire no new POSIX cancellation point.Unix interrupts native calls with an otherwise unused host SIGUSR2. Windows cancels synchronous file/pipe calls or cancels and drains the exact overlapped socket request. Operation buffers, HostWrite and ownership guards end before cleanup/TSD or deferred handlers run. Disabled cancellation remains pending. Successful, zero and partial results preserve their result when cancellation arrives concurrently, following FreeBSD 11.4 libthr's failure-only leave check.
Per-descriptor admission prevents close/replacement/reuse races without hidden public aliases, additional descriptor slots, child inheritance or record-lock release. POSIX close, sceKernelClose and guest-socket close interrupt admitted POSIX operations before waiting for ownership to drain. Wake registrations are independent of cancellation state and thread adoption, and retry until native submission/completion races are resolved. Completed/partial results win; otherwise the read returns EINTR without signal restart. Descriptor replacement and stdio closure may still wait indefinitely. No blanket socket shutdown was added: surviving-alias coverage applies to pipes and GuestSockets-managed sockets; borrowed libSceNet retains its existing shutdown behavior. Windows append handle replacement joins this ownership. FILE locking precedes descriptor exclusion. Asynchronous caught signals wake the native wait and run after these guards release; restart requires unchanged descriptor generations.
Addressing Dean’s close/read comment: close now wakes the admitted native requests before waiting, and keeps descriptor ownership until native requests and HostWrite scopes have drained. Windows pre-call interruption also initializes errno, _doserrno and GetLastError, avoiding stale EAGAIN/EOF remapping. This interrupts/drains the read rather than reproducing the reported console behavior where the read stays blocked after close returns; no console verification is claimed.
Saved-context implementation metadata distinguishes suspended fibres from abandoned frames, including copied records and manually constructed stacks. It uses opaque jump-buffer tail/machine-context spare storage; ABI sizes/offsets, architectural registers/FPU/masks and uc_link/stack/flags remain. This internal format has no console verification and makes no foreign-libc saved-record compatibility claim. POSIX getcontext treats uc_mcontext as opaque.
Milestone 1 of four: reads/receives, writes/sends, accept/connect, multiplexed waits. No game/title unblock, console or performance claim.
Prerequisites: #10817 through e782144 and #10755 through 92f4276. All original commits/authorship are preserved. Main through 228cd64 is composed with them. Original implementation comparison excluding prerequisites (before this close fix): cf6dbf6...e93539c.
Overlap coordination: requests and confirmations. The author of both PRs has selected #11872 before #10856 and #11625; both later PRs now list Depends on: #11872 for merge ordering, with unchanged source heads. Specific order requests have been posted to the authors of #7090, #7135, #8916, #9938, #10552, #11341, #11831 and the new MSG_WAITALL PR #11896. #11896 now confirms #11872 first and declares that dependency; the other external requests remain pending. No combined runtime validation is claimed. The requested receive integration preserves descriptor/HostWrite ownership and cancellation/signal/close wakeups, including post-partial MSG_WAITALL waits. The FILE, caller-stack, Darwin and platform/debt seams are specified in the linked requests. Existing code prerequisites remain #10817 and #10755. Reverse-order requests will need an explicit agreed dependency before merge. Inconclusive bot comparisons remain unresolved. The recvfrom warning is the moved existing wrapper, not a duplicate export; the public-header audit passes.
Tested
Current head
8c68efe40774567be1ea5801f330c0351727f518, Windows 11/WinLibs GCC 15.2 and Linux/WSL imav-tao/GCC 13:libs,relinkerand registered focused test target builds pass on both platforms.git diff --checkandpython3 tools/check_conventions.py --base origin/mainpass with no convention warnings. The overlap report from publication remains below; subsequent coordination is tracked in the linked comment.The standalone I/O signal ownership repair is eebaac5b24, within the read prerequisite. It acquires owning host TLS before publishing signal state, and teardown looks up existing I/O ownership without allocating an FLS slot/value. The empty-I/O and restart-only FLS-exhaustion controls fail before their fixes (0.45 s and 0.35 s) and pass afterward. New coverage includes 96 ordinary/raw native threads and guest return/exit cleanup, TSD and join ordering. This removes allocation introduced by I/O retirement; it makes no claim about unrelated existing libc-internal TLS allocation.
Launching Human Fall Flat PPSA02429 v01.017.000 found a second integration defect: patched libc renamed the descriptor class's C++ methods while consumers imported the original names. Existing
patched_librariesfails on the preceding production (Windows 0.20 s), and title startup fails on both OSes. 4527224683 uses inline class wrappers over C-linkage_nid_no_patchhelpers, retaining the descriptor registry and operation TLS inside libc. Patched-library loading and title startup now pass that boundary. The title reaches rendering/save setup and actual blocking UDP receives; the transport comparison belongs to #11969's follow-up evidence. No title unblock, console, TCP title or performance claim is made.Fresh independent read-only MICRO and MACRO pairs inspected both exact repair candidates and their production callers. No confirmed defect remained in either final candidate. A further fresh MICRO/MACRO pair checked the 8c68efe merge of main's signal-pointer fixes and found no confirmed integration defect. Earlier Windows address-waking fixture flakiness remains documented in #11969; this head's full serial focused run passes.
Coverage: real blocked pipes and loopback sockets; ordinary/zero/partial completion, cancellation, disabled/pending states, 32-round completion races, scalar/vector/positioned APIs, WAITALL progress, timeout/nonblocking/DONTWAIT, zero-total recvmsg, exactly-once cleanup/TSD, dup2/close/reuse, descriptor capacity and alias visibility, child inheritance, cross-process record locks, borrowed libSceNet closure, FILE order, Windows positioned/vector serialization, append replacement, protected stdin, invalid-source dup2, registry bounds, native/raised signals, native masks, context copies/non-LIFO suspension/nonlocal exits/manual stacks and adopted threads. Exception scenarios are split into bounded cases while preserving their address-wakeup iteration coverage. New cases have 15-second CTest limits; split exception cases have 25-second limits.
Additional close coverage: pipe read/readv, socket read/readv/recv/recvmsg, partial MSG_WAITALL, disabled pending cancellation and exactly-once cleanup/TSD, unregistered callers, multiple readers, direct guest-socket close, mixed sceKernelClose and POSIX reads, surviving managed aliases and reuse, borrowed-socket POSIX close, and deterministic Windows pre-call native error slots.
Negative controls against actual production paths:
Exact preceding production (main plus preserved prerequisites,
24c755f1c8) fails all three selected new pipe/read, socket/recv and borrowed-socket POSIX-close regressions; all pass after the fix. The original production sources were restored byte-for-byte after this control.Initial close candidate
0403250fails the two mixed sceKernelClose regressions and Windows stale native-error regression; all pass after the review fixes.Removing native file/socket wakeups makes both blocked-cancellation cases fail because external I/O is required.
Reinstating public-alias retention fails capacity and alias-visibility cases.
The older signal implementation fails Windows/Linux handler-close cases; numeric/bounds retirement fails fibre/mask cases.
Reinstating the POSIX stack registry fails context-churn RSS; removing early dup2 validation fails the invalid-source/blocked-target case in about two seconds.
An exact prerequisite production build also reproduces the old Linux socket fixture's assumption that reset become visible within 100 immediate sends. That fixture now waits for socket error readiness with a finite deadline; production send behavior is unchanged.
Independent fresh MICRO and MACRO reviews inspected each immutable candidate and production integration. Confirmed fixes include alias close/reuse and record-lock loss, EMFILE/inheritance, global/FILE lock inversions, pipe-vector interleaving, registry growth, protected-close rejection, zero-total recvmsg, append handle replacement, handler self-close, suspended-stack retirement, non-LIFO delivery and invalid dup2 source admission. The adopted-thread deletion concern prompted an idempotent retirement guard and exit coverage, although the proposed baseline crash was not reproduced. A later errno/restart finding was dismissed after source and runtime investigation: __error aliases host errno, and delivery already saves/restores it. All 4 Windows/13 Linux errno-changing-handler cases pass without a production change.
Fresh MICRO and MACRO reviews of
fdd4ba317c3b62dedbcf5b941dae8914030932b7against24c755f1c8report no confirmed correctness findings. Both independently inspected immutable production code. Their preceding review found the mixed sceKernelClose/POSIX-read deadlock and stale Windows pre-call native error slots; both were reproduced, fixed and re-reviewed by two new tasks. The only later change is the concise debt wording inbcf5df1c7756b848fce3c3baf16fe1eb07d36e96. Focused close-fix comparison: 24c755f...bcf5df1.Limits: no PS5, title, macOS, Emscripten or ARM64 run; no general asynchronous cancellation; Unix SIGUSR2 must be free; native drivers without interruption support can delay cancellation and close indefinitely; stdio/replacement can still wait indefinitely; Windows blocking MSG_PEEK|MSG_WAITALL remains unsupported; existing recvmsg output-copy/SCM_RIGHTS gaps remain in debt. WSL used the existing memory-comparison fallback because userfaultfd write protection is unavailable.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)Validation log excerpts
Overlap report
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
main)core/libs/prx/libkernel/Socket/src/Udp.cppL1717-1719docs/dev/debt/libkernel.mdL206-208docs/dev/debt/libkernel.mdL206core/libs/prx/libkernel/Pthread/src/Thread.cppL762-763core/libs/prx/libkernel/File/src/Stdio.cppL1561-1562, L1593-1594docs/dev/debt/libkernel.mdL187core/libs/prx/libc/include/FileStream.hppL7, L63-65, L150, L163-165, L205-206, L337-338, L340-341, L343core/libs/prx/libkernel/Socket/src/Udp.cppL772-773, L790-798, L804-806, L1395-1401, L1551-1557, L1717-1727docs/dev/debt/libkernel.mdL233-234Exports introduced here that are already present in
main:recvfromSome comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
core/libs/prx/libkernel/Pthread/src/Thread.cppcore/libs/prx/libkernel/System/src/Signals.cppcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/Pthread/include/Pthread.hppcore/libs/prx/libkernel/Pthread/src/Thread.cppcore/libs/prx/libkernel/System/src/Exception.cppcore/libs/tests/GuestRaiseException.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libc/CMakeLists.txtcore/libs/prx/libc/include/FileStream.hppcore/libs/prx/libc/src/JumpBuffer.cppcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/Open.cppcore/libs/prx/libkernel/File/src/Stdio.cppcore/libs/prx/libkernel/Pthread/include/Pthread.hppcore/libs/prx/libkernel/Pthread/src/Thread.cppcore/libs/prx/libkernel/Socket/src/Udp.cppcore/libs/prx/libkernel/System/src/Exception.cppcore/libs/prx/libkernel/System/src/SignalSets.cppdocs/dev/debt/libc.mddocs/dev/debt/libkernel.mdcore/libs/prx/libSceNet/Export.cppcore/libs/prx/libc/CMakeLists.txtdocs/dev/debt/libkernel.mdcore/libs/prx/libc/CMakeLists.txtdocs/dev/debt/libc.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/Open.cppcore/libs/prx/libkernel/File/src/Stdio.cppcore/libs/tests/GuestKernelAppArchive.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libc/CMakeLists.txtcore/libs/prx/libc/src/FileAndHeap.cppdocs/dev/debt/libc.mdcore/libs/prx/libkernel/File/include/File.hppcore/libs/prx/libkernel/Socket/src/Udp.cppdocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/File/src/DescriptorControl.cppcore/libs/prx/libkernel/File/src/Open.cppcore/libs/prx/libkernel/File/src/Stdio.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/Pthread/include/Pthread.hppcore/libs/prx/libkernel/Pthread/src/Thread.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/System/src/Exception.cppdocs/dev/debt/libkernel.mddocs/dev/debt/libc.mdcore/libs/prx/libc/CMakeLists.txtcore/libs/prx/libc/include/FileStream.hppcore/libs/prx/libc/src/FileAndHeap.cppcore/libs/prx/libkernel/File/src/Stdio.cppdocs/dev/debt/libc.mddocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mddocs/dev/debt/libc.mdcore/libs/prx/libc/CMakeLists.txtdocs/dev/debt/libc.mdcore/libs/prx/libkernel/Pthread/include/Pthread.hppcore/libs/prx/libkernel/Pthread/src/Thread.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libc/CMakeLists.txtdocs/dev/debt/libc.mdcore/libs/prx/libc/include/Api.hppdocs/dev/debt/libc.mdcore/libs/prx/libkernel/Pthread/include/Pthread.hppcore/libs/prx/libkernel/Pthread/src/Thread.cppcore/libs/prx/libkernel/System/src/Exception.cppcore/libs/prx/libkernel/Socket/src/Udp.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libc/CMakeLists.txtcore/libs/prx/libc/include/Api.hppdocs/dev/debt/libc.mdcore/libs/prx/libkernel/CMakeLists.txtdocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/Stdio.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/Stdio.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/Stdio.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/Stdio.cppdocs/dev/debt/libkernel.mddocs/dev/debt/libc.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/System/src/Signals.cppdocs/dev/debt/libkernel.mddocs/dev/debt/libc.mdcore/libs/prx/libkernel/File/src/Open.cppcore/libs/prx/libkernel/File/src/Stdio.cppcore/libs/prx/libkernel/CMakeLists.txtdocs/dev/debt/libkernel.mdcore/libs/prx/libc/src/FileAndHeap.cppdocs/dev/debt/libc.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/DescriptorControl.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/DescriptorControl.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/CMakeLists.txtdocs/dev/debt/libkernel.mdcore/libs/prx/libSceNet/Export.cppcore/libs/prx/libkernel/Socket/src/Udp.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/Socket/src/Udp.cppcore/libs/prx/libc/CMakeLists.txtcore/libs/prx/libc/CMakeLists.txtdocs/dev/debt/libc.mdcore/libs/prx/libSceNet/Export.cppcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/File/src/Open.cppdocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/File/src/Open.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libc/CMakeLists.txtcore/libs/prx/libc/include/Api.hppcore/libs/prx/libc/include/FileStream.hppcore/libs/prx/libc/src/FileAndHeap.cppdocs/dev/debt/libc.mdcore/libs/prx/libc/src/FileAndHeap.cppdocs/dev/debt/libc.mddocs/dev/debt/libkernel.mdcore/libs/prx/libSceNet/Export.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libc/src/FileAndHeap.cppdocs/dev/debt/libc.mddocs/dev/debt/libkernel.mdcore/libs/prx/libc/include/Api.hppcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/System/src/Signals.cppdocs/dev/debt/libc.mddocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/CMakeLists.txtcore/libs/prx/libkernel/System/src/Signals.cppdocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mdcore/libs/prx/libSceNet/Export.cppdocs/dev/debt/libkernel.mdcore/libs/prx/libkernel/Socket/src/Udp.cppdocs/dev/debt/libkernel.mddocs/dev/debt/libkernel.mddocs/dev/debt/libc.mdHuman Fall Flat read/transport/write startup socket evidence on Windows and Linux, including exact heads, call counts and limitations: #11969 (comment). No full-playthrough or title-unblock claim.