Skip to content

fix(libc): open NUL on standard descriptors without a stream on Windows - #9945

Merged
oneandonlydean merged 8 commits into
boykopovar:mainfrom
p3n1cillin:claude/windows-missing-standard-handles
Oct 24, 2026
Merged

oneandonlydean merged 8 commits into
boykopovar:mainfrom
p3n1cillin:claude/windows-missing-standard-handles

Conversation

@p3n1cillin

@p3n1cillin p3n1cillin commented Oct 22, 2026 •

Copy link
Copy Markdown

What

A Windows process started without a standard handle has descriptors 0 to 2 whose _get_osfhandle is -2. The UCRT documents that value for a standard descriptor with no stream. It happens to a detached process and to a --windows-gui executable started from Explorer. DescriptorIsOpen counts such a descriptor as open, but GetFileType fails on it, so fcntl throws:

terminate called after throwing an instance of 'std::runtime_error'
  what():  fcntl: descriptor 0 has an unknown Windows file type, error=6

Halo: Combat Evolved (PPSA99500, catalog build-113) stops there at start-up. Its PS5 host checks descriptors 0 to 2 with fcntl(fd, F_GETFD) before opening anything, because, as its comment says, a title starts without them.

On Linux and macOS, libc's static initializer already opens /dev/null on a standard descriptor that is closed when libc loads. The libc debt entry said the Windows case was not handled. This PR adds the Windows side, PrepareStandardStreams_nid_no_patch, which FileStream calls before it captures a descriptor:

  • A descriptor 0 to 2 whose handle is -2 or -1 is opened on NUL, read-only for 0 and write-only for 1 and 2.
  • The UCRT stream is then attached to it with _wfreopen, because _dup2 alone leaves the stream's _fileno at -2. A stream bound to the raw descriptor closes itself through _wfreopen, so the descriptor is not closed twice.
  • A stream that is already valid on another descriptor is kept, and the raw descriptor is restored from it rather than reopened.

The stream attachment and the follow-up cases come from @Queuse's branch, cherry-picked with authorship kept, after @harisibrar1 found the stream gap. The debt entry describes both hosts and links the title's check. What the console gives a title as its standard descriptors stays unmeasured. As on Linux, a title here finds the descriptor open where the Halo port expects it closed, and it leaves that descriptor alone.

Tested

Windows 11 Pro (26300), WinLibs GCC 15.2.0, Release, full build on main 2ace103 plus these commits:

  • guest_closed_standard_descriptors_windows starts itself as a detached child without stdin, without stdout, without stderr, and without all three. For each missing descriptor the child checks a character-device handle, the guest fcntl answers for F_GETFD and F_GETFL, and that the native and guest streams use that descriptor. It also checks that writes and flushes on stdout and stderr succeed, and that stdin reads EOF without an error. On main the child exits on the -2 handle. A non-detached child doesn't show the problem, because Windows gives a new console's process console handles in place of null ones.
  • guest_closed_standard_streams_load_windows loads libc into a process whose standard descriptors were closed, and into one whose stdout is already valid on a separate descriptor. It checks that no invalid-parameter callback runs, that the valid stdout is kept, and that native and guest writes succeed.
  • All 360 tests that core/libs/prx/libc and core/libs/prx/libkernel register pass.
  • Halo: Combat Evolved (PPSA99500), started on a separate desktop with a null stdin handle and stdout and stderr redirected to a file. On main it aborts at 0 s with the fcntl error above. With this PR it runs its start-up, with SDL 2.30.12, RADV's vkCreateInstance and glslang. It then quits on its own check for the game files: the game needs /data/halo, with the game's disc image in it. I don't have the disc image, so nothing past that point was run.

WSL Ubuntu 26.04.1, GCC 15.2.0: libc builds, and guest_closed_standard_descriptors and the three guest_standard_streams tests pass. The non-Windows branch of the initializer is unchanged.

tools/check_conventions.py --base origin/main passes. tools/pr_overlap.py --branch reports only #9596, which edits the fgetws debt line next to this entry's line. #8150 and #7201 also touch StandardStreams.cpp, in other functions.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: PPSA99500 (Halo: Combat Evolved) when started without a standard input handle
  • GPUs tested: n/a
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

@github-actions github-actions Bot added the unblocks title Names a title ID on the Unblocks line label Oct 22, 2026
@MarcZX10

Copy link
Copy Markdown
Contributor

I reviewed head 1366ebbe8d9f66ad2bb287b43efa2ee5a3244246; diff-check is clean and the convention checker reports zero errors (one existing libc debt-length warning). The Windows initializer handles UCRT’s -2 no-stream descriptor, opens NUL with direction-appropriate access, and the child-process test checks all missing-handle combinations and guest fcntl flags. The author reports the 353 libc/libkernel tests passing and a title reaching its menu; hosted Windows relinker is still running, while product jobs are skipped. I found no independent defect and did not run the C++ build locally.

@MarcZX10

Copy link
Copy Markdown
Contributor

Status refresh: the Windows relinker job has now passed as well. The Ubuntu, Windows, macOS, ASAN, register-context and write-watch jobs in this matrix are all green; hosted product Linux/Windows builds remain skipped.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4563 core/libs/prx/libkernel/CMakeLists.txt
PR 5021 core/libs/prx/libc/include/FileStream.hpp
core/libs/prx/libc/src/StandardStreams.cpp
core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libc.md
PR 5303 docs/dev/debt/libc.md
PR 6044 core/libs/prx/libkernel/CMakeLists.txt
PR 6830 core/libs/prx/libkernel/CMakeLists.txt
PR 6936 core/libs/prx/libkernel/CMakeLists.txt
PR 7090 core/libs/prx/libkernel/CMakeLists.txt
PR 7201 core/libs/prx/libc/include/FileStream.hpp
core/libs/prx/libc/src/StandardStreams.cpp
core/libs/prx/libkernel/CMakeLists.txt
PR 7544 docs/dev/debt/libc.md
PR 7648 docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 core/libs/prx/libc/src/StandardStreams.cpp
docs/dev/debt/libc.md
PR 8417 docs/dev/debt/libc.md
PR 8965 docs/dev/debt/libc.md
PR 9187 core/libs/prx/libkernel/CMakeLists.txt
PR 9232 docs/dev/debt/libc.md
PR 9284 core/libs/prx/libkernel/CMakeLists.txt
PR 9293 core/libs/prx/libkernel/CMakeLists.txt
PR 9297 core/libs/prx/libkernel/CMakeLists.txt
PR 9302 core/libs/prx/libkernel/CMakeLists.txt
PR 9360 docs/dev/debt/libc.md
PR 9408 docs/dev/debt/libc.md
PR 9447 docs/dev/debt/libc.md
PR 9526 docs/dev/debt/libc.md
PR 9527 docs/dev/debt/libc.md
PR 9552 docs/dev/debt/libc.md
PR 9557 core/libs/prx/libkernel/CMakeLists.txt
PR 9561 core/libs/prx/libkernel/CMakeLists.txt
PR 9566 core/libs/prx/libkernel/CMakeLists.txt
PR 9575 core/libs/prx/libkernel/CMakeLists.txt
PR 9669 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libc.md
PR 9693 core/libs/prx/libkernel/CMakeLists.txt
PR 9833 docs/dev/debt/libc.md
PR 9863 docs/dev/debt/libc.md
PR 9873 docs/dev/debt/libc.md
PR 9929 docs/dev/debt/libc.md
PR 9947 core/libs/prx/libkernel/CMakeLists.txt
PR 10007 docs/dev/debt/libc.md
PR 10453 docs/dev/debt/libc.md

@p3n1cillin

Copy link
Copy Markdown
Author

Thanks @MarcZX10. One correction to the summary: Halo doesn't reach a menu here. With this PR it runs its start-up (SDL, RADV's vkCreateInstance, glslang) and then quits on its own check for the game files, which I don't have. On main it aborts before any of that, on the fcntl error.

AI-assisted: yes (Claude Code)

@klosixgaming-arch

Copy link
Copy Markdown
Contributor

Tested this change on Windows 11 with an NVIDIA GeForce RTX 5090 (driver 617.42) on AMD Ryzen 9 9950X3D.

Test Results

  • Clean merge on current origin/main (b1496678ac1052416fc22e4b10b43ebf88560c14).
  • Built with MinGW-w64 GCC 15.2.0 (Release).
  • Negative test verification: On origin/main without StandardStreams.cpp patched, guest_closed_standard_descriptors_windows fails with:
    missing standard handles, stdin: child exited with 70
    0% tests passed, 1 tests failed out of 1
    1570 - guest_closed_standard_descriptors_windows (Failed)
    
  • Positive test verification: With this PR applied, guest_closed_standard_descriptors_windows passes cleanly (100% tests passed, 0 tests failed in 0.18s).
  • Regression suite:
    • guest_host_standard_error_windows: Passed (0.12s)
    • guest_kqueue: Passed (3.50s)
    • windows_mappings_release: Passed (0.56s)
    • windows_exceptions: Passed (1.13s)
    • windows_backtrace_report: Passed (0.28s)
      All 5 tests passed cleanly with no regressions.
  • Conventions check (tools/check_conventions.py): 0 errors, 1 pre-existing debt-length warning.

Code review of the diff looks clean, properly handles UCRT's -2 no-stream handle through _set_thread_local_invalid_parameter_handler and _get_osfhandle, opens NUL with the correct direction flags, and mirrors the Linux/macOS static initializer contract.

AI-assisted: yes. Written with AI assistance and reviewed by me.

@harisibrar1 harisibrar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked against main 3875f8c and the UCRT _get_osfhandle docs, plus a standalone probe I compiled with WinLibs GCC 15.2.0 (UCRT):

  • The approach matches main's Linux/macOS initializer (StandardStreams.cpp on main only has the #ifndef _WIN32 branch), and it still merges cleanly. Thanks for the detached-child test.
  • I ran a standalone exe (not the libc DLL) that runs your initializer in a static initializer and starts itself DETACHED_PROCESS with null std handles. After the _dup2, _get_osfhandle(0..2) are real handles and _write(1) succeeds. But _fileno(stdin/stdout/stderr) stays -2: fputs(stderr) returns -1 with errno 9, fflush(stdout) returns -1 with errno 9, and fgetc(stdin) returns -1 with ferror set.
  • Main's FileStream takes its guest descriptor from _fileno (FileStream.hpp:78-82), so _Stdout/_Stderr/input would keep descriptor -1, and guest stdio on them would still give EBADF, where on Linux the same streams write to /dev/null. Could the debt entry say this, or could the streams be re-attached? A plain freopen("NUL", ...) landed on fds 3/4/5 in my probe, so it doesn't solve it as-is.
  • Optional: a check in the child that fputs to stdout succeeds would cover this.

Reviewed with AI assistance (Claude Code); the checks above were run locally on my machine.

@Queuse

Queuse commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

@p3n1cillin @harisibrar1 I reproduced the stream attachment and initialization-order problems with the actual DLL. The expanded test catches native stdin retaining -2 and the guest stdout prefix caching -1 before guest fileno can refresh it.

I found two more cases while checking the fix: closing an already closed descriptor again invokes the UCRT invalid-parameter handler, and reopening an already valid FILE on NUL destroys its existing destination. The latter matters for the preloaded-CRT state handled by #10157, where stdout can be valid on fd 3 while raw fd 1 is still -2.

The updated follow-up branch handles all three states: unbound streams, bound streams whose raw descriptors were closed, and valid streams on separate descriptors. It initializes before FileStream captures descriptors, lets bound streams close themselves through _wfreopen, and restores missing raw descriptors from an existing valid stream without reopening that stream.

Cherry-pick order, including the earlier two commits:

git cherry-pick 06f41e44d5 81f2bd1d7c 49f4682994 385b4f7136 245043ff17 dfb7e98909

The detached loader test fails on the previous version when stdout's descriptor changes, and passes with the follow-up. It also checks zero invalid-parameter callbacks and successful native/guest writes. DLL paths with spaces and Unicode pass too.

Tested together with #10020 on main 85f2f2d478: 10/10 Windows (Windows 11, WinLibs GCC 15.2.0-r7) and 8/8 Linux/WSL2 (Ubuntu 26.04.1, GCC 15.2.0), no skips. Combined tree. Conventions and whitespace checks pass; trial integration with consulted main d1fc16348e is clean. I haven't run Halo or #10157's relinker test. This leaves the fgetwc conversion work in #8897 unchanged.
The refreshed overlap check also flags #9596: a pairwise merge conflicts only in docs/dev/debt/libc.md, where its wide-I/O entry and this closed-standard-descriptor entry both need to be retained. There is no code dependency between them. Another 40 comparisons are blocked by those PRs' conflicts with main; I checked the affected StandardStreams.cpp/FileStream.hpp changes in #5021, #7201 and #8150, and they do not implement this initialization fix.

@oneandonlydean oneandonlydean self-assigned this Oct 23, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

@p3n1cillin The descriptor fallback still leaves a stream-initialization gap: as @harisibrar1 and @Queuse reported, the UCRT streams retain _fileno == -2 after _dup2, and _Stdout/_Stderr capture -1 during static initialization. Please take the fixes in 06f41e44d5 and 81f2bd1d7c, or an equivalent that attaches the streams before FileStream captures them. Extend the child test to check successful guest/native stdout and stderr writes and flushes, plus stdin EOF without an error; rerun the Windows and Linux closed-standard-descriptor tests.

@MarcZX10

Copy link
Copy Markdown
Contributor

The refreshed overlap scan reports nearby StandardStreams.cpp and libc debt hunks. My proposed #8897 follow-up changes fgetwc conversion/state, while #9945 addresses standard-handle initialization. Since #9945 is still being revised for the UCRT stream attachment issue, would it be acceptable for #9945 to land first and for me to rebase afterward, preserving the initialization fix? I would record Depends on #9945 if agreed. AI-assisted.

p3nicillin and others added 7 commits October 24, 2026 00:51
A Windows process started without a standard handle, such as a detached
process or a --windows-gui executable started from Explorer, has
descriptors 0 to 2 whose _get_osfhandle is -2. fcntl then threw on their
unknown file type, which stopped Halo: Combat Evolved (PPSA99500) at
start-up. Open NUL on each of them when libc loads, as Linux and macOS
already open /dev/null on a closed standard descriptor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@p3n1cillin
p3n1cillin force-pushed the claude/windows-missing-standard-handles branch from 1366ebb to 41a1e6a Compare October 24, 2026 00:12
@p3n1cillin

Copy link
Copy Markdown
Author

@harisibrar1, @Queuse, @oneandonlydean: thanks, you were right that _dup2 alone left the UCRT streams on _fileno -2 and the guest streams on -1. I cherry-picked all six of @Queuse's commits (06f41e4, 81f2bd1, 49f4682, 385b4f7, 245043f, dfb7e98) with authorship kept, onto the branch rebased on main 2ace103. Head is 41a1e6a.

Rerun on Windows 11, WinLibs GCC 15.2.0, Release, full build:

  • guest_closed_standard_descriptors_windows passes. It covers stdin, stdout, stderr and all three, with native and guest writes and flushes and stdin EOF without an error.
  • guest_closed_standard_streams_load_windows passes.
  • All 360 libc and libkernel tests pass.
  • Halo: Combat Evolved (PPSA99500) with a null stdin still runs its start-up and stops only on its own game-file check, as before.

WSL Ubuntu 26.04.1, GCC 15.2.0: guest_closed_standard_descriptors and the three guest_standard_streams tests pass on a fresh libc.

@MarcZX10, landing this before #8897 and rebasing that afterwards is fine with me.

AI-assisted: yes (Claude Code)

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 24, 2026
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 24, 2026
@oneandonlydean
oneandonlydean merged commit 3eebf30 into boykopovar:main Oct 24, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants