Skip to content

fix(libc): follow the console fgetws for short buffers and read errors - #9596

Open
JJenkx wants to merge 1 commit into
boykopovar:mainfrom
JJenkx:windows/fgetws-console-contract
Open

JJenkx wants to merge 1 commit into
boykopovar:mainfrom
JJenkx:windows/fgetws-console-contract

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What

fgetws now follows the console's fgetws in two cases where it differed. The console code was read from the libc.prx that titles ship in sce_module/. I read it from disassembly, not by measuring on a console. The copies of PPSA13934, PPSA07923 and PPSA05916 have the same 133-byte fgetws (F81hKe2k2tg). The addresses below are from PPSA13934's copy (fgetws 0x6cab0, fgetwc 0x6c7a0). The reading is also posted on #8897 (comment).

Case Console Before Now
size < 2 Returns null at once and writes nothing: no buffer, errno or stream flag (cmp $0x2,%esi; jge …, else xor %r14d,%r14d). size == 1 stored a NUL and returned the buffer. size <= 0 set the stream error and errno 22. Returns null and writes nothing.
fgetwc returns -1 after some characters were stored Stores the NUL and returns the buffer. Null only when nothing was stored (cmp %r14,%rbx; je). It never checks ferror. Returned null when ferror was set. Returns the terminated buffer.

Already matching, unchanged:

  • Each fgetwc result is stored as its low 16 bits (mov %ax,(%rbx)).
  • The newline test compares all 32 bits (cmp $0xa,%eax).

This means the 32-to-16-bit narrowing reported in #8897 is the console's own behaviour, not a loss in this implementation.

Not reproduced, listed in the debt entry:

  • A null buffer or stream: the console dereferences it, and here it keeps failing with errno 22.
  • The calls the console makes before and after the read loop.
  • The call it makes when bits 0xc of the stream's second flag byte are set.

Changed tests: three guest_standard_streams assertions encoded the old guesses, and now expect the console results:

  • sizes 1, 0 and -1
  • an invalid byte after a stored character, which now gives the buffer "A"

The fgetwc error-indicator check after that invalid byte is unchanged.

Tested

  • Platform: Windows 11 (10.0.22621), WinLibs GCC 15.2.0 UCRT r7, Release, BUILD_TESTING=ON.
  • Results: guest_standard_streams, guest_standard_streams_closed_stdout and guest_standard_streams_exclusive_stdout pass, as do guest_wide_formatting, guest_wide_strings, guest_wide_conversion and guest_wide_stream_scan.
  • Conventions: tools/check_conventions.py passes.
  • Rebased onto main b786b90 (2026-10-22), head 5ada941: guest_standard_streams, guest_standard_streams_closed_stdout, guest_standard_streams_exclusive_stdout and the other libc stream, wide-character and formatting tests (26) pass in three runs out of three. check_conventions.py and git diff --check pass. With feat(libc): implement gets line input #9342, libc.md conflicts only on adjacency: feat(libc): implement gets line input #9342's new gets entry goes directly above this PR's fgetws entry, and both lines are kept unchanged.
  • Not done: no title was run, and nothing was measured on a console.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none (related: libc: align fgetwc conversion/state with mbrtowc and verify fgetws edge cases #8897)
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

🤖 Generated with Claude Code

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The revised fgetws handling matches the documented disassembly cases: sizes below two return before touching stream/buffer state, and a read error after at least one character returns a terminated buffer while retaining the stream error flag. The tests cover both regressions and preserve the neighboring wide-stream cases. The author clearly labels the source as disassembly rather than console measurement and records the remaining ABI questions in debt. Hosted conventions, eligibility, intake, labels, report, register-context, relinker matrix, ASAN and write-watch checks pass; hosted Linux/Windows product builds are skipped. The Windows focused suite is author-reported. I did not run tests locally.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 9342 docs/dev/debt/libc.md L74-75
PR 9945 docs/dev/debt/libc.md L75-76

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3965 docs/dev/debt/libc.md
PR 5021 docs/dev/debt/libc.md
PR 5303 core/libs/prx/libc/src/FormattingWide.cpp
docs/dev/debt/libc.md
PR 6044 core/libs/tests/GuestStandardStreams.cpp
PR 7081 docs/dev/debt/libc.md
PR 7648 docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 docs/dev/debt/libc.md
PR 8306 docs/dev/debt/libc.md
PR 8965 docs/dev/debt/libc.md
PR 9133 docs/dev/debt/libc.md
PR 9408 docs/dev/debt/libc.md
PR 9447 docs/dev/debt/libc.md
PR 9526 docs/dev/debt/libc.md
PR 9527 core/libs/prx/libc/src/FormattingWide.cpp
docs/dev/debt/libc.md
PR 9552 docs/dev/debt/libc.md
PR 9669 docs/dev/debt/libc.md
PR 9929 docs/dev/debt/libc.md

@MarcZX10 MarcZX10 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fgetws changes match the cited bundled-libc disassembly: sizes below two return before touching the buffer, errno or stream, and a read error after stored characters yields a terminated buffer while preserving the stream error. The tests cover those cases; the debt entry distinguishes bundled-binary disassembly from console measurement and retains the unresolved calls/decoding details. Exact-head git diff --check and conventions pass. Hosted conventions, eligibility, intake, labels, report, relinker matrix, ASan, register-context and write-watch pass; hosted Linux/Windows product builds are skipped. The author reports the focused Windows tests. I did not run runtime tests locally. I need to correct my earlier approval against current repository state. This branch is 91 commits behind current main (01b38ec is based on 6ef5a1c), so the code and tests have not been revalidated on the current integration base. Please rebase and rerun the focused standard-stream tests, then update the tested-head/base information. The bundled-binary source is useful evidence for behavior, but does not itself establish a current title stop. The overlap bot flags #9342 on the adjacent docs/dev/debt/libc.md hunk. I checked its patch: it adds the separate gets entry immediately before this PR's fgetws entry; there is no implementation overlap. Please coordinate/rebase the debt edits with that author and declare a dependency only if an actual merge order is needed. Fixes: none (related: #8897) remains accurate because the reported supplementary-character question is not resolved by these two behavior changes.

I’ve noticed that holding this PR because of #5239 isn’t necessary. It’s fine for PRs to keep coming in; they’ll be handled in due course. Please disregard my earlier request to hold or defer it.

@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @MarcZX10, for re-checking it against current state. You're right that it's 91 commits behind and was validated on the old base. I'm converting it to a draft under the #5239 hold, since no title run shows the need. Before marking it ready again I'll rebase it onto current main, rerun guest_standard_streams and the libc suite, and check the libc.md hunk against #9342 (which only adds the gets entry next to it).

@JJenkx
JJenkx marked this pull request as draft October 22, 2026 10:00
@JJenkx
JJenkx force-pushed the windows/fgetws-console-contract branch from 01b38ec to d15e72d Compare October 22, 2026 12:09

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fgetws changes remain supported by the cited bundled-libc disassembly, the Windows standard-stream tests, and the reported 26-test libc stream/wide suite. I rechecked exact head d15e72dab340aa2daa0e04221e2f7fb55d6efc75 against current main 3ca07d086d5f44fbe3c00370e7f03953ab46ca1b: whitespace and conventions pass, the current-main merge tree is clean, and both changed C++ translation units pass local GCC 12.2 C++20 syntax-only checks with -Wall -Wextra -Werror (suppressing existing unused/missing-field warnings in FormattingWide.cpp). I did not run the runtime suite locally; hosted Linux/Windows product jobs are skipped.

The branch is based on the earlier 8b3a6ba88d8fc74e768f1a7e8e9e36fb69842768 main, so please update to current main and rerun guest_standard_streams plus the focused libc stream/wide tests, then refresh the Tested head. Keep the #9342 debt-file coordination when rebasing; the reported change adds the separate gets entry adjacent to this one. The disassembly is correctly labeled as bundled-binary evidence, not console measurement.

I withdraw the #5239 intake-hold rationale: if the draft status was only because of that old intake direction, it need not stay draft for that reason. This review request is for current-main validation only; readiness remains your call.

@JJenkx
JJenkx force-pushed the windows/fgetws-console-contract branch from d15e72d to 5ada941 Compare October 22, 2026 13:28
@JJenkx
JJenkx marked this pull request as ready for review October 22, 2026 13:28
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 rebased onto main b786b90 (head 5ada941). The 26 libc stream, wide and formatting tests, including guest_standard_streams, pass in three runs out of three on Windows. The Tested section is refreshed and records the #9342 gets/fgetws adjacency resolution, with both entries kept verbatim. The PR was a draft only because of the intake hold, so I've marked it ready.

@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Back to draft: @MarcZX10 clarified that the #5239 intake guidance is still active (#9688 (comment)). While the open queue is above 700, a PR should name a title shown to need the change on current main. This one names none yet, so it waits here. I'll mark it ready once a title run shows the need, or once the queue drops below the threshold. The code and test evidence above are unchanged.

@JJenkx
JJenkx marked this pull request as ready for review October 22, 2026 17:09
@MarcZX10

MarcZX10 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

I’ve noticed that asking you to hold this PR was unnecessary. It’s fine to keep submitting PRs; maintainers will handle them in due course.

The fgetws in the libc.prx that titles ship (the same 133 bytes in the
copies of PPSA13934, PPSA07923 and PPSA05916, read from disassembly)
differs from this one in two cases:

- A size below 2 returns null at once, without writing the buffer,
  errno or the stream (cmp $0x2,%esi; jge, else the null result). Here a
  size of 1 stored a NUL and returned the buffer, and a size of 0 or less
  set the stream error and errno 22.
- When fgetwc returns -1 after characters were stored, the console
  null-terminates and returns the buffer; it returns null only when
  nothing was stored, and never checks ferror. Here a read error after
  some characters returned null.

The rest already matches: each fgetwc result is stored as its low 16 bits
(mov %ax,(%rbx)), and the newline test compares all 32 bits. A null buffer
or stream, which the console dereferences, keeps failing with errno 22.

guest_standard_streams now expects the console results for sizes 1, 0
and -1 and for an invalid byte after a stored character; its fgetwc error
indicator check is unchanged. The libc debt entry cites the reading and
what it leaves unverified.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JJenkx
JJenkx force-pushed the windows/fgetws-console-contract branch from 5ada941 to ea6aa0a Compare October 22, 2026 20:30
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main 3bbd232 without conflicts; head is now ea6aa0a. Windows (GCC 15.2.0 UCRT r7, Release) build of that exact head: ctest -R 'stream|wide|formatting|libc' selects 26 tests, including guest_standard_streams, guest_standard_streams_closed_stdout and guest_standard_streams_exclusive_stdout, and all 26 pass in three runs out of three; tools/check_conventions.py --base origin/main passes. The log of those three runs is kept locally; this replaces the earlier result, whose run log was not kept.

@JJenkx

JJenkx commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor Author

Linux run at this PR's exact head ea6aa0a80, which has no Linux result yet (the description lists Windows only, and the hosted Linux product jobs are skipped).

Linux 7.2.9 (Arch), Ryzen 9 7950X, g++ (GCC) 16.2.1 20260810, glibc 2.44, Release, BUILD_TESTING=ON. Head ea6aa0a80 merged onto main 18b9b17664 (clean, merge fe1c645f11).

  • guest_standard_streams, guest_standard_streams_closed_stdout, guest_standard_streams_exclusive_stdout, guest_wide_formatting, guest_wide_strings, guest_wide_conversion and guest_wide_stream_scan: 7/7 pass in each of three runs.
  • Baseline: with this PR's test kept and main's FormattingWide.cpp, guest_standard_streams aborts (Subprocess aborted) on Linux, and the other two stream tests still pass. So the new cases cover the change on glibc as well as UCRT.

No title was run. No findings.

AI-assisted: yes (Claude Code).

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed updated head ea6aa0a. The focused diff now follows the described bundled libc.prx behavior for sizes below two and for partial reads after fgetwc returns -1, and the tests cover both without broadening the unverified decoding behavior. The refreshed description records 26 focused Windows libc tests passing on three runs, 7 focused Linux tests passing on three runs with a main-vs-PR comparison, conventions passing, and the evidence source accurately labeled as disassembly rather than console measurement. Current hosted checks pass for conventions and the relinker matrix; full Linux/Windows product jobs are skipped by repository policy. No blocking issue found.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants