Skip to content

fix(libkernel): extend a file with ftruncate without writing zero bytes on Windows - #4968

Closed
xXPBSXx wants to merge 2 commits into
boykopovar:mainfrom
xXPBSXx:fix/ftruncate-no-zero-fill-v2
Closed

xXPBSXx wants to merge 2 commits into
boykopovar:mainfrom
xXPBSXx:fix/ftruncate-no-zero-fill-v2

Conversation

@xXPBSXx

@xXPBSXx xXPBSXx commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

On Windows, ftruncate, truncate and sceKernelFtruncate extended a file through _chsize_s. That call writes the whole new length as zero bytes. FreeBSD extends the file without writing data (the new range is a hole on a file system that has holes) and fails a length the file system cannot hold with EFBIG.

The difference shows as write load and as an unbounded loop:

  • ftruncate(fd, 256 MiB) wrote 256 MiB of zeros (measured once by hand); ftruncate(fd, INT64_MAX) kept writing at the disk's speed until the volume was full (found when a failure-path probe of mine did this and filled a test drive; stopped by hand).
  • It is a robustness and semantics difference, not a game measurement: no title I ran was seen to call ftruncate with a large length.

NativeFtruncate now sets the end of file with SetFileInformationByHandle(FileEndOfFileInfo), which does not write the extension and does not move the file offset. The errors follow FreeBSD where Windows distinguishes them: ERROR_INVALID_PARAMETER on a disk file (a length beyond what the volume can hold; seen with INT64_MAX, the size from which NTFS refuses it was not measured) is EFBIG (27) (on another kind of handle it is EINVAL), ERROR_FILE_TOO_LARGE is EFBIG as well, ERROR_DISK_FULL and ERROR_HANDLE_DISK_FULL are ENOSPC (28), ERROR_ACCESS_DENIED (a read-only descriptor) stays EACCES, which ftruncate already turns into EINVAL. Shrinking works as before. truncate (by path) and sceKernelFtruncate use the same function. Linux uses ::ftruncate as before.

Limits, listed in the TechnicalDebt entry for libkernel: the file is not sparse, so NTFS reserves the clusters and a length above the free space fails with ENOSPC where FreeBSD's hole succeeds; and a write behind the end of the file still makes NTFS write zero bytes over the gap (a one-byte pwrite far behind the end takes as long as writing the gap, lseek + write the same). Making the file sparse would change that and is not part of this PR.

Overlap: #4923 ("reject ftruncate on non-file Windows handles") is merged and changed the same lines of NativeFtruncate; the conflict is resolved in the merge commit described under Tested (its GetFileType check, non-disk handle: EINVAL, stays in front of the SetFileInformationByHandle call of this PR).

Tested

Windows 10 (19045), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release, on main c2565f014; moved since onto main 0214069e3 (rebased cleanly) with its test registration and debt entry at their own places, the test length reduced from 256 MiB to 64 MiB and the check changed from a timing comparison to a byte count:

  • guest_truncate_extend (new, length capped at 64 MiB, removes its files, also after a failure) passes: on Windows it reads the process's WriteTransferCount (GetProcessIoCounters) before and after ftruncate to 64 MiB and requires fewer than 4 MiB written (a zero fill writes the whole length; SetFileInformationByHandle writes nothing), so no time and no disk speed enter the check; it prints extending by 64 MiB wrote 0 bytes with the change. It also checks the size, the unchanged file offset, the old bytes kept, the extension reading as zeros, shrinking, EINVAL for a negative length and for a read-only descriptor, and truncate by path. On other hosts only these functional checks run.
  • It fails without the change to Stdio.cpp (extending a file must not write its new length as zero bytes: the process wrote about as many bytes as the new length; checked with Stdio.cpp taken from main). The test does not call ftruncate with INT64_MAX, as main would fill the disk.
  • ftruncate(fd, INT64_MAX) with the change returns EFBIG at once (run once by hand with a 20 s kill timer; not part of the test). The EFBIG and ENOSPC branches are not covered by the test; the EINVAL for a negative length comes from the check that was already in front of NativeFtruncate.
  • guest_filesystem and the other guest_file* tests pass. python tools/check_conventions.py --base upstream/main passes.
  • Re-run on 14 October 2026 (Windows 10 19045, MinGW-w64 GCC 15.2.0, Release, -j4), after main 016db923a was merged into the head (merge commit 5e5e41708; one conflict in NativeFtruncate in Stdio.cpp with fix(libkernel): reject ftruncate on non-file Windows handles #4923, which rejects a descriptor that is not a disk file with EINVAL: both changes are kept, the check for a non-disk handle now runs before SetFileInformationByHandle, so the EINVAL branch of the error mapping for a non-disk handle is gone and ERROR_INVALID_PARAMETER is always EFBIG): guest_truncate_extend passes (0.04 s, ctest) together with guest_filesystem (which holds fix(libkernel): reject ftruncate on non-file Windows handles #4923's tests), guest_kernel_sys and guest_file_vector (4 of 4, 0 failed); with Stdio.cpp taken back to main and the new test kept it fails (extending a file must not write its new length as zero bytes: the process wrote about as many bytes as the new length, exit code 0xc0000409). python tools/check_conventions.py --base upstream/main passes. This replaces the earlier figures. Not run: the socket tests, Linux, a console, a title.
  • Not tested: Linux (not changed), non-NTFS volumes (exFAT, FAT32, ReFS), network drives, a console, a title run.

Checklist

  • Based on current main; no other open PR implements the same functions (overlap with fix(libkernel): reject ftruncate on non-file Windows handles #4923 described above)
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt (docs/dev/debt/libkernel.md)
  • No notes, investigation .md files or images added to the repository
  • New third-party code is a submodule built from source (none added)
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes

AI-assisted (Claude Code).

🤖 Generated with Claude Code

This was referenced Oct 13, 2026
@xXPBSXx
xXPBSXx force-pushed the fix/ftruncate-no-zero-fill-v2 branch from ce4f8ca to 5605bb4 Compare October 13, 2026 07:06
…es on Windows

ftruncate, truncate and sceKernelFtruncate extended a file through _chsize_s, which writes the whole new length as zero bytes: 256 MiB took about 700 ms and ftruncate(fd, INT64_MAX) kept writing until the disk was full. FreeBSD extends the file without writing data and fails a length the file system cannot hold with EFBIG. Set the end of file with SetFileInformationByHandle instead and map ERROR_INVALID_PARAMETER on a disk file to EFBIG, ERROR_DISK_FULL to ENOSPC and a read-only descriptor to EACCES (EINVAL for the caller, as before).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xXPBSXx
xXPBSXx force-pushed the fix/ftruncate-no-zero-fill-v2 branch from 12a1071 to 2a74997 Compare October 13, 2026 16:53

@CnRJay CnRJay left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SetFileInformationByHandle(FileEndOfFileInfo) is the right call: it extends without writing, and reads of the extension still return zeros because of NTFS's valid-data length.

One mapping to check: ERROR_ACCESS_DENIED → EACCES. FreeBSD's ftruncate(2) fails with EINVAL when the descriptor was opened without write permission (kern_ftruncate: if ((fp->f_flag & FWRITE) == 0) return (EINVAL)), and EACCES is only for truncate on a path. If a read-only descriptor can reach NativeFtruncate (I couldn't tell whether the result != -2 path or #5112 rejects it before this point), it would now report EACCES. If it can't reach it, a Require or a comment-free assertion in the test, covering ftruncate on an O_RDONLY descriptor (expect EINVAL), would pin it.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
# Conflicts:
#	core/libs/prx/libkernel/File/src/Stdio.cpp
@xXPBSXx

xXPBSXx commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. A read-only descriptor does reach NativeFtruncate on Windows, and the mapping is handled one level up: ftruncate_nid_postfix turns EACCES into EINVAL before it returns (if (error == EACCES) error = GUEST_EINVAL;), and sceKernelFtruncate goes through ftruncate_nid_postfix. So ftruncate and sceKernelFtruncate on a descriptor opened without write permission answer EINVAL, as kern_ftruncate does; EACCES remains only for truncate by path (NativeTruncate opens the file for writing and reports the host error), which is what FreeBSD's truncate(2) does. The new test already pins it: guest_truncate_extend has the check for a read-only descriptor (EINVAL), and it passes on the current head 5e5e41708 (Windows, with and without the fix as described in the text).

AI-assisted (Claude Code).

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

Closing: no title is known to reach this. With the queue at 700+ open PRs, please hold off on new PRs that don't name a title needing the change until it's down. If a title stops on this on current main, reopen with the title ID and the log line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants