Repository navigation
Conversation
…ait in it A pipe and a socket have no length. macOS and the BSDs report the bytes that wait in one as st_size. The test asks Bun.stdin for its size (or calls exists()) while the first chunk waits in it, then sends a second chunk and expects the read to return both. This commit has the test alone, so that the macOS lanes run it against main's source.
|
Updated 5:05 AM PT - Oct 21st, 2026
❌ @robobun, your commit d8c45be has some failures in 🧪 To try this PR locally: bunx bun-pr 45718That installs a local version of the PR into your bun-45718 --bun |
|
Status PR: #45718. The fix is green on both macOS lanes. The branch now has main merged in (a conflict in the test file), and CI runs on that merge. How I reproduced it:
|
…of the file macOS and the BSDs report the bytes buffered in a pipe or a socket as its st_size. resolve_file_stat cached that count as the size of the file store. .size reported it, and a later read through that handle stopped there. resolve_file_stat now runs one stat and one assignment. It caches st_size for a regular file, and for a directory as before. A pipe, a socket and every other file type get no length. The tests put the first chunk in a socketpair or a pipe before the reader starts, so they need no FIFO and no shell. Each row checks what fstat reports, asks, and then reads to the end. They ask and read in more ways than before. On Linux, the fd of a symlink shows the same rule. Two more tests hold the size of a directory.
…-size-is-not-a-length
robobun
changed the title
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I diffed the old and new max_size rule in resolve_file_stat: regular files and directories (including a directory with st_size 0, which was already Infinity) resolve exactly as before, and the only other consumer of file.max_size (the !seekable && max_size == MAX_SIZE branch in get_size, Blob.rs:2078) still yields Infinity for a pipe or socket. I also checked the O_PATH literal in the Linux-only symlink test: 0o10000000 is the value on both x86_64 and aarch64 Linux.
Extended reasoning...
The change narrows the st_size caching in Blob.rs's resolve_file_stat to regular files and non-empty directories, collapsing two duplicated Path/Fd arms into a stat_file helper, and adds pipe/socket/symlink/directory tests; it touches no auth, crypto or injection surface. The refactor preserves the regular-file and directory branches, and the sole remaining writer of max_size is this function, which is what this note records as ruled out.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🟣
src/runtime/webcore/Blob.rs— On macOS a FormData part built from a pipe or socket is still truncated after this change, now at the node_fs initial buffer size instead of the buffered byte count, so the PR's "reads to the end" claim does not hold for this reader. Blob.rs:3686 passes blob.size (now MAX_SIZE) as rf_args.max_size, which sets has_max_size at node_fs.rs:7208 and disables buffer growth at node_fs.rs:7279. The pipe is read only up to the initial capacity and the rest is dropped silently. Fix: pass max_size as None when blob.size is MAX_SIZE so read_file grows to EOF for every unsized store, or resolve the pipe through the streaming reader.Why this was flagged
Trigger:
new FormData().append(name, Bun.stdin)or a fetch body over a pipe on macOS, then serialisation through Blob.rs:3661-3687. On the base branch macOS gave max_size = the buffered count N and read exactly N bytes. After the change blob.size is MAX_SIZE, Blob.rs:3686 still wraps it in Some, node_fs.rs:7208 sets has_max_size = true, node_fs.rs:7264 caps every read at buf.capacity() and node_fs.rs:7279 refuses to grow because has_max_size is set, so the part stops at the initial capacity (about 262,160 bytes per the PR's own Linux measurement). The dismissal called this pre-existing because Linux already passed MAX_SIZE, but the changed line is what now routes macOS into this truncating branch, and the PR text names no test for it. Population: macOS users uploading piped stdin larger than the initial buffer; silent data loss on every such upload. Remedy: send None instead of Some(MAX_SIZE) at Blob.rs:3686 so read_file reads to EOF.Verification: pre-existing. acknowledged in diff: the PR description's "Not fixed here" list states "A FormData part from a pipe stops at 262,160 bytes (seen on Linux with a 1,000,000 byte pipe, node_fs.rs:7210). A fetch body takes the same path." — and that bound is accurate per the code below; the author's claim that it is "the same on main and on this PR" is accurate on Linux and slightly off on macOS (base cut at the buffered count N, PR cuts at 262,160), so the PR makes this reader less wrong, not worse.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 1 minute. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…-size-is-not-a-length # Conflicts: # test/js/bun/util/bun-file.test.ts
Each way to ask and to read now runs on a Bun.file() of a socketpair
and of a pipe in the test process. A child process runs only the cases
that need the pipe as its stdin: Bun.stdin, process.stdin, Bun.file(0)
and Bun.file("/dev/stdin"). The file spawns 6 children where it
spawned 16.
|
Two notes from the review, and what this PR does with them. A
The last push merges main (one conflict in |
Problem
.size,exists()or aResponsebody statsBun.stdin. No error.head -c 1000000 /dev/zero | bun -e 'await Bun.stdin.exists(); console.log((await Bun.stdin.bytes()).length)'prints65536(macOSst_sizeemulated on Linux).resolve_file_stat(src/runtime/webcore/Blob.rs) cached any positivest_sizeas the file size. macOS and the BSDs report the bytes buffered in a pipe or a socket there.Fix
resolve_file_statcachesst_sizeonly for a regular file, and for a directory as before. Other types have.sizeInfinity.test/js/bun/util/bun-file.test.ts. Both macOS lanes of build 127436 fail 8 cases on main. With the fix, both pass (build 128969). On Linux, the symlink case needs the fix.exists()on a socket.Background
Blobis a window on a store.Bun.stdinuses a file store that caches one stat. An unknown size means "read to the end".resolve_size: the wrong count stays in the store. A regular-file-only rule changesBun.file(dir).size.Downsides
Bun.stdin.sizeno longer means "stdin is piped".fs.openAsBlob()of a pipe changes too..sizeof a file runs 5 more instructions (264 to 269, x64)..textshrinks by 256 bytes.BunFile.existsmakesBun.writewrite nothing #4930). See Notes.Notes
How the bug shows
Nobody reported this. It came from a read of the code, and then from the macOS lanes below. #27849 is the same pattern on Linux, and #27852 repaired it.
Infinityand no cut. Soecho hi | bun ...on a Mac usually shows the whole input.Bun.write(Bun.stdout, Bun.stdin)is whole (bun-write.test.js:546holds that on the macOS lanes). The reader exits with code 0. The writer can getSIGPIPE.What puts the count on the handle:
.size,exists(),structuredClone(),Bun.inspect()orconsole.logof aResponseover it, and.bodyof thatResponse.lastModifiedputs the count in the store only:.sizethen reports it, but atext()right afterlastModifiedis whole. Each of these has a row in the test.What is then cut:
text(),bytes(),stream(),slice(), aResponsebody, andprocess.stdinwhen the stat comes before its first use (ProcessObjectInternals.ts:119builds it fromBun.stdin.stream()).The clause
|| stat.st_size > 0is from d5c81b7 (2022). That commit made an empty FIFO reportInfinity. It keptst_sizefor every other case, and no test named a file type for that.Evidence
6andfirst,whereInfinityandfirst,secondare expectedst_sizeof each pipe and socket rewritten to itsFIONREADcount by aptracetoolBun.file(fd).sizeis the length of the target), also on 1.4.2The rows changed twice after build 127436, and they keep the same questions and the same readers.
fstatreported: 6 on macOS, 0 on Linux. So a row that passes on macOS also shows that the kernel reported the waiting bytes, which is the input that main takes for a size.Bun.file(fd)in the test process. 6 rows stay in a child, becauseBun.stdin,process.stdin,Bun.file(0)andBun.file("/dev/stdin")need the pipe as stdin. The two Linux lines of the table are for this last shape. On macOS, two of its 18 rows ran in build 128969 in the same shape.Build 128969 also confirms two expectations that came from source:
stat("/dev/stdin")of a pipe reports the buffered bytes on macOS (XNUfdesc_attrcallspipe_stat).st_size0 on Windows (fstat_handleinsrc/sys/lib.rs), soBun.file(dir).sizeisInfinitythere.Measurements
x64 Linux, release builds of main (017b243) and of this PR.
resolve_file_stat(nm -S).text(size).sizeof a regular file by path, getter entry to return, user space (gdbsingle-step).sizeof a directory.sizesize; exists(); lastModified; text()on a 64 KiB file, per iteration (ptracecounter, 300 and 600 iterations)newfstatat, 1fstat, 1read.sizethentext()of a pipe,st_sizeemulated:fstaton fd 0read(0, buf, 6) = 6, then stop.size, 5exists(), 1lastModifiedonBun.stdin:fstaton fd 0Bun.file(dir).size, ext4, by path and by fdBun.file("/proc").sizeInfinityInfinityContent-Lengthofnew Response(Bun.file(dir))File.max_sizeWhat else changes
Bun.stdin.size !== Infinityto see if stdin is piped: withprintf 'hello\n'piped in and the macOSst_sizeemulated, main printsread 6and this PR printsskip. Linux printsskiptoday.!process.stdin.isTTYorfs.fstatSync(0).isFIFO()answer that question.fs.openAsBlob(path)isBun.file(path), so its size for a pipe changes with it.fs.openAsBlob(dir).sizestaysst_size.Responseover a pipe sendscontent-length: 0where it sent the buffered count (RequestContext.rs:2739). Linux sends 0 today. Not run on macOS.O_PATH | O_NOFOLLOWhas.sizeInfinitywhere it had the length of the target. No read of that fd works before or after. On macOS, an object fromshm_openhas no file type inst_mode, so it moves the same way (from source, not run).st_size0 there.cargo check --workspacepasses forx86_64-unknown-freebsd,aarch64-apple-darwinandx86_64-pc-windows-msvc.Not fixed here
These are the same on main and on this PR, except where a line says so.
exists()on a missing file, then a read after the file appears (CallingBunFile.existsmakesBun.writewrite nothing #4930, BunFile.text()does not return correct content after BunFile.write()#23902). Draft Bun.file: do not use a cached stat size as the byte budget of a read or a copy #43910 removes that copy and still needs this rule for.sizeof a pipe. The merge order does not matter.st_size0) reads""after.size.FormDatapart from a pipe stops at 262,160 bytes (seen on Linux with a 1,000,000 byte pipe,node_fs.rs:7210). Afetchbody takes the same path. On macOS, main stops this reader at the buffered count. With this PR it stops at 262,160 bytes, as on Linux. Open PRs FormData: read a Bun.file() part whose stat size is 0 #41590 (aFormDatapart) and fetch: stream a Bun.file() body that is a FIFO, socket, or device instead of reading it on the JS thread #41488 (afetchbody) change those readers.await Bun.stdin.exists()isfalsewhen stdin is a socket, on every platform (Blob.rs:1260accepts only a regular file and a FIFO). The stdin thatBun.spawn({ stdin: "pipe" })gives is a socketpair. No issue tracks this.Three other places read
st_sizeof a source that can be a pipe. They are different code with the same fact. This PR leaves them out on purpose: no CI lane can run a change to them, and none of them cuts a read on macOS or Linux. No issue tracks them. The table is from a read of the code, not from a run.fetch.rs:1604st_sizeof 32 KiB or more, goes through sendfile withmin(size, st_size)as its lengthISREGguard at:1595. On Linux a pipe reports 0.copy_file.rs:878(macOS) and:920(FreeBSD)Bun.writecompares thest_sizeof the source with the destination window beforeftruncatefcopyfilemust first accept the pipe.RequestContext.rs:1957Bun.servestoresmin(size, st_size)on the blob of a file body that is not regularsendfile.remain, which does not use that value.Open PRs #41257 and #43896 rewrite the same block and keep the old clause. This PR copies the
stat_fileshape of #41257 so that the later merge is small. The PR that lands second must keep the new predicate. If the old clause comes back, the symlink row fails on Linux and the pipe rows fail on macOS.Suites run on the debug build of this PR
bun-file.test.ts,regression/issue/27849.test.ts,bun-stdin-slice.test.ts,bun-file-exists.test.js,bun-file-fd-read.test.ts,bun-file-read.test.ts,filesink.test.ts,body-clone.test.ts(100 pass),blob.test.ts(147 pass),bun-serve-file.test.ts(172 pass),fetch-file-upload.test.ts, andfetch.test.ts -t "size is Infinity on a fifo".bun-write.test.js,body.test.tsandserve-file-slice-read-error.test.tseach have one or two tests that pass the 5 second default timeout on a debug build. The same tests do that on a debug build of main.no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/util/bun-file.test.ts