Repository navigation
Conversation
A sliced Bun.file() at stdin gave the child the whole file: the child got the path or the descriptor of the store, and the window of the Blob was lost. Stdio::extract_blob now opens a window of a regular file in the parent, before a child exists, and reads its first chunk. Stdio::FileWindow carries it to the StaticPipeWriter as Source::FileWindow. When the writer has sent a chunk, it reads the next one (64 KiB at most) and waits for the next writable event. The parent holds one chunk for any length of the window. A window that the parent cannot open throws from spawn(). A window at stdio[3] or above throws: only stdin has a writer in the parent. A file that is not regular has no window, and the child gets that file as before. Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
|
Status at ec3c407, two commits on top of #41209 (d36bbf6). Draft: the transport needs a maintainer's choice. The description has the four questions. Reproduced on 1.4.3 (canary 367d939) with a 26-byte file const proc = Bun.spawn({ cmd: ["cat"], stdin: Bun.file(p).slice(10, 20), stdout: "pipe" });
console.log(await proc.stdout.text()); // abcdefghijklmnopqrstuvwxyzWith this PR the child gets The new block in |
… opened On Windows the path that the open gets is the NT form of the path, so the error carried a path with the \\?\ prefix.
|
Updated 6:01 AM PT - Sep 30th, 2026
✅ @robobun, your commit ec3c407fc600eca1a5de8bf90ed6e663d66ce3ae passed in 🧪 To try this PR locally: bunx bun-pr 44301That installs a local version of the PR into your bun-44301 --bun |
Problem
Bun.spawn({ stdin: Bun.file(p).slice(10, 20) })sends the child the whole file. So dospawnSync,new Response(slice),slice.stream()and the shell.Stdio::extract_blob(src/runtime/api/bun/spawn/stdio.rs:565) turns a file Blob intoStdio::PathorStdio::Fd. Neither holds the window.Fix
extract_blobopens the window in the parent before a child exists, and reads the first chunk.Stdio::FileWindowcarries it to theStaticPipeWriter. After each chunk the writer reads the next (at most 64 KiB) and waits for a writable event.test/js/bun/spawn/spawn.test.ts(43 cases, 32 fail on base). Self-reviewed: 9 concerns raised, 4 addressed (Notes).Background
StaticPipeWriteris the writer in the parent that sends a stdin payload from memory.Blob::file_window().Downsides
ENOENT(main creates the file). A window atstdio[3]or above throws (main: the whole file)./dev/stdingivesENXIO. A parent that exits afterunref()sends 0 bytes. A read error after the first chunk ends stdin early with no report.preadon the JS thread per writable event. Other callers: 0 added syscalls.Notes
Questions for a maintainer.
spawn(), and on Linux the child gets a memfd. Its cost grows with the window. Honor Bun.file().slice() bounds in Bun.write #41209 carried a port of that buffer until 8206895, and a pre-merge check of that head measured +537 MB of RSS and 616 ms on the JS thread for 512 MiB. A 3 GiB window in a 2 GB container killed the parent. This PR holds one chunk (+2.1 MB for 64 MiB and for 512 MiB,spawnSync+3.6 MB), and it pays with the costs in Downsides. A design review ranked six designs: this one first, the buffer last. A later self-review said that a person must choose between the two.landing-prs.md:31forbids a synchronous filesystem call on the JS thread in an async completion path. The refill is onepreadof at most 64 KiB in the writable callback.REVIEW.md:66asks for agreement before a feature PR adds a shared abstraction. This PR addsStdio::FileWindow,Source::FileWindowandSource::refill()to the writer thatBun.spawn, the shell andbun installshare.Self-review. Addressed: the open no longer creates a missing file. A window at
stdio[3]throws, and the test no longer pins the whole file there. A groundwork commit that kept anunsafemove is gone, because #40204 rewrites the same arms. This change adds nounsafe. The exclusions are named below. Open: the four questions above, and the merge order.Merge order. #41209 first.
slice(-5)of a file that nothing has stat'd gives an empty stdin until #41257 merges (main: the whole file).Not changed.
cat < ${Bun.file(path).slice(1, 3)}):shell_body.rsputs the path text into the command, so the window is gone beforeStdioexists. Shell builtins are the same.Bun.stdin.slice()at stdin: the slot inherits.Measurements (Linux x64, debug build).
VmHWM) at this head:spawn+2,056 KiB for a 64 MiB window and +2,104 KiB for 512 MiB.spawnSync+3,660 and +3,600 KiB.pread64of 65,536 bytes and 1,024sendto, with at most 1pread64between twoepollwaits. At parse a path window isnewfstatat,openat,pread64. A descriptor window isfstat,fcntl(F_DUPFD_CLOEXEC),pread64. Both are the same at 1 MiB and at 512 MiB. An unsliced file adds none."ignore","pipe","inherit",Bun.file(path),Bun.file(fd), a Blob, a TypedArray, the stream of an unsliced file, a JSReadableStream), 20 spawns each: the same counts ofopenat,fstat,fcntl,socketpair,memfd_create,sendtoandvforkas the base.size_ofis unchanged:Stdio384,Source24,StaticPipeWriter<Subprocess>384.Test runs (Linux x64, debug build with ASAN).
spawn.test.ts: one timeout ingcTick > spawn > stdout can be read, which takes 3 to 4.5 s of its 5 s here and passes alone. The machine had a load average above 240 on 16 cores.spawnSync.test.ts17 pass,spawn-empty-arrayBufferOrBlob.test.ts3 pass,spawn-stdin-pipe-fd-leak.test.ts4 pass,bunshell.test.ts436 pass.bun run rust:check-all: 12 targets pass.\\?\C:\...). ec3c407 reports the path that the caller gave. So the Windows writer and the restore of the file position ran in CI. I did not run Windows or macOS by hand.Overlap. #40204 (remove
unsafefromsubprocess.rsandshell/subproc.rs) and #42819 (remove libuv on Windows) rewrite the files of the writer. This PR adds separate match arms and leaves the existing arms as they are.no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/spawn/spawn.test.ts