Skip to content

dotenv: read an env file to its end instead of trusting st_size - #44026

Open
robobun wants to merge 2 commits into
mainfrom
robobun/3c9497f1/env-file-zero-st-size
Open

robobun wants to merge 2 commits into
mainfrom
robobun/3c9497f1/env-file-zero-st-size

Conversation

@robobun

@robobun robobun commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • An env file that reports st_size 0 and has content loads nothing. Bun prints no message and exits 0. Node v26.3.0 and Bun 1.4.0 load the file.
  • read_env_file_contents (src/dotenv/env_loader.rs:893) returns Empty on st_size == 0, before any read. dotenv: skip .env entries that are not regular files #40711 added the check.

Fix

  • The loader calls fstat once and uses st_size only to presize the buffer. It reads until a read returns 0 bytes. This applies to --env-file and to the default .env files.
  • One policy change: a default .env that fails to read is reported and skipped. Before, most errnos stopped bun with no message.
  • Verified: test/cli/run/env.test.ts (4 new tests, 3 fail with main's src/), no-envfile.test.ts, run-process-env.test.ts.
  • Self-reviewed: 14 findings asked for one rework, done here. A failed read of an --env-file keeps its result (cli: fail when an explicit --env-file cannot be loaded #36523 owns it).

Background

  • Loader::load loads the --env-file list or the default .env* names. Both end in read_env_file_contents.
  • st_size is the size that the file system reports. procfs makes the content on read and reports 0.
  • Found by inspection. No user reported it.
  • Considered the removal of the size check alone, the 1.4.0 reader. It loads the fixtures too. The read loop also removes the second fstat (4 to 3 syscalls).

Downsides

  • A truly empty env file costs 2 syscalls, not 1, and one malloc(8192).
  • A regular file with st_size 0 and no end is read until allocation fails. /proc/self/pagemap under a 4 GiB limit: 512 MiB read, then skipped, exit 0. Node aborts.
Notes

Repro on main (Linux). The newlines in the variable put a dotenv line into the file.

CARRIER=$'\nPROBE=from-procfs\n' bun  --env-file=/proc/self/environ -e 'console.log(process.env.PROBE)'   # undefined
CARRIER=$'\nPROBE=from-procfs\n' node --env-file=/proc/self/environ -e 'console.log(process.env.PROBE)'   # from-procfs

A .env that is a symlink to the same file is skipped the same way. Released versions: 1.3.0 skips the file, 1.4.0 loads it, 1.4.1 and 1.4.2 skip it. The Zig loader had the check, the Rust port dropped it, and #40711 restored it together with the skip of directories, FIFOs and sockets.

The fixture is not a use case. A plain /proc/<pid>/environ is not a dotenv file. Its entries end with a NUL, not with a newline, so it parses as one variable. Bun also keeps a NUL inside a value, and Node cuts the value there. The tests put a line of their own into the file through a variable with newlines.

Until a release has this. bun --env-file=<(cat FILE) and process.loadEnvFile(FILE) load such a file on main.

Read failures. sysfs answers a read of /sys/class/net/lo/speed with EINVAL.

input main this PR Node v26.3.0
--env-file=a.env,/sys/class/net/lo/speed exit 1, no output exit 1, no output exit 9, invalid format
.env symlink to it, beside .env.local exit 1, no output loads .env.local, exit 0 no default files
bun install in that directory error: An internal error occurred (EINVAL) EINVAL error loading .env file, then installs

The first row is unchanged on purpose. #21105 and #36523 decide what an --env-file that cannot be loaded does. Default discovery is best effort, so it reports the file and continues. It already did that for ENOMEM, EPIPE, EACCES and EISDIR.

Measurements. Linux x64, release builds of main and of this branch.

Syscalls on the env file descriptor between open and close (strace):

input main this PR Node v26.3.0
33 B regular 4: fstat fstat pread pread 3: fstat read read 2
empty regular 1: fstat 2: fstat read 1
procfs, 28 B, st_size 0 1: fstat, loads nothing 3: fstat read read 2
100 KiB regular 4 3 14
FIFO (--env-file) 3 3 2
FIFO behind a .env symlink 1, skipped 1, skipped
1 TiB sparse .env, 4 GiB limit 2, allocation fails, skipped 1, allocation fails, skipped

The default .env files give the same numbers as --env-file. The whole process, bun -e 0 with a 33 B .env: 249 to 248 syscalls. A .env is opened once per discovery pass: once for the main thread, once per Worker, once for bun install. An empty .env with one Worker: 2 to 4 syscalls. A 33 B .env with one Worker: 8 to 6.

Inside read_env_file_contents (gdb, three runs agree):

input instructions, main instructions, this PR allocator calls, main allocator calls, this PR
33 B 227 193 1 mi_malloc(49) 1 mi_malloc(49)
100 KiB 601 567 1 mi_malloc(102416) 1 mi_malloc(102416)
empty 36 343 0 1 mi_malloc(8192), 1 mi_free
procfs, st_size 0 36 280 0 1 mi_malloc(8192)

Binary size (size -A): .text 58299580 to 58299580 (+0 B), stripped file 80995912 to 80995912 (+0 B), bun-profile -616 B. Two builds of main are byte-identical, so the floor is 0 B.

Startup wall-clock, bun -e 0, 300 interleaved runs, medians: 7.74 ms and 7.60 ms with a 33 B .env, 6.98 ms and 6.99 ms with an empty .env. A second set of copies of the same binaries moved the medians by 1.2 ms, so there is no difference above the floor.

No size bound. Node has none, and #40837 documented the same for --env-file=/dev/zero. A sparse .env of any st_size is already read to its end on main. Under prlimit --as=4294967296, /proc/self/pagemap as --env-file or behind a .env symlink: main does fstat and no read. This PR does 17 reads for 512 MiB in 1.3 s, the next allocation fails, the file is skipped, and bun exits 0. Node exits 134 with std::bad_alloc after 4.8 s. A bound is a separate decision and a small follow-up if it is wanted.

pread to read. The descriptor is new, so its offset is 0 and both calls read the same bytes. The one exception is macOS, where /dev/fd/N is a dup that shares the offset of the caller's descriptor. There the file is now read from that offset, as cat and Node read it.

The design. A review before the code was written picked this reader over two others. The removal of the check for --env-file only leaves the default files with the bug. A reader without fstat, as in Node, costs more for an env file of 8192 bytes or more: 14 syscalls for 100 KiB. The review planned two PRs. Both rewrite the same 30-line function, so this is one.

Same class, not in this PR. Each reader has its own PR: #41590, #43910, #41593, #43871, #43964.

Suites. Debug build: env.test.ts 111 pass, no-envfile.test.ts 9 pass, run-process-env.test.ts 2 pass. cargo check -p bun_dotenv for x86_64-pc-windows-msvc and aarch64-apple-darwin, cargo clippy -p bun_dotenv clean. test/cli/test/parallel.test.ts: 43 pass, and 2 cases fail the same way with main's src/ in this debug build (each worker has a unique JEST_WORKER_ID, lazily scales workers). The new tests run on Linux only. Windows and macOS have no file of this kind to test with.

Tools. strace 7.2 built from its release tarball, gdb 16.3, binutils size. bloaty, hyperfine, perf and valgrind are not installed here. prlimit --as replaces a memory cgroup.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/run/env.test.ts

A file system can report st_size 0 for a regular file that has content.
read_env_file_contents returned Empty for such a file before any read,
so the file loaded nothing and bun printed no message. Node reads the
file until a read returns 0 bytes.

The loader now calls fstat once, uses st_size only to presize the
buffer, and reads with read(2) until a read returns 0 bytes. This
applies to an --env-file argument and to the default .env files. A
default entry that is not a regular file is still skipped.

A default .env file that fails to read is now reported (unless quiet)
and skipped for every errno. Before, an errno outside
ENOMEM/EPIPE/EACCES/EISDIR stopped the process. An --env-file argument
that fails to read keeps its result.
@robobun

robobun commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 7:34 PM PT - Oct 6th, 2026

✅ @robobun, your commit 709162c5e492c9d3fd07599fd81e1c526af98f76 passed in Build #123678! 🎉


🧪   To try this PR locally:

bunx bun-pr 44026

That installs a local version of the PR into your bun-44026 executable, so you can run:

bun-44026 --bun

@robobun

robobun commented Sep 26, 2026

Copy link
Copy Markdown
Collaborator Author

Status: the fix is pushed, and it waits for CI and for review.

How I reproduced it (Linux x64, Bun 1.4.2 and main). The newlines in the variable put a dotenv line into a file that reports st_size 0:

CARRIER=$'\nPROBE=from-procfs\n' bun  --env-file=/proc/self/environ -e 'console.log(process.env.PROBE)'   # undefined
CARRIER=$'\nPROBE=from-procfs\n' node --env-file=/proc/self/environ -e 'console.log(process.env.PROBE)'   # from-procfs

strace shows openat, fstat, close on that file, and no read. With this PR the first command prints from-procfs.

Test: test/cli/run/env.test.ts, block env file on a file system that does not know the size. With main's src/, 3 of the 4 new tests fail.

@coderabbitai

coderabbitai Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration
  • Configuration used: Repository: oven-sh/bun/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Essentials
  • Run ID: 2a6ede9a-e8ce-4179-8cd0-1bebee87211c
📥 Commits

Reviewing files that changed from the base of the PR and between e2df59c and 709162c.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 116f634c-c1be-426c-b541-ecebb356dd6c

📥 Commits

Reviewing files that changed from the base of the PR and between 97246d0 and e2df59c.

📒 Files selected for processing (2)
  • src/dotenv/env_loader.rs
  • test/cli/run/env.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.


Walkthrough

read_env_file_contents now reads eligible files to EOF and handles buffer sizing and read errors. CLI tests cover empty files, zero-size files, and failing symlink reads.

Changes

Environment File Reading

Layer / File(s) Summary
File reading and validation
src/dotenv/env_loader.rs, test/cli/run/env.test.ts
The loader uses positive st_size values as buffer-capacity hints, returns Empty after a zero-byte read, and reports read errors according to the file source. Tests cover empty files in a list, zero-size files, and read failures.

Suggested reviewers: jarred-sumner

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to e2df5

The env-file reading change has no identified merge-blocking issue after normal checks.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: reading env files to EOF instead of relying on st_size.
Description check ✅ Passed The description is detailed and covers the problem, fix, behavior changes, verification, tests, and trade-offs. It does not use the template headings exactly, but it provides the required information …

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline findings, I also checked the Ok(0) => Empty mapping and the st_size presize: read_to_end_into goes through read_fill_vec, which returns the total bytes appended (not the last chunk), so a non-empty file cannot be misreported as empty; and both the up-front try_reserve_exact and the per-chunk growth are fallible, so an absurd st_size surfaces as ReadErr(oom) rather than an abort.

Extended reasoning...

The change rewrites the ~30-line read_env_file_contents in src/dotenv/env_loader.rs to read every env file to EOF via read_to_end_into and widens the recoverable-errno set for default .env* discovery, plus new Linux-only procfs/sysfs tests. It touches no auth, crypto, or injection surface; the two posted findings (silent skip on any errno under quiet, and unbounded reads of zero-size endless files) are the reasons a human should weigh the policy change.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread src/dotenv/env_loader.rs
use bun_sys::E;
match err.get_errno() {
E::ENOMEM | E::EPIPE | E::EACCES | E::EISDIR => Ok(ReadEnvFile::ReadErr(err)),
_ if source == EnvFileSource::Default => Ok(ReadEnvFile::ReadErr(err)),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Script runs now exit 0 with the default .env silently unloaded on any read error, where the base exits non-zero. The new arm at src/dotenv/env_loader.rs:910 turns every errno for EnvFileSource::Default into ReadEnvFile::ReadErr, and the bun run/script path sets quiet = true (src/runtime/cli/run_command.rs:601), so nothing is printed. Fix: keep unexpected errnos loud on every caller, e.g. print unlisted errnos regardless of quiet (or keep propagating them) while still skipping the file for the listed ENOMEM/EPIPE/EACCES/EISDIR set; the new test asserting script.stderr is "" certifies the silence and should change with it.

Why this was flagged

A user runs bun index.ts in a directory whose .env is a regular file that fails on read(2): EIO from a failing disk or network mount, EINVAL/ENOTSUP from an unusual file system, or the /sys/class/net/lo/speed symlink the PR's own test uses. load_env_file at src/dotenv/env_loader.rs:794 calls read_env_file_contents with EnvFileSource::Default; the new arm at src/dotenv/env_loader.rs:910 maps the errno to ReadEnvFile::ReadErr. The caller at src/dotenv/env_loader.rs:796-803 prints only if !self.quiet, and the script entry point sets env_loader.quiet = true at src/runtime/cli/run_command.rs:601 (also src/runtime/cli/run_command.rs:3512 and src/install/lockfile.rs:1805). The process then continues with the .env values missing and exits 0. On the base branch the same errno hit _ => Err(err.into()) and the run failed with a non-zero exit. The PR text says such a file is 'reported and skipped', but reporting only happens for non-quiet callers such as bun install; the new test '.env that fails to read, behind a symlink' in test/cli/run/env.test.ts asserts…

Verification: normal — acknowledged in diff: the new test ".env that fails to read, behind a symlink" pins expect(script.stderr).toBe("") and exitCode 0 for the script run, and the PR table row says "loads .env.local, exit 0"; the author's claim that default discovery "reports the file and continues" holds only for non-quiet callers (bun install), not for bun index.ts, where nothing is…

Comment thread src/dotenv/env_loader.rs
return Ok(ReadEnvFile::ReadErr(bun_sys::Error::oom()));
}
// `read(2)`, not `pread`: a pipe or a device cannot seek.
match file.read_to_end_into(&mut buf) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) A checkout whose .env points at an endless zero-size procfs file now grows bun's heap without bound until the kernel OOM-kills it; the base skipped the file with one fstat. src/dotenv/env_loader.rs:903 reads every regular file to EOF, and the base's st_size == 0 short-circuit is gone, so /proc/self/pagemap (S_IFREG, st_size 0, ~256 GiB of zeros) is read in doubling chunks. Fix: keep a bound for the discovery path and --env-file alike, e.g. a total byte cap on read_to_end_into for env files, or restore the size-0 skip for Default sources. Under a cgroup limit malloc does not fail, so the PR's "skipped, exit 0" only holds with RLIMIT_AS.

Why this was flagged

Trigger: a .env (or .env.local etc.) that is a symlink to a regular file with st_size 0 and no EOF, e.g. /proc/self/pagemap, reached through load_env_file at src/dotenv/env_loader.rs:794 on every script start, every bun install, and once per Worker; also --env-file=/proc/self/pagemap via src/dotenv/env_loader.rs:838. The base at 97246d0 src/dotenv/env_loader.rs:893-895 returned ReadEnvFile::Empty for a regular file with stat.st_size == 0 before any read, so this population was never read; the dismissal's "base already reads sparse files to the end" only covers st_size > 0. Now src/dotenv/env_loader.rs:903 calls read_to_end_into, whose loop in src/sys/lib.rs:4420-4432 grows the Vec via amortized try_reserve(8192) until a read returns 0. On Linux with overcommit or a cgroup memory limit, try_reserve keeps succeeding and read touches the pages, so the process is SIGKILLed by the OOM killer (or thrashes swap) rather than getting the ENOMEM-then-skip path the PR measured under prlimit --as. The user sees bun killed with no message where the base ran normally.…

Verification: normal (trigger is adversarial/contrived: a .env* in cwd, or an --env-file, that is a symlink to an st_size-0 regular procfs file such as /proc/self/pagemap) — acknowledged in diff: the PR description lists "A regular file with st_size 0 and no end is read until allocation fails. /proc/self/pagemap under a 4 GiB limit: 512 MiB read, then skipped, exit 0" and "A bound is a separate…

@robobun

robobun commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

@robobun wake up!!

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun wake up!!

Comment thread src/dotenv/env_loader.rs
Comment on lines +872 to +873
/// An `--env-file` argument: read whatever its kind, as in Node. A read
/// errno outside ENOMEM/EPIPE/EACCES/EISDIR stops the process.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/dotenv/env_loader.rs
Comment on lines +895 to +896
// `st_size` only presizes the buffer: a file system can report 0 for a file
// that has content. The file ends where a read returns 0 bytes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

@robobun

robobun commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

State of this PR:

  • Merged main (bbdc5a5) into this branch. There was no conflict, and the diff against main is the same as before the merge.
  • The bug is still on main. On a debug build of main at bbdc5a5, the three tests of this PR in test/cli/run/env.test.ts fail (env file on a file system that does not know the size). The other 108 tests in the file pass.
  • I did not build this merge locally. CI is its first build.
  • The mordant check was red on the previous head. This run shows if that comes from this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants