Skip to content

test(tool-server): stop unit tests inheriting the developer's ARGENT_* overrides - #707

Draft
latekvo wants to merge 1 commit into
mainfrom
fix/tests-inherit-argent-env
Draft

test(tool-server): stop unit tests inheriting the developer's ARGENT_* overrides#707
latekvo wants to merge 1 commit into
mainfrom
fix/tests-inherit-argent-env

Conversation

@latekvo

@latekvo latekvo commented Aug 3, 2026

Copy link
Copy Markdown
Member

Problem

The unit suite inherits the developer's ARGENT_* shell configuration, so tests that assert argent's shipped defaults instead assert a property of the machine running them.

I hit this as six failures in boot-device-hotboot.test.ts on a Linux box:

AssertionError: expected [ '-avd', 'Pixel_7_API_34', …(9) ] to deeply equal ArrayContaining ["-gpu", "swiftshader"]
-   "swiftshader"
+   "host"

swiftshader is the Linux default and auto the macOS one. host is neither — it came from ARGENT_EMULATOR_GPU_MODE=host exported in my shell, which is the documented escape hatch for hosts where swiftshader is too slow.

selectGpuMode (boot-device.ts:210) reads the env override and returns before the process.platform === "linux" ? "swiftshader" : "auto" fallback. That precedence is correct and is not what this changes. The it.each(PLATFORMS) parameterisation is also fine — setPlatform really does control process.platform; the env override simply shadows it.

Proof that the source is not implicated:

env -u ARGENT_EMULATOR_GPU_MODE npx vitest run test/boot-device-hotboot.test.ts
→ Tests  25 passed (25)      # unchanged source

The failure reads as a source regression

This is the reason it is worth fixing rather than documenting. Someone who has ever set ARGENT_EMULATOR_GPU_MODE — the variable we tell people to set — gets six red tests naming a -gpu value they did not choose, in a file they did not touch, and nothing in the output mentions the environment. The obvious next move is to go read selectGpuMode, which is correct.

Not one file

ARGENT_EMULATOR_GPU_MODE is the one I tripped over; the class is every ARGENT_* override, since each is by definition a user-facing switch that changes asserted behaviour. Measured by exporting each and re-running:

variable file failures
ARGENT_EMULATOR_GPU_MODE boot-device-hotboot.test.ts 6
ARGENT_EMULATOR_NO_WINDOW boot-device-hotboot.test.ts 4
ARGENT_SIMULATOR_NO_WINDOW boot-device.test.ts 1 (suppresses the open -a Simulator.app it asserts)
ARGENT_PORT / ARGENT_HOST bind-failure-telemetry.test.ts 2 each

bind-failure-telemetry.test.ts is the pointed one: its header comment already worries about colliding with the developer's real tool-server and mocks the probe for exactly that reason — but it never considered the env override that retargets the bind.

With all eleven ARGENT_* variables set at once, across the three files:

before →  Test Files  3 failed (3)      Tests  9 failed | 42 passed (51)
after  →  Test Files  3 passed (3)      Tests  51 passed (51)

Fix

A suite-wide setup file deletes every ARGENT_* key from process.env before the test module graph is imported, so module-level env reads see the cleared state too.

Clearing the prefix rather than a hand-maintained allowlist is deliberate: an override added to src later is covered without a second edit here, which is precisely the kind of follow-up edit that does not happen. A test that exercises an override sets it itself, so nothing depends on the ambient value — verified by the full suite staying green.

Also reorders the four it.each(PLATFORMS) titles. Their %s placeholders ran against the [platform, gpu] tuple and rendered as "picks the hot-boot spawn args + -gpu linux on swiftshader" — platform and GPU mode transposed, in the first string a diagnoser reads.

Deliberately not scrubbed

ANDROID_HOME, ANDROID_SDK_ROOT, ANDROID_AVD_HOME, ANDROID_USER_HOME, ANDROID_SDK_HOME, XDG_CONFIG_HOME, LOCALAPPDATA. I ran the full package suite with all seven pointed at nonexistent paths: 297/297 green, so those tests already stub what they need. Scrubbing them would be defensiveness nothing pins, and they are shared SDK variables a test could legitimately want to read.

Tests that pin process.platform explicitly and restore it (android-binary-windows, command-on-path, lens-tools-platform-gate, linux-preflight) are correct as written and untouched. adb-resolve-avd-path.test.ts:54's it.skipIf(process.platform !== "win32") is a deliberate gate on Windows-only path parsing and stays.

Verification

  • The six -gpu failures reproduced at origin/main with the variable exported, and pass with it cleared — same command, same shell.
  • Hostile-env sweep above: 9 → 0.
  • Mutation: changing the Linux arm of selectGpuMode from swiftshader to auto is killed by four named on linux … tests, while the darwin arm stays green. Without this change that mutation is masked whenever the variable is set.
  • Full packages/tool-server: 297 files, 3090 passed, 1 skipped, 0 failed (was 6 failed | 3084 passed).
  • eslint . 0, prettier --check . 0, tsc --build 0 — run unpiped, exit codes read directly.

Unrelated, found while sweeping

test/vega-cli-timeout.test.ts fails roughly 1 run in 5 at origin/main with this change absent — it touches only PATH, never ARGENT_*. Two real-subprocess timing tests (rejects on its own deadline when the CLI never returns, reaps the ENTIRE worker tree on timeout) racing under parallel load. Not caused by this branch and not fixed here.

…* overrides

`selectGpuMode` reads ARGENT_EMULATOR_GPU_MODE before falling back to the
platform default, so a shell exporting `ARGENT_EMULATOR_GPU_MODE=host` — the
documented escape hatch for hosts where swiftshader is too slow — made the six
platform-parameterised `-gpu` assertions in boot-device-hotboot resolve `host`
instead of `swiftshader`/`auto` and fail. The same class bites elsewhere:
ARGENT_EMULATOR_NO_WINDOW fails four more in that file,
ARGENT_SIMULATOR_NO_WINDOW fails the Simulator.app attach assertion in
boot-device, and ARGENT_PORT/ARGENT_HOST fail both bind-failure-telemetry
tests.

Clear the whole ARGENT_* prefix from a suite-wide setup file so assertions test
the shipped defaults. Tests that exercise an override already set it themselves.

Also reorder the four `it.each(PLATFORMS)` titles, whose `%s` placeholders ran
against the [platform, gpu] tuple and rendered as "-gpu linux on swiftshader".
@latekvo

latekvo commented Aug 3, 2026

Copy link
Copy Markdown
Member Author

Arrived at this independently while sweeping out-of-scope findings from the #610 review, and opened a duplicate (#708) three minutes after this one - closing mine in favour of this. Same diagnosis, same fix shape (blanket ARGENT_* clear in a suite setup file). Dropping the extra measurements here so they are not lost with it.

Per-variable attribution, one full-suite run each, on main, with this box's own ARGENT_EMULATOR_GPU_MODE cleared first so each variable is isolated:

ambient variable failures file
ARGENT_EMULATOR_GPU_MODE=host 6 boot-device-hotboot.test.ts
ARGENT_EMULATOR_NO_WINDOW=1 4 boot-device-hotboot.test.ts
ARGENT_SIMULATOR_NO_WINDOW=1 1 boot-device.test.ts
ARGENT_SCREENSHOT_SCALE=0.5 0 -
ARGENT_CHROMIUM_PORTS=9333 0 -
ARGENT_IDLE_TIMEOUT_MINUTES=5 0 -
ARGENT_EVENT_LOG=1 0 -
ARGENT_PROJECT_ROOT=/tmp/x 0 -
ANDROID_HOME, ANDROID_SDK_ROOT, ANDROID_AVD_HOME 0 -

Two things that may be worth folding in:

  • ARGENT_SIMULATOR_NO_WINDOW breaks a second file, boot-device.test.ts, which is not in this diff. That file already saves and restores the variable inside the one test that exercises the override, while a different test in the same file asserts the default and fails - a good illustration of why the per-file pattern kept missing this. The blanket clear here already covers it; only noting that the blast radius is two files, not one.
  • The ANDROID_* results are clean, which is the evidence for keeping the guard scoped to argent's own namespace rather than widening it.

Verified against this PR's approach: with the setup file in place, the full suite is 3090 passed | 1 skipped both with all three variables exported and with none set - identical to the control.

Unrelated flake found by the same sweep, recording it rather than guessing at it: test/vega-cli-timeout.test.ts failed in two full-suite runs and then passed 3/3 in isolation with the same variables set. It spawns real subprocesses against a 400ms timeout and a 3s sentinel, so it is load-sensitive, not env-sensitive - I first mis-attributed it to ARGENT_PROJECT_ROOT, which is read only by update-argent.ts and never on that path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant