Skip to content

Hand off in-flight telemetry to a detached sender at exit - #1217

Merged
theoephraim merged 4 commits into
mainfrom
fix-telemetry-exit-latency
Oct 13, 2026
Merged

theoephraim merged 4 commits into
mainfrom
fix-telemetry-exit-latency

Conversation

@theoephraim

@theoephraim theoephraim commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Fixes #1216

varlock load (and so varlock/auto-load) paid a variable network round trip on every run: the exit hook awaited in-flight telemetry requests for up to 500ms, and the cli_schema_loaded event was only started at exit, so the wait was nearly always hit.

Now:

  • Events are queued and only sent in-process once the command has outlived a 200ms delay (unref'd timer). Short commands never pay the lazy fetch() client init (10-30ms of main-thread time) at all; long-running ones (varlock run, proxy) send and confirm delivery while they run, as before.
  • Anything still unconfirmed at exit is handed to a detached copy of the CLI (varlock --flush-telemetry, events passed in its environment as __VARLOCK_TELEMETRY_EVENTS), spawned with stdio ignored and unref'd; the parent exits immediately. The handoff costs about 1ms (a spawn syscall); the child's startup happens after the parent is gone. The child posts all events in one request with a 10s timeout and runs with VARLOCK_TELEMETRY_DISABLED set so it can never record or spawn telemetry of its own, and with __VARLOCK_CLI_CHILD set so a bunfig-preloaded varlock/auto-load inside it bails instead of spawning a varlock load (the CLI clears that marker from its own env at startup, so the handoff has to restore it). Nothing is written to disk.
  • The exit hook is now synchronous, so it also runs on a plain process.exit().
  • Each event carries a client uuid and timestamp so a resend of an event that was in flight at exit can be de-duplicated (PostHog honors uuid; the relay still mints its own event_id, worth switching to the client uuid there).

This is the same shape as Expo CLI (in-process for long commands, detached for short), Vercel CLI, Netlify CLI, Prisma, and next dev. Astro and Nuxt just fire-and-forget because all their commands are long-lived; wrangler awaits with a 1s cap and eats the latency.

Measured locally (varlock load --format json-full --compact, release build, node):

before after opted out
npm build 240-660ms 140-160ms 130ms
standalone binary 100-110ms 100ms

Verified the handoff and sender under node, bun, and the compiled standalone binary, and that execFileSync of the CLI (what auto-load does) returns as soon as the CLI exits.

Docs: added a "How Events Are Sent" section to the telemetry guide describing the handoff step by step.

Telemetry requests were awaited (up to 500ms) at exit and the schema event was
only started at exit, so every short command inherited a network round trip.
Events are now queued, sent in-process only once the command has outlived a
short delay, and anything still unconfirmed at exit is handed to a detached
copy of the CLI (--flush-telemetry) which delivers it after the parent is gone.
Events carry a client uuid and timestamp so a resend can be de-duplicated.

Fixes #1216
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

bumpy-frog

The changes in this PR will be included in the next version bump.

patch Patch releases

  • @varlock/native-helper-darwin 1.22.0 → 1.22.1
  • @varlock/native-helper-linux-arm64 1.22.0 → 1.22.1
  • @varlock/native-helper-linux-x64 1.22.0 → 1.22.1
  • @varlock/native-helper-win32-x64 1.22.0 → 1.22.1
  • varlock 1.22.0 → 1.22.1

Bump files in this PR

Click here if you want to add another bump file to this PR


This comment is maintained by bumpy.

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

📦 Bundle size

⚠️ grows the bundle by 15.3 KB (+0.3%)

Metric main This PR Δ
Total dist 5051.5 KB 5066.8 KB +15.3 KB (+0.3%)
JS 1912.0 KB 1917.7 KB +5.8 KB (+0.3%)
Sourcemaps 3009.4 KB 3018.9 KB +9.5 KB (+0.3%)
Type defs 130.2 KB 130.2 KB —
Other 0.0 KB 0.0 KB —

dist/ only; native binaries are versioned separately and not counted here.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
varlock-website 2760190 Commit Preview URL

Branch Preview URL
Oct 13 2026, 09:12 PM

@pkg-pr-new

pkg-pr-new Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

varlock

npm i https://pkg.pr.new/varlock@1217

@varlock/native-helper-darwin

npm i https://pkg.pr.new/@varlock/native-helper-darwin@1217

@varlock/native-helper-linux-arm64

npm i https://pkg.pr.new/@varlock/native-helper-linux-arm64@1217

@varlock/native-helper-linux-x64

npm i https://pkg.pr.new/@varlock/native-helper-linux-x64@1217

@varlock/native-helper-win32-x64

npm i https://pkg.pr.new/@varlock/native-helper-win32-x64@1217

commit: 7eec5ed

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

The detached sender can trigger an unbounded process-spawning loop when Bun preloads varlock/auto-load. Restore the existing CLI-child guard on the sender before merging.

Reviewed changes: Reviewed the telemetry queue, exit handoff, hidden sender entry point, and related documentation and tests.

  • Deferred sending: Events gain a UUID and timestamp and remain queued until an unref'd 200ms timer starts in-process delivery.
  • Detached handoff: A synchronous exit hook transfers unconfirmed events and pending schema usage to a detached CLI sender with ignored stdio and a 10-second request limit.
  • Coverage and docs: Tests cover queue and handoff behavior, and the guide describes background delivery. All 75 focused tests passed after building the required workspace dependencies; the deployed Cloudflare relay's batch handling was not verified.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using gpt-6.1-sol | 𝕏

…t recursion

The sender now reads its events from a temp file (deleted once read) instead
of an env var, so the payload has no size limit and is not visible in ps; the
env var remains only as a fallback when the temp dir is unwritable. The sender
is spawned with VARLOCK_TELEMETRY_DISABLED set so it can never record or spawn
telemetry of its own. The telemetry guide now describes the handoff step by step.
The payload is a couple of small anonymized events and the environment is
readable only by the same user, so the temp file only added a disk write at
exit, a fallback path, and litter when a sender is killed before reading it.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The new temp-file handoff leaves files behind when the detached sender cannot be started. The existing preload-guard thread also remains partially addressed: the opt-out flag stops the unbounded telemetry chain, but sender startup can still re-run auto-load.

Reviewed changes: Reviewed the changes since 83b2688f to the sender payload transport and telemetry opt-out behavior.

  • Moved payloads to files: Wrote pending events to a user-readable-only temp file, passed its path to the sender, and retained an environment fallback when writing fails.
  • Disabled sender capture: Set VARLOCK_TELEMETRY_DISABLED=1 on detached senders while allowing delivery of handed-off events.
  • Expanded tests and docs: Covered file delivery, deletion, and environment fallback, and documented the handoff lifecycle. All 76 focused tests passed after building the required workspace dependencies.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using gpt-6.1-sol | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues found in this delta. Approval remains withheld because the existing preload-guard thread is still partially addressed.

Reviewed changes: Reviewed the changes since 0ece447a to the detached sender's payload transport.

  • Moved handoff to the environment: Replaced temp-file delivery and its fallback with __VARLOCK_TELEMETRY_EVENTS, removing disk writes and orphaned-file failure paths.
  • Updated sender coverage and docs: Removed file-path handling and cleanup tests, and documented environment-only delivery. The Varlock build and all 72 focused telemetry and command-tracking tests passed.

Pullfrog  | Fix it ➔ | View workflow run | Using gpt-6.1-sol | 𝕏

…oad bails

The CLI clears __VARLOCK_CLI_CHILD from its own env at startup, so the detached
sender was spawned without it. Under bun with bunfig preload = ["varlock/auto-load"]
the sender would have run auto-load, spawning a varlock load whose exit spawns
another sender, forever. Restore the marker on the sender's env.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes: Reviewed the changes since 27601905 to the detached sender's preload guard.

  • Restored the CLI-child guard: Set CLI_CHILD_MARKER on the detached sender so Bun-preloaded varlock/auto-load skips resolution before sender dispatch.
  • Extended regression coverage: Asserted that the parent marker is absent and the sender marker is restored. All 74 focused telemetry and CLI-child marker tests passed after building the required workspace dependencies.

Pullfrog  | View workflow run | Using gpt-6.1-sol | 𝕏

@theoephraim
theoephraim merged commit e08600b into main Oct 13, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telemetry send on the load path adds a variable ~0.1-1s to every varlock load / auto-load

1 participant