QuickJS engine: threshold-based VM-memory snapshotting (WORKFLOW_SNAPSHOT_THRESHOLD) - #3251
QuickJS engine: threshold-based VM-memory snapshotting (WORKFLOW_SNAPSHOT_THRESHOLD)#3251TooTallNate wants to merge 2 commits into
Conversation
🦋 Changeset detectedLatest commit: c139c2a The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results
⏳ Tests are running... _Started at: _ ❌ Some tests failed ❌ Failed E2E Tests▲ Vercel Production (2 failed)example-quickjs (1 failed):
fastify-quickjs (1 failed):
📦 Local Production (2 failed)nextjs-turbopack-stable-node (1 failed):
nextjs-webpack-stable-node (1 failed):
E2E Test SummarySummary
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
❌ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
|
VaguelySerious
left a comment
VaguelySerious
left a comment
There was a problem hiding this comment.
AI review: blocking issues found
00d4d3e to
7f4d517
Compare
…KFLOW_SNAPSHOT_THRESHOLD)
…based PRNG fast-forward, unified host-callback list, lifecycle hardening - SnapshotMetadata gains eventCount, rngDraws and formatVersion. The max-events guard now compares restored total + delta (both at entry and per loop turn) — previously a run that kept snapshotting could never accumulate enough delta to trip the ceiling it exists for. - Correlation-id generation is position-based across snapshots: the runtime seeds from the BASE seed and fast-forwards the persisted draw count instead of mixing the snapshot cursor into the seed. Ids are now identical across snapshot generations AND identical to a no-snapshot run, so overlapping invocations straddling a snapshot save still collide on the world's dedup (new test pins restored ids == full-replay ids). Snapshots without a draw count fall back to full replay. - Host callbacks are declared in ONE list that drives both the fresh-boot install and the restore re-registration, so adding a callback can't silently skip the restore path. - Preloaded events are used again with snapshotting enabled (the first qualifying suspension skips its save — no cursor yet); short runs keep the zero-round-trip fast path. - Snapshot persist runs off the response path (waitUntil), with a 32MB plaintext size ceiling (skip + warn). Loads that fail format/shape checks warn instead of silently miming a miss. Terminal deletes are gated on a snapshot actually existing and now also fire on the runGone path; a server-side TTL remains the backstop for unobserved cancellations.
81784aa to
c139c2a
Compare
📊 Workflow Benchmarks❌ The benchmark run for Partial results from the failed run: commit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 382767ms → this run 371482ms (Δ -11285ms, -3%) 1020 steps (queue-hop) Cumulative STSO time: main 6857ms → this run 7633ms (Δ +776ms, +11%) ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — SO 250/500/1000 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
Note
Supersedes #3053, which GitHub auto-marked as merged (and auto-deleted the head branch) when a restacking mistake briefly force-pushed the head branch to the same commit as its base. Same content, freshly rebased on the stack.
Summary
PR 4 of the QuickJS VM roadmap: threshold-based VM-memory snapshotting — the middle ground that motivated reviving this effort (see #1298 / #1300 discussion). Instead of snapshotting at every suspension (the original branch's model, which cost ~25% on e2e wall clock), snapshots are taken only once
WORKFLOW_SNAPSHOT_THRESHOLDevents have been processed since the last one:How it works
WORKFLOW_SNAPSHOT_THRESHOLDenv var (default0= disabled) or per-runexecutionContext.snapshotThreshold, stamped atstart()for run affinity likeWORKFLOW_VM.session.snapshot()) → compress (zstd/gzip via the shared serialization pipeline; QuickJS heaps compress ~4×, measured 16.5 MB → 3.9 MB) → encrypt with the run's key when configured →world.snapshots.savewith the events cursor at the VM's feed frontier.world.snapshots.load→ decrypt → decompress →QuickJS.restoreover the cached WASM module, re-register host callbacks, fetch events from the snapshot's cursor and feed only the delta. Runs seamlessly through PR 2's inline continuation loop.Determinism model (restore + partial replay)
The threshold model's new mechanism vs. the original branch: a resumption may restore a snapshot older than the log head (suspensions since the snapshot weren't persisted) and must deterministically re-derive everything in between:
eventsCursor: the heap already consumed pre-snapshot draws, so re-seeding from the base would replay the first-N draws and collide with recorded correlationIds. The cursor is identical for every resume from the same snapshot (concurrent resumes still collide ids for the world's dedup) and advances only when a newer snapshot is taken.Validation
WORKFLOW_SNAPSHOT_THRESHOLD=1(maximum churn: snapshot on every qualifying suspension), wall clock within ~10% of the node baselinerestored: trueresumptions, save/restore/delete lifecycle, and threshold gating (threshold=100 short run ⇒ zero snapshots, pure replay)quickjs-snapshotmatrix leg (nextjs-turbopack, threshold=1) across local dev/prod/postgres e2e jobsNotes / follow-ups
quickjs-wasibuild that produced them. Per the deployment contract (runs continue on the version they started on — free on Vercel), this is a non-issue in production; environments without skew protection are covered by the restore-failure fallback to full replay.WORKFLOW_SNAPSHOT_THRESHOLDsection added to v5 Runtime Tuning.