Add world.snapshots storage interface (local, postgres, vercel) - #3250
Add world.snapshots storage interface (local, postgres, vercel)#3250TooTallNate wants to merge 2 commits into
Conversation
🦋 Changeset detectedLatest commit: 968cfa1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results
⏳ Tests are running... _Started at: _ ❌ Some tests failed ❌ Failed E2E Tests▲ Vercel Production (3 failed)hono-quickjs (1 failed):
nuxt-quickjs (1 failed):
vite-quickjs (1 failed):
📦 Local Production (1 failed)nextjs-webpack-stable-node (1 failed):
📋 Other (1 failed)e2e-vercel-prod-tanstack-start-quickjs (1 failed):
E2E Test SummarySummary
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
❌ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 📋 Other
✅ vercel-multi-region
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
a4e1f58 to
c935825
Compare
…l implementations
…torage
The security bot flagged path traversal: an unvalidated runId was
interpolated directly into {basedir}/snapshots paths, letting ../ (or /,
\, NUL, absolute fragments) escape the snapshots dir on save/load/delete.
Apply the same two-layer scheme as the other world-local storages:
assertSafeEntityId as the primary defense plus resolveWithinBase
containment at the join. Hostile-runId test covers all three operations.
ff2c6e4 to
968cfa1
Compare
📊 Workflow Benchmarks❌ The benchmark run for Partial results from the failed run: commit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 382767ms → this run 371563ms (Δ -11204ms, -3%) 1020 steps (queue-hop) Cumulative STSO time: main 6857ms → this run 7908ms (Δ +1051ms, +15%) ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — SO 250/500/1000 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
Note
Supersedes #3050, which GitHub auto-marked as merged (and auto-deleted the head branch) when a restacking mistake briefly force-pushed the head branch to the same commit as its base. Same content, freshly rebased on the stack.
Summary
PR 3 of the QuickJS VM roadmap: the
world.snapshotsstorage interface and its implementations, ported from the original snapshot-runtime branch (#1300). Inert until PR 4 — nothing in the runtime calls these APIs yet.What's included
@workflow/world:Storage.snapshotsinterface —save(runId, data, metadata)/load(runId)/delete(runId)— plusSnapshotMetadata(eventsCursor,createdAt). The cursor is storage-layer metadata stored alongside the opaque snapshot bytes so a restore only fetches events recorded after the snapshot.@workflow/world-local: filesystem storage under{basedir}/snapshots/({runId}.bin+{runId}.jsonsidecar), with round-trip unit tests.@workflow/world-postgres:workflow_snapshotstable (migration0016, renumbered from the original branch to follow main's0012–0015).@workflow/world-vercel: client for the workflow-server snapshot endpoints (PUT/GET/DELETE /v2/runs/:runId/snapshot, binary body + metadata headers — endpoints are live in workflow-server since vercel/workflow-server#339). Includes:undici.request()(notfetch()) PUT path so Buffer bodies survive RetryAgent retries (Retry Handler Fails When Request Has a Body nodejs/undici#3288),makeRequest, per the repo's trace-propagation rule), covered by newtrace-propagation.test.tscases.Notes
@workflow/core's responsibility (PR 4's save pipeline); worlds must not add their own compression (ciphertext doesn't compress).Storageis a breaking change for community World implementations — appropriate for the v5 beta line; called out in the changeset.WORLD_SNAPSHOT_DIAGwarn logging from the original branch was demoted toconsole.debug.