fix: version stream tail cursors across worlds - #3475
Conversation
🦋 Changeset detectedLatest commit: e15cc02 The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
NathanColosimo
changed the title
🧪 E2E Test Results❌ Some tests failed ❌ Failed E2E Tests▲ Vercel Production (27 failed)astro-node (1 failed):
astro-quickjs (2 failed):
example-node (1 failed):
example-quickjs (1 failed):
express-node (1 failed):
express-quickjs (1 failed):
fastify-node (1 failed):
fastify-quickjs (1 failed):
hono-node (1 failed):
hono-quickjs (1 failed):
nest-node (1 failed):
nest-quickjs (1 failed):
nextjs-turbopack-node (1 failed):
nextjs-turbopack-quickjs (1 failed):
nextjs-webpack-node (1 failed):
nextjs-webpack-quickjs (1 failed):
nitro-node (1 failed):
nitro-quickjs (1 failed):
nuxt-node (1 failed):
nuxt-quickjs (1 failed):
sveltekit-node (1 failed):
sveltekit-quickjs (1 failed):
tanstack-start-node (1 failed):
tanstack-start-quickjs (1 failed):
vite-node (1 failed):
vite-quickjs (1 failed):
E2E Test SummarySummary
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ vercel-multi-region
|
📊 Workflow Benchmarkscommit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 194368ms → this run 183701ms (Δ -10667ms, -5%) 📜 Previous results (4)4f319ccThu, 13 Aug 2026 00:17:06 GMT · run logs
5d22ad6Wed, 12 Aug 2026 20:58:44 GMT · run logs
b827cd8Wed, 12 Aug 2026 17:33:55 GMT · run logs
69e8f56Tue, 11 Aug 2026 21:58:51 GMT · run logs
ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
Sim WorldSimulated world deterministic testing for races. Traces 🟠 Mint-ordered log — 3 fail of 41 total
Full trace: 🟢 Append-only log — 0 fail of 41 total
Full trace: |
VaguelySerious
left a comment
VaguelySerious
left a comment
There was a problem hiding this comment.
AI review: blocking issues found
| /** Array of stream chunks in index order */ | ||
| data: StreamChunk[]; | ||
| /** Cursor for the next page, or `null` when no more pages are available */ | ||
| /** Position after the returned chunks, or `null` when none were returned or the stream ended */ |
There was a problem hiding this comment.
AI Review: Blocking
Redefining what cursor means on an already-published interface is a flag day for anyone whose World and consumer versions do not move together, and @workflow/world-local, @workflow/world-postgres and @workflow/web are independently versioned. I tested both skew directions against the world-local implementation in this PR rather than reasoning about it:
- New World, old consumer. The current
@workflow/webreader absorbs the old contract's final-page overlap by skipping as many frames as the previous response contained. With a tail cursor there is no overlap left, so it skips live chunks instead: 3 chunks render andskipFramesbecomes 3, two new chunks arrive,getChunkscorrectly returns just those two, and the reader renders neither. Silent data loss, including the terminal chunk of a completing stream. - Old World, new consumer. fix: resume open streams from tail cursors #3430 removes that skip and appends everything returned. Against an old-contract World the open tail has no cursor, so the reader cannot advance and re-appends the whole stream every poll. I measured 3 chunks becoming 12 rendered entries after two polls; it grows quadratically in poll count for as long as the run is active.
This also reaches third-party Worlds. docs/content/worlds/v4/building-a-world.mdx and the v5 copy are the contract community Worlds implement, and any World written against the current text returns cursor: null at the open tail, so it lands in the second case.
The additive alternative avoids all of this: return the checkpoint as a new field (resumeCursor or tailCursor) and leave cursor meaning "next page". Consumers opt in as they upgrade, old Worlds keep working with new consumers, and no version pairing becomes load-bearing. If cursor really has to change meaning in place, this needs a major on @workflow/world plus a peer-version floor on the consumers, not three patches.
| '@workflow/world-postgres': patch | ||
| --- | ||
|
|
||
| Return a checkpoint cursor after every non-empty open-stream chunk page so consumers can resume after the last delivered chunk without replaying it. |
There was a problem hiding this comment.
AI Review: Blocking
getChunks is public API and the docs still describe the old contract, so the doc updates belong in this PR (per the repo rule that user-facing changes ship with their docs):
docs/content/docs/v4/api-reference/workflow-runtime/world/streams.mdxand the v5 copy, around line 147:cursoris documented as "Cursor for the next page". Under this change it is a resume checkpoint that is present precisely when there is no next page.- Same files, around lines 201-209, document the pagination loop as
do { ... cursor = result.cursor ?? undefined } while (cursor). Under the new contract that loop terminates only when it observes an empty page, so on a stream producing chunks faster than the loop drains it never terminates and silently becomes an unbounded tail-follow. It needs to becomewhile (result.hasMore). docs/content/worlds/v4/building-a-world.mdxand the v5 copy: thegetChunksdescription for third-party World authors does not say when a cursor must be returned, and now it has to, since a World that gets this wrong breaks the dashboard reader rather than just paginating differently.skills/workflow/SKILL.mdaround line 694 shows the same{ limit, cursor }usage. Per CLAUDE.md, editing a skill file needs its frontmatterversionbumped.
One thing worth writing down explicitly wherever the contract lands: an empty page returns cursor: null, so consumers must retain their previous cursor rather than assign the new one. Every consumer in this stack independently reimplements some form of cursor = result.cursor ?? cursor, which is a good sign the contract should carry that rule rather than leaving each caller to rediscover it.
| /** Maximum number of chunks to return per page (default: 100, max: 1000) */ | ||
| limit?: number; | ||
| /** Opaque cursor from a previous response to fetch the next page */ | ||
| /** Opaque cursor from a previous response at which to resume */ |
There was a problem hiding this comment.
AI Review: Note
packages/world-sim/src/streams.ts implements getChunks too and was not updated, so it still returns cursor: next < state.chunks.length ? String(next) : null, which is the old contract exactly. That leaves the simulation World disagreeing with world-local, world-postgres and world-vercel on the one behavior this PR changes. It is the kind of gap that lets a consumer regression pass World Sim and fail everywhere else. Worth updating in the same change, or noting why it is exempt.
| data: resultChunks, | ||
| cursor: nextCursor, | ||
| cursor: | ||
| resultChunks.length > 0 && (hasMore || !streamDone) |
There was a problem hiding this comment.
AI Review: Note
Moving from "walk every file, count data files" to "index directly from startIndex" loses the EOF observation when the cursor is past the end. On main a cursor beyond the last data chunk still walked into the EOF marker and reported done: true; here the loop body never executes. I ran both:
main, cursor{"i":50}on a closed 1-chunk stream:{"data":0,"done":true,"cursor":null,"hasMore":false}- this PR:
{"data":0,"done":false,"cursor":null,"hasMore":false}
A consumer polling until done never terminates on that stream. It is not reachable from a cursor this implementation issued, since the highest index it hands out equals the data-file count and lands exactly on the EOF file, so it takes a cursor that outlived its chunk files: a .workflow directory cleared under a dashboard that still holds a cursor, or a cursor crossing tag scopes. Cheap to close if you want it airtight: after the loop, when !streamDone && startIndex >= chunkFiles.length, peek the last file for the EOF byte.
|
|
||
| for (const file of chunkFiles) { | ||
| for ( | ||
| let fileIndex = startIndex; |
There was a problem hiding this comment.
AI Review: Note
The Postgres streamer in this same PR gained real cursor validation (StreamCursorSchema with z.number().int().nonnegative()), but world-local still does startIndex = decoded.i unvalidated, with a catch that only covers base64 and JSON failures. That gap got worse rather than staying neutral, because startIndex now seeds a for loop bound instead of feeding < comparisons:
main, cursor{"i":"nope"}:{"data":1,"done":false}, falling back to reading from the start.- this PR:
{"data":0,"done":false,"cursor":null}, a permanently empty page. And becausecursorisnull, the consumer cannot tell it apart from an idle tail, so it never recovers.
Reusing the same schema shape you added for Postgres would make the two consistent and turn this into a clean restart-from-zero.
Signed-off-by: Nathan Colosimo <110621881+NathanColosimo@users.noreply.github.com>
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
2 participants
Description
Define a stream cursor as the checkpoint after a returned page. A non-empty page at the tail of an open stream keeps that checkpoint so a later read resumes after the delivered chunks. A completed final page returns cursor: null, including when its size exactly equals the requested limit.
Apply that contract to the Local, Postgres, Sim, and Vercel Worlds. The Vercel World reads chunk pages through the versioned v3 API. Local reads begin directly at the cursor index, while Postgres decodes one cursor containing both the row key and logical chunk index.
Release workflow and the three concrete Worlds together. The acyclic peer constraints are workflow -> Local/Vercel (optional) and Postgres -> workflow. Literal peers in both directions are not representable in this Turborepo package graph because workflow already depends on Local/Vercel through core, while Postgres test infrastructure already depends on workflow; reverse peer edges form build cycles.
This is the bottom PR in the stack; #3430 updates dashboard and core consumers on top.
How did you test your changes?
PR Checklist - Required to merge