feat(durable): send, wait for, publish, and subscribe to durable messages

[subnetmarco]

Tracking

Tracks: volcano-hosting#1826 (server support for durable messages)

Summary

• Authoring. ctx.waitForMessage(name, { timeout }) suspends until a message with that name arrives, and resolves null on timeout. ctx.publish(name, data) is a checkpointed step that resolves { id, seq }. The first publish in a context opens a messages session through a callback (3 operations, once). Each publish sends sha256(session token):position as its idempotency key, where the position counts publishes in that context, so a rerun after a crash returns the original message. Publish data with a NUL in any string or key is refused before the step runs, as in Python.
• Client. volcano.durable.messages.send(target, name, data, { idempotencyKey }), list(target, { direction, name, after, limit }) and subscribe(target, handler, { realtime }). target is { functionName, executionId, clientToken?, projectId? }; with projectId the calls use the project route and the session, without it the application route and the client credential.
• Subscribe follows the execution's durable: realtime channel, delivers in seq order once each, and lists from the last seq after each resubscription to fill what the channel missed. The channel sends the client token in its subscribe request; this isn't exposed on the public realtime API.
• The execution start returns now includes client_token when that start created it.
• Approvals and message callbacks now share one platform request path (durable-platform-request.ts), with the same 30-second retry on brief refusals.
• OpenAPI synced from Hosting, with the new operations added to the orval allowlist. The README, docs/durable-functions.md and docs/realtime.md are updated.
• Purely additive. Release Please should cut 2.1.0, which the Hosting docs name as the minimum for waitForMessage and publish.

Verification

Run under Node 22.23.3 (.node-version):

• pnpm quality:policy && pnpm quality:checks: audit, lint, unused code, OpenAPI drift, types, tooling, 3,245 Jest tests, typed runtime, package quickstart and examples
• Stryker over the 14 changed durable and realtime modules. The first run left survivors in four of them (durable-message-stream, durable-message-response, durable-messages-facade, durable-platform-request); after the fixes those four score 100.
• Under Node 24, the tooling test "native reporter preserves cancelled case" (node-completeness.test.mjs) times out; it passes on 22, which CI pins. This PR doesn't touch .quality-tools.

Cross-language impact

Classify this change: public facade / wire contract / shared behavior

• Requirement IDs and affected behavior: SDK-DURABLE-011 to 020, covering message waits (queued and live), timeout, published-message order, gap-free subscribe, user isolation, client-token anon sends, idempotent sends, ended executions, and read-only tokens.
• JavaScript / Python / Ruby companion PRs, or why each is unaffected: volcano-sdk-python#391 (authoring and client) and volcano-sdk-ruby#343 (client only, since Ruby has no durable authoring API).
• Canonical scenarios, bindings and active/staged copies: tests/sdk-contract/features/durable-messages.feature, with bindings for all three languages, in volcano-hosting#1827.
• Public examples updated in each affected language: docs/durable-functions.md, docs/realtime.md and the README here; the Python and Ruby docs in their PRs; docs/public/functions/durable-messages.md in Hosting.
• Native checks and shared acceptance evidence (with tested revisions): native checks above on 1495a2d. Shared acceptance hasn't run yet; it runs from volcano-hosting#1827 once the three SDK PRs merge and volcano-hosting#1826 is on staging.

[subnetmarco]

Read the authoring side (durable-message.ts, the durable.ts context wiring, durable-platform-request.ts), the stream and the facade. The hold/catch-up/release logic in DurableMessageStream is careful work. Comments:

• The mutation gate is red, and it's a real finding. Shard 8 has 8 timeouts: durable-message-stream.ts:73-87 (the readPages loop) and durable-platform-request.ts:118. If a mutant flips the has_more/empty-page exit or the closed-code check, the loop never ends or the request retries for 30s, so the tests hang instead of failing. Bounding the loop helps the code too: if a page doesn't advance lastSeq, stop rather than re-request it forever. And give the closed-code test a fake clock (or a responder that fails after a couple of calls) so a retrying mutant gets killed instead of timing out.
• Gap handling depends on #1826. deliver drops seq <= lastSeq, so a broadcast the server loses, or two that arrive out of order, are skipped permanently on an open subscription (details on #1826). Whatever the server ends up sending to make gaps detectable, the stream needs to catch up when it sees one.
• One session per context. durableContext() calls messagePublisher for every child, map item and parallel branch, and the server caps sessions at 20, so ctx.map over 21+ items that publish fails. Each session also costs 3 operations and a resume. If children can reuse the parent's session, key publishes by context path + position. If they can't, this needs a higher cap on the server.
• The synced OpenAPI brings unrelated changes. Sandbox session/execution memory_mb and the other new models come from hosting main, not from this feature. That's fine if intended, but the changelog entry from Release Please will credit them to durable messages. A separate chore: sync openapi commit would keep the history honest.

[subnetmarco]

Addressed in a45a877.

• Mutation gate. It's now at 100% on durable-message-stream.ts, durable-message-response.ts and durable-platform-request.ts, with no timeouts. The fixes are in the code as well as the tests:
  • readPages stops when a page delivers nothing new, not only when has_more is false. A page that doesn't advance lastSeq is never requested again.
  • postToPlatform returns a success before it decides whether to retry. A mutant can no longer turn a success or a final refusal into a 30s retry loop.
  • The list mocks answer once, so a read that keeps going fails instead of spinning. The runtime test's responder returns a final 400 for any URL it doesn't know.
• Gaps. hosting#1826 now sends each broadcast with prev_seq, the seq published before it. When prev_seq > lastSeq, the stream holds the live message, catches up from the list, then releases it. A failed catch-up still releases the held message. Resubscribing after a reconnect also catches up. Tests cover a lost broadcast, out-of-order arrival, and a non-integer prev_seq (read as no gap).
• Sessions per context. We kept one session per context and raised the cap on hosting to 1,000 sessions over the execution's life, the same as the publish cap, so the 3,000-operation limit is the real bound. Reusing the parent's session would have needed publish keys built from the context path and more replay handling in every SDK. The hosting docs and the agent skills give the per-item cost and tell users to publish from the parent across a ctx.map.
• OpenAPI sync. It's already a separate commit, 9c157af chore(api): sync OpenAPI contract. This repo only allows squash merges, though, so the separate commit won't survive into the changelog. The sandbox memory_mb changes it brings in only touch src/generated, which is internal; no public facade changes with them. If you'd rather the history credit them separately, I can open a sync-only PR from hosting main to land first and rebase this branch onto it. That needs a force-push here, so say if you want it.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

AI review: no P0-P2 findings

Reviewed 5f9a30f. Run log

Adds durable messages to the SDK. ctx.waitForMessage and a checkpointed ctx.publish (with a per-context messages session and replay-stable idempotency keys) are added to the authoring API. On the client, volcano.durable.messages.send/list/subscribe is added, with a gap-filling realtime stream. Approval registration is folded into a shared platform-request helper. I read the hand-written source closely: the publish ordering, session reuse, NUL and size checks, the retry and close paths of the stream's catch-up, and the realtime channel wiring. I found no defects. The main risk is the new credential handling (client tokens and session tokens) and the reliance on server behavior in volcano-hosting#1826.

Not checked: The AI review read the diff, the code, and the PR's text; it ran no tests or builds and read nothing outside this repository.

• That Hosting completes session and wait callbacks with the payload shapes the SDK parses, and broadcasts prev_seq on durable: channels.
• That the realtime server accepts client_token in subscribe data for durable: channels and enforces the visibility rules the docs describe.
• The shared acceptance scenarios in volcano-hosting#1827 have not run against this revision.
• That the docs' links to /platform/functions/durable-messages and the volcano durable messages send CLI command exist where they point.
• Verify integration at this head against Kong/volcano-hosting#1826 and #1827, particularly token isolation, replay idempotency, and realtime gap recovery.

Human review: needed

• Should an execution's client_token travel in the realtime subscribe payload (realtime-client.ts) and as X-Volcano-Client-Token, given it lets an anon key reach the execution?
• Should app and anon credentials be able to message executions and read their output (durable-messages-facade.ts), when starting was their only durable operation until now?
• Should execution client tokens authorize message sends, reads, and realtime subscriptions, and callback-issued session tokens authorize publishing (src/durable-messages-facade.ts, src/durable-message…

The AI review found no P0-P2 problems at 5f9a30f, so the code needs no second review for bugs. Decide the points above: approve to accept them, or request changes.

Suggested reviewer: swkeever (primary maintainer; touched the realtime docs, OpenAPI sync and build config this PR changes). Any collaborator with write access other than the author can approve.

[subnetmarco]

Second pass at 17c8532. The earlier points are resolved:

• The mutation gate is at 100% with no timeouts.
• readPages stops at a page that delivers nothing new.
• The stream holds a message and catches up when prev_seq shows a gap.
• Hosting raised the session cap to 1,000.

I re-read the stream, the facade, durable-message.ts, durable-platform-request.ts and the context wiring, and checked them against hosting#1826. Nothing blocking:

• I traced the gap handling against the server's numbering, including a racing rerun that numbers a seq it never inserts. In that case the next broadcast's prev_seq points to a seq that doesn't exist. The catch-up lists past it and delivers the real message, then drops the held copy as already seen. Good.
• On the AI review's open questions:
  • hosting#1826 never ends a durable: subscription itself: nothing in internal/realtime unsubscribes when an execution finishes. So leaveIfServerEnded only logs on a real refusal.
  • The session token isn't exposed through the API either. The trace returns operation kind and timing, not callback results.
• The description says each publish sends its step's operation id as the idempotency key. The code uses sha256(session token):position, which is the right choice, but the description should say so. The Python PR has the same wording.
• When readsAnswer is false, postToPlatform returns without touching the response body. In Node, an unread fetch body holds its connection until GC; await response.body?.cancel() before settled() frees it. It's only a few requests per execution, so this is minor.
• Parity nit: Python's ctx.publish refuses data containing a NUL before anything runs. Here the server answers 400 instead, after the session has opened, and the publish step fails. The outcome is the same, it just fails later.

[subnetmarco]

Addressed in fc42644.

• Unread answers. When the answer isn't read, postToPlatform now cancels the body before it settles. A failed cancel is ignored: Volcano already answered, so it's no reason to send the request again. Tests cover a cancelled body, a cancel that throws, and a 201 with no body.
• NUL parity. ctx.publish now refuses data with a NUL in any string or key before the session opens, with ctx.publish() data must not contain NUL characters. The check walks the data rather than searching the JSON text, so a string that spells out \u0000 still goes through. docs/durable-functions.md lists it with the other refusals.
• Description. It now says the publish key is sha256(session token):position. I'm fixing the Python PR's wording too.

The mutation gate is at 100% on both changed files (329 killed). Lint, formatting, types and the full suite (3300 tests) pass.

On the AI review's CLI question: volcano durable messages send ships with Kong/volcano-cli#302, which merges with this feature.

[subnetmarco]

The mutation job failed in 77cca25's parent because Stryker counted a RuntimeError as an error, not a test failure. A subscribe retry left running by one mutant fired real timers during a later mutant's test in the same reused runner. The durable.messages.subscribe tests now run on fake timers, which end with each test. durable-message-stream.ts now scores 100% with no errors, and pnpm lint and pnpm test pass.