[kriszyp]

Companion to Cut off a JSON response that fails mid-stream: how a streamed JSON response reports a failure after its 200, and the storage.maxTransactionOpenTime limit that commonly causes one.

⊙ Problem

From v5.4.0, a JSON response that fails mid-stream is cut off instead of ending as a complete 200. Clients need to know a cut-off body is a failed read. The usual cause, a client pausing a large read past storage.maxTransactionOpenTime, is not documented, and neither is ReadSnapshotExpiredError.

🤔 Is this the right scope?

Yes. Every claim was checked against core main and the core PR's diff. The setting turned out to be RocksDB-only (LMDB has a fixed 30 s), so the docs say so.

💡 Solution

• A new When a Streamed Response Fails section on Content Types: the JSON cut-off, what clients should do, and the retry risk for writes. Review most closely.
• A storage.maxTransactionOpenTime entry in Storage Tuning, with advice for slow readers, plus its configuration index line.

🔧 Changes

• export_local with format: json now ends its job ERROR and removes the partial file, in Jobs and Operations.
• Updated the full-text and REST paging reference pages (REST Querying, Query Optimization, Full-Text Search Overview, and Full-Text Search Querying) so JSON-array failure guidance and the v5.4 upgrade checklist cover cut-off/aborted streams and missing terminal diagnostics; remaining sibling-page drift outside reference/ (learn/developers/full-text-search-product-catalog.mdx:185) and an inline version note on reference/configuration/options.md:334 are left out of this PR.
• A 5.4 release note.

🤔 CSV is left out

A failed CSV stream currently hangs instead of aborting. That is a core bug (harper#3658), so the docs make no claim for CSV. The MessagePack note covers aborting without an error record.

🤔 Milliseconds only

Core reads maxTransactionOpenTime as a raw number; a duration string like 30s breaks the monitor. The entry says to use milliseconds. The core fix (parse it like maxTransactionQueueTime) is reported for triage, not made here.

✅ Verification

• npm run build, format:check and lint pass. New anchors (#when-a-streamed-response-fails, #storagemaxtransactionopentime, #limitend-or-limitstartend) resolve in the built HTML.
• Facts traced in core: ReadSnapshotExpiredError and the 422 abort (resources/DatabaseTransaction.ts), the monitor's one-to-two interval timing, LMDB's fixed 30 s (resources/LMDBTransaction.ts), the job ERROR path (server/jobs/jobProcess.ts), and the NDJSON/SSE record shape (server/serverHelpers/contentTypes.ts).
• Badges are v5.4.0: no v5.4.0 tag exists yet and the core PR targets main with milestone v5.4.
• Nothing was run against a live Harper; the behavior is unmerged core code.

Depends-on: HarperFast/harper#3664

Refs HarperFast/harper#2392

_{Claude Opus 5.5 (dispatch dev agent)}

🤖 Generated with Claude Code

Related PRs: #691 independent, #739 independent, #754 independent, #772 overlaps

Deferred findings

• 928eef2f738616e1 minor reliability — reference/rest/content-types.md — CSV streamed-failure behavior undocumented; core toCsvStream drops the source error — NOT FILED
• 855af50e91dbc6af minor reliability — reference/database/storage-tuning.md — Core reads maxTransactionOpenTime raw; a duration string spins a 1ms monitor and disables expiry — NOT FILED
• f802001270de35e1 nit documentation — reference_versioned_docs/version-v4/database/jobs.md — Job status list omits CREATED (v5) and ERROR (archived v4 copy) — NOT FILED
• 17c9faa6deaf2092 nit maintainability — reference/database/storage-tuning.md — Open documentation#772 edits the same page; merge-order conflict risk — NOT FILED
• 54e0824c2f2c92e1 nit documentation — reference/rest/server-sent-events.md — SSE guide omits named terminal-error handling — NOT FILED
• 3f1609c476fffb13 nit documentation — reference/database/jobs.md — S3 JSON export failure contract is undocumented — NOT FILED
• 0e0b7c81e1eb85d9 nit documentation — reference/operations-api/operations.md — Operations API omits S3 export failure behavior — NOT FILED

Origin — the dispatch brief this PR was written from

Document cut-off JSON streams and maxTransactionOpenTime for long streaming reads

Document how Harper reports a streamed JSON response that fails after it has started, and the storage.maxTransactionOpenTime limit for long streaming reads. Code change: HarperFast/harper#3664 (fixes HarperFast/harper#2392), targeted at v5.4.

What changed (from the PR, verify against its diff):

• A REST collection query (or any resource returning an iterable) served as application/json streams a JSON array. If the source fails after the 200 head is sent, Harper now writes an {"error": "<Name>: <message>"} element as a best-effort diagnostic, omits the closing ], and aborts the connection (HTTP/1.1: no chunked terminator; HTTP/2: stream reset). Before, the array was closed and the response ended as a normal 200, so a partial result looked complete.
• Clients should treat a cut-off body (network error, res.json() rejecting) as a failed read. HTTP 200 alone does not mean the stream completed.
• NDJSON ($harperStreamError record) and SSE (event: harper-error) are unchanged.
• A mutating request (e.g. POST) that returns a stream has already committed its writes before the stream fails; retrying it can repeat them. The diagnostic keeps replicationUnconfirmed: true / retryable: false when they apply.
• export_local with format: json now rejects when the query fails mid-export and removes the partial file it wrote.
• Long streaming reads: a streamed read holds a read snapshot. If the client stops reading for longer than storage.maxTransactionOpenTime (default 30000 ms), the transaction monitor closes the stalled read; when the client resumes, the read fails with ReadSnapshotExpiredError (status 503, but the head is already sent, so the response is cut off as above). Advice: read promptly, raise the limit, or page with limit/offset.

Where: the REST/HTTP reference (streaming responses / content negotiation), the storage.maxTransactionOpenTime configuration entry, and the export operations reference. Neither maxTransactionOpenTime nor ReadSnapshotExpiredError appears in the docs today. Mark the behavior with the v5.4 version badge used elsewhere. Keep it short; no customer names.

_{Dispatch: task docs-harper-3664 · queued by task:harper-rest-stream-error-trailer · ran by claude/opus/high · worker kzyp-xps-1}

_{Review-Coverage: authored=gemini; ran=cursor-composer,codex; pending=cursor-grok; late=cursor-grok:pending; adjudicated=domain,domain(codex); blocked=claude(quota); declined=muse; rounds=7; full=2 @ dbe8661}

_{Review-Attention: read ~3m (decisions: centralize-stream-contract, export-scope; raised: degraded review) @ dbe8661}

[gemini-code-assist]

Code Review

This pull request updates the documentation to reflect changes in version 5.4.0, including the introduction of the maxTransactionOpenTime configuration option, new job status values, and updated behavior for failed mid-stream JSON responses and local exports. The review feedback focuses on improving documentation consistency and style, specifically suggesting the addition of a version badge for the new configuration option and recommending the use of plain text instead of inline VersionBadge components within running prose.

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-762

This preview will update automatically when you push new commits.

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-762

This preview will update automatically when you push new commits.