Skip to content

perf(continuity): stream first user prompt instead of parsing the whole transcript#63

Merged
alexgreensh merged 1 commit into
alexgreensh:mainfrom
danikdanik:fix/continuity-first-prompt-streaming
Jun 16, 2026
Merged

perf(continuity): stream first user prompt instead of parsing the whole transcript#63
alexgreensh merged 1 commit into
alexgreensh:mainfrom
danikdanik:fix/continuity-first-prompt-streaming

Conversation

@danikdanik

Copy link
Copy Markdown
Contributor

Why

firstUserPromptFromSession resolved the first user message by calling loadMessagesFromSessionFile, which reads and JSON-parses every line of the session transcript. It only needs the first user message (which sits near the top of the file), and it runs on the cross-session continuity path that can fire on several early session:patch events before the first user prompt has landed on disk — so each of those calls re-parsed an ever-growing transcript.

What

Replace that call with a streaming reader (firstUserPromptFromSessionFile) that reads the file in 64 KB chunks and stops at the first non-empty user message:

  • A StringDecoder buffers partial multi-byte UTF-8 sequences across chunk boundaries, so text is never corrupted mid-character.
  • The line-level parse (userTextFromJsonlLine) mirrors loadMessagesFromSessionFile's content handling (plain string or array of text blocks), so behavior is unchanged.
  • loadMessagesFromSessionFile remains imported — it is still used elsewhere.

How (verified)

Verified equivalent to the previous implementation across 6 cases: assistant-first ordering, array-block content, tool_result-only user turns, no-user files, a multi-byte char forced onto a chunk boundary, and a 5000-line file where it early-exits. tsc build is green; committed dist/ rebuilt.

…le transcript

Why: firstUserPromptFromSession() resolved the first user message by calling
loadMessagesFromSessionFile(), which reads AND JSON-parses every line of the
session transcript. It only needs the first user message (which sits near the
top of the file), and it runs on the cross-session continuity path that can
fire on several early session:patch events before the first user prompt has
landed on disk -- so each of those calls re-parsed an ever-growing transcript.

What: replace that call with a streaming reader (firstUserPromptFromSessionFile)
that reads the file in 64KB chunks and stops at the first non-empty user
message. A StringDecoder buffers partial multi-byte UTF-8 sequences across
chunk boundaries so text is never corrupted mid-character. The line-level
parse (userTextFromJsonlLine) mirrors loadMessagesFromSessionFile's content
handling (plain string or array of text blocks), so behavior is unchanged.
loadMessagesFromSessionFile is still imported -- it remains in use elsewhere.

How: verified equivalent to the previous implementation across 6 cases
(assistant-first ordering, array-block content, tool_result-only user turns,
no-user files, a multi-byte char forced onto a chunk boundary, and a
5000-line file where it early-exits). tsc build green; dist rebuilt.
@alexgreensh alexgreensh merged commit 9099b65 into alexgreensh:main Jun 16, 2026
1 check passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 16, 2026
@alexgreensh

Copy link
Copy Markdown
Owner

Merged in v5.11.13. Streaming the first prompt instead of parsing the whole transcript is a clean win, and the UTF-8 chunk-boundary handling checks out. Thanks!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants