feat: LLM streaming with UX improvements #29

Jurij89 · 2026-02-08T13:29:39Z

Summary

SSE streaming
Server-sent events transport for real-time LLM responses on web (native path unchanged).
Includes server-side processStreamingCompletion with .stream() → .invoke() fallback, client-side SSE parser, and rAF-throttled UI updates.
Plain text during streaming
Renders raw <Text> instead of Markdown while tokens arrive, eliminating visual glitches from incomplete markup (unclosed fences, bold, partial links).
Switches to full Markdown on completion.
Code block copy button
Fenced code blocks display a copy icon (top-right) that swaps to ✓ Copied feedback for 2 seconds.
Scroll UX — user message to top on send
No auto-scroll during streaming (matches Claude/ChatGPT behavior).
When the user sends a message, viewport-relative spacers inside the ScrollView cause scrollToEnd() to position the user's message near the top of the viewport, giving the AI response a full page of space to stream into.
Top spacer (15% viewport) ensures consistent spacing from the nav bar for both first and subsequent messages. Bottom spacer (85% viewport) provides the scroll target. Spacers are hidden on the landing screen.

Files changed

File	What
`apps/agent/src/shared/chat.ts`	SSE types, `processStreamingCompletion` (server), `makeStreamingCompletionRequest` (client), `writeSSE` helper
`apps/agent/src/server/index.ts`	Streaming middleware plugin (intercepts `Accept: text/event-stream` on `/llm`)
`apps/agent/src/app/(protected)/chat.tsx`	`streamingContent` state, streaming functions, plain text rendering, scroll-on-send, viewport spacers
`apps/agent/src/components/Markdown.tsx`	`CopyCodeButton` component, custom `fence` render rule

Test plan

Send a message on web — verify tokens stream in real-time with plain text
Verify final message renders with full Markdown (bold, code blocks, lists)
Verify no auto-scroll during streaming — user can read at their own pace
Verify sending a new message scrolls user message to top of viewport
First message has same spacing from nav bar as subsequent messages
Verify code block copy button copies content and shows ✓ Copied feedback
Verify native (non-streaming) path is unaffected
Verify tool calls work correctly through the streaming path
Verify landing screen layout is unaffected (no spacers when no messages)

…provements - Render plain <Text> during streaming instead of Markdown to eliminate visual glitches from incomplete markup (unclosed fences, bold, etc.) - Add copy button with "✓ Copied" feedback to fenced code blocks - Remove auto-scroll during streaming and on completion so users can read at their own pace (matches Claude/ChatGPT behavior) - Scroll to bottom only when user sends a new message

Bojan131 · 2026-02-09T08:47:11Z

Tested and some markdown parts seem not to be working. Bolded text and code blocks dont seem to be visible. When you ask it to put it in a code block it does do it but it doesn't put it automatically and also it has a delay where it puts it in a code block when the whole message stops streaming instead of streaming the code into the code block

Please take a look at the image attached.
Other things work as expected

Replace the plain <Text> streaming renderer with the existing <Markdown> component so formatting (bold, code blocks, lists, etc.) appears progressively as tokens arrive, eliminating the visual "pop" when the stream completes. Add normalizeStreamingMarkdown() to close unclosed code fences mid-stream, and stripThinkTags() to hide <think> blocks during streaming. Remove now-dead streamingTextStyle.

zsculac · 2026-02-10T14:13:02Z

apps/agent/src/app/(protected)/chat.tsx

+    }
+  }
+
+  async function sendMessageStreaming(newMessage: ChatMessage) {


requestCompletionStreaming and sendMessageStreaming seem like very similar functions. It would be beneficial to extract a shared helper instead of having duplicate code

Good call — refactored this. Extracted a shared streamCompletion(messages) helper that contains all the streaming callback logic (delta accumulation, RAF-throttled UI updates, tool call handling, KA content gathering, error handling). Both requestCompletionStreaming and sendMessageStreaming now delegate to it. This also fixed a subtle bug where sendMessageStreaming was missing the KA content gathering logic that requestCompletionStreaming had.

zsculac · 2026-02-10T15:40:49Z

apps/agent/src/shared/chat.ts

+          currentData = "";
+        }
+      }
+    }


If the server stream ends without sending an explicit event: done (e.g. server crash, network drop), onDone is never called. This leaves the UI stuck: streamingContent is never cleared, isGenerating stays true, and the input remains disabled. Consider calling onDone() as a fallback after the while loop exits, or at minimum call onError so the UI can recover.

Great catch — fixed. Added a streamFinalized flag in makeStreamingCompletionRequest that gets set when either a done or error SSE event is received. After the read loop exits, if the flag is still false (server crash, network drop), we call onError("Connection lost — the server stopped responding") as a fallback so the UI can recover. Also added a finally block in the caller to cancel any pending requestAnimationFrame on unexpected errors, preventing stale UI updates.

zsculac · 2026-02-10T15:42:36Z

apps/agent/src/app/(protected)/chat.tsx

+              {isGenerating && streamingContent === null && <Chat.Thinking />}
+              {streamingContent !== null && (
+                <Chat.Message icon="assistant">
+                  <Markdown>
+                    {normalizeStreamingMarkdown(stripThinkTags(streamingContent))}
+                  </Markdown>
+                </Chat.Message>
+              )}


There's no auto-scroll to follow the streaming content as it arrived. If the response is long enough to overflow the view, the user has to manually scroll down. Consider adding a throttled scrollToEnd in the onDelta callback, and/or a scrollToEnd in onDone

Good catch, but this is actually intentional! Both Claude and ChatGPT follow the same pattern — they don't force auto-scroll during streaming. The reasoning is that if a user scrolls up to re-read earlier content while a response is still generating, auto-scrolling would yank them back down, which is disruptive. The user stays in control of their scroll position, and can scroll down manually when they're ready to see the latest output.

…eam drop recovery - Extract shared `streamCompletion()` helper to eliminate duplication between `requestCompletionStreaming` and `sendMessageStreaming` (also fixes missing KA content gathering in the latter) - Add `streamFinalized` flag in SSE client parser so the UI recovers when the server stream ends without an explicit done/error event (crash, network drop) - Add `finally` block to cancel pending requestAnimationFrame on unexpected errors, preventing stale UI updates

Add viewport-relative spacers to the chat ScrollView so that scrollToEnd() positions the user's new message near the top of the viewport instead of the bottom, giving the AI response a full page of space to stream into. - Track ScrollView height via onLayout - Bottom spacer (85% viewport) gives scrollToEnd room - Top spacer (15% viewport) matches spacing for the first message - Spacers hidden on landing screen (messages.length === 0)

Jurij Skornik added 2 commits February 8, 2026 12:56

feat: Added LLM streaming for a better UX

72ad10b

Bojan131 and others added 2 commits February 9, 2026 10:32

update package-lock and update package-lock github actions tests

1837e30

Balki-OriginTrail approved these changes Feb 10, 2026

View reviewed changes

Bojan131 added 2 commits February 10, 2026 12:12

delete fallback model if LLM_MODEL not defined

4cf3584

fix github actions tests

e72e2ae

zsculac reviewed Feb 10, 2026

View reviewed changes

Bojan131 added 5 commits February 10, 2026 15:14

fix github actions tests for LLM_MODEL in yml file

570d92f

fix github actions tests

3ee5437

fix github actions tests

5d2b3ab

fix github actons tests

2374813

fix github tests

8bfecfc

zsculac reviewed Feb 10, 2026

View reviewed changes

Bojan131 and others added 5 commits February 10, 2026 17:25

fix github actions tests

2e88cd0

fix github actions tests

c08d404

final fix github actions tests

06ca9f2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: LLM streaming with UX improvements #29

feat: LLM streaming with UX improvements #29

Uh oh!

Jurij89 commented Feb 8, 2026 •

edited

Loading

Uh oh!

Bojan131 commented Feb 9, 2026 •

edited

Loading

Uh oh!

zsculac Feb 10, 2026

Uh oh!

Jurij89 Feb 10, 2026

Uh oh!

zsculac Feb 10, 2026

Uh oh!

Jurij89 Feb 10, 2026

Uh oh!

zsculac Feb 10, 2026

Uh oh!

Jurij89 Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat: LLM streaming with UX improvements #29

Are you sure you want to change the base?

feat: LLM streaming with UX improvements #29

Uh oh!

Conversation

Jurij89 commented Feb 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Files changed

Test plan

Uh oh!

Bojan131 commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zsculac Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Jurij89 Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

zsculac Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Jurij89 Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

zsculac Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Jurij89 Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Jurij89 commented Feb 8, 2026 •

edited

Loading

Bojan131 commented Feb 9, 2026 •

edited

Loading