Skip to content

Live chat and board SSE silently stop updating on slow connections #27

Description

@ujjwalredd

Summary

Both SSE broadcasters treat a false return from res.write() as a dead client and drop the subscriber. But res.write() returns false for backpressure — the socket buffer passed highWaterMark — on a connection that is perfectly healthy and still reading.

for (const subscriber of taskSubscribers) {
  if (!writeEvent(subscriber, event)) taskSubscribers.delete(subscriber);
}

Impact

The dropped response is never end()ed. The socket stays open, so EventSource never fires error and never reconnects. The result:

  • Live chat freezes mid-stream. The agent run continues and completes server-side, and the transcript is persisted to Hermes, but the browser shows a half-finished reply with the spinner stuck. The user has to reload to see that the task actually finished.
  • The board stops updating. Tasks stop moving between columns and new tasks never appear.

Neither recovers on its own.

Why it triggers in practice

The live-chat stream is well positioned to hit the threshold:

  • broadcastRunSnapshot() re-sends the entire accumulated run — every message, all thinking content, all tool events — after each goal turn (server/routes/chat.ts). Over a 20-turn goal run that is quadratic bytes on a single socket.
  • text_delta events stream continuously on top of that.
  • Any backgrounded tab, throttled connection, or slow reader accumulates the buffer until write() returns false.

Reproduction

Standalone script showing write() returning false on a live, undestroyed socket with a client that connects and reads slowly:

res.write() returned FALSE after 1114 events (~576.6 KiB)
  socket destroyed? false
  'close' fired?    false
  connection still alive 200ms later? true

At that moment the current code deletes the subscriber, and the client never receives another event.

Suggested fix

Real disconnects are already handled correctly by the 'close' handlers in subscribe() and addClient(), so the return value never needed to be acted on. Removing the check is a net-negative diff.

Happy to send a PR with a regression test.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions