Skip to content

feat(assemblyai): warn when audio stops flowing to the WebSocket#1

Closed
gsharp-aai wants to merge 1 commit into
dlange-aai:assemblyai/debug-loggingfrom
gsharp-aai:gsharp/stt-send-side-silence-log
Closed

feat(assemblyai): warn when audio stops flowing to the WebSocket#1
gsharp-aai wants to merge 1 commit into
dlange-aai:assemblyai/debug-loggingfrom
gsharp-aai:gsharp/stt-send-side-silence-log

Conversation

@gsharp-aai
Copy link
Copy Markdown

Summary

Extends the recv-silence warning with a symmetric send-side warning.

The recv-silence warning on its own can't distinguish "AssemblyAI is stalled" from "the plugin stopped receiving audio from upstream and is correctly silent." When a session appears wedged, both interpretations are on the table and the recv-side log alone can't separate them.

This adds the complementary check: inside the same 15s recv-silence block, if _last_frame_sent_at is also >15s old, log AssemblyAI no audio frames sent for Ns session=<id>. The two absence-of-activity warnings appear back-to-back during a stall, giving an unambiguous diagnosis:

send-stopped warning recv-silence warning Diagnosis
silent firing AssemblyAI stalled — frames flowing, no response
firing firing Upstream stalled — plugin got no audio
silent silent healthy

Implementation

  • Adds self._last_frame_sent_at: float | None = None, set on every successful ws.send_bytes(...) in send_task.
  • In recv_task, the existing if consecutive_timeouts % 3 == 0: block now also checks send-side idleness and logs the second warning when applicable.
  • No new watchdog coroutine; reuses the 15s cadence already established.
  • Guarded on _last_frame_sent_at is not None so it doesn't fire during the initial connect window before any audio has been sent.

Impact

  • Warning-level only, and only when both sides are idle together.
  • Healthy sessions: silent.
  • Overhead: one time.time() assignment per audio frame.

Test plan

  • make format lint type-check pass
  • Healthy session: no send-stopped warning emitted
  • Simulated upstream stall (close input_ch while keeping WS open): both warnings fire at 15s, 30s, 45s
  • Simulated AAI stall (mock WS that accepts but never responds): recv-silence fires, send-stopped stays silent

Complementary to the recv-silence warning: if frames are still being
pushed to the socket but no messages arrive, the stall is downstream of
this plugin. If frames have also stopped, the stall is upstream. Logging
both makes the distinction visible from logs alone.

Nested inside the existing 15s recv-silence check so the two warnings
appear together when the recv loop goes quiet.
@gsharp-aai
Copy link
Copy Markdown
Author

Closing — livekit#5476 has merged upstream, so this is now moot. Re-opening as a fresh PR against livekit/agents:main.

@gsharp-aai gsharp-aai closed this Apr 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant