feat(voice): trigger /canvas/speak on outbound agent chat when human is in room#1320
Merged
Conversation
…is in room
task-1777140469973-770cy4yma — outbound voice loop on canvas
Today canvas has two parallel TTS rails:
- browser-driven /api/tts plays per-tab via useVoice — fires on every
qualifying agent reply but has NO room fan-out
- node-driven /canvas/speak triggers voice_output SSE → voice.output
Supabase broadcast → handleVoiceOutput on all canvas tabs (PR #2840),
but only fires for explicit callers (bootstrap intro today)
Result: agent chat replies are visually shared (PR #2839 reply card) but
acoustically per-tab. This closes that gap on the smallest possible seam.
What this PR does:
- chatManager.sendMessage now calls triggerVoiceOnChat(message) after
eventBus.emitMessagePosted — fires-and-forgets a POST to local
/canvas/speak
- triggerVoiceOnChat gates on:
• channel === 'general' (not all rooms broadcast voice)
• from is not 'system' / 'human' / 'user' (humans speak themselves)
• content passes the speakability filter (no @-only, no commit
hashes, no PR/CI status, no code-heavy, no terminal output, ≤350
chars, ≥8 chars after cleaning)
• listRoomParticipants().length > 0 — only when a human is present
in the room (reuses room-presence-store from #1316)
• module-level dedup + 8s global cooldown (mirrors useVoice)
- speakability filter ported verbatim from
apps/web/src/app/presence/use-voice.ts so both rails apply the same
"no police scanner" bar
What this PR does NOT do (per kai msg-1777815083752):
- no new endpoint
- no new event type
- no new channel
- no provider rewrite (Kokoro stays primary)
- no push-bridge widening
- no AGENTS.md changes
- browser useVoice path stays untouched as fallback — no dedupe, no
delete (future-deletable; not this PR)
Audio playback is already wired: voice_output SSE → useCanvasStreams
→ window.handleVoiceOutput (asker tab) AND useExpressionChannels
→ roomVoice.publish → voice.output Supabase broadcast → useRoomVoice
→ window.handleVoiceOutput (peer tabs).
Tests: 18 unit tests in tests/voice-on-chat.test.ts cover the
speakability filter (every reject branch + accept branch), the cleaned-
text builder, and the gate decision (channel, sender, speakable, humans
present). Cooldown + dedup state lives in module-level vars stamped by
the trigger; verified end-to-end on canonical staging.
Will not auto-merge. Awaiting team ack + canonical-host proof on
e4e35463-02d7-420d-a00d-65e765ade5a2.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per kai + link in #general (msg-1777816280065 / msg-1777816286111) on PR #1320: add a one-line decision log so the gate is observable in fly logs without a behavior change. Cheap canonical-proof unblocker before merge. - info on willSpeak=true (from + chars + humans count) - info on non-noisy false reasons (no-humans-in-room / cooldown / not-speakable) - skip channel-not-general / sender-not-agent — fires on all task-comments and system noise; would drown the log No new dep, no new path. Same 18/18 tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Outbound voice loop on canvas — closes the gap where agent chat replies are visually shared (via PR #2839 reply card) but acoustically per-tab (browser-driven
/api/ttsonly plays in the asker's tab).chatManager.sendMessagenow callstriggerVoiceOnChat(message)aftereventBus.emitMessagePosted. When all gates pass, it fires-and-forgets a POST to local/canvas/speak— reusing the existingvoice_outputSSE →voice.outputSupabase broadcast path that PR #2840 already wired up to fan out audio to all canvas tabs (asker + peers + big screen).Lane locked by kai (msg-1777815083752): smallest honest seam, gate on human presence, reuse existing path, no new endpoint/event, browser
useVoicestays untouched as fallback this PR.task:
task-1777140469973-770cy4ymaGates (all required to fire)
listRoomParticipants().length > 0— at least one human is present in the room (reuses room-presence-store from feat(chat-context): inject ROOM STATUS line when a human is in the room #1316)Speakability filter is ported verbatim from
apps/web/src/app/presence/use-voice.ts:378-405so both rails apply the same "no police scanner" bar.Bar held (per kai)
/canvas/speakvoice_outputSSEroom:${hostId}Supabase broadcastvoice_outputSSE only reaches canvas subscribersuseVoiceuntouched in this PR — kept as fallback, no dedupe, no deleteFiles
src/voice-on-chat.ts(NEW, ~160 LOC) — filter + gate + HTTP self-call triggersrc/chat.ts(+5 LOC) — single hook aftereventBus.emitMessagePostedtests/voice-on-chat.test.ts(NEW, ~120 LOC) — 18 unit tests for filter + gateTest plan
npx tsc --noEmit— cleannpx vitest run tests/voice-on-chat.test.ts— 18/18 passingnpx vitest run tests/chat-context-room-active.test.ts tests/chat-dedup.test.ts tests/chat-monotonic-timestamps.test.ts— 28/28 passing (no regressions on chat-adjacent suites)e4e35463-02d7-420d-a00d-65e765ade5a2— agent posts to #general → human in room hears voice on canvas tab + big-screen tab/canvas/speakcall (silent gate)Will not auto-merge. Awaiting team ack + canonical-host proof.
🤖 Generated with Claude Code