Skip to content

feat(voice): trigger /canvas/speak on outbound agent chat when human is in room#1320

Merged
itskai-dev merged 2 commits into
mainfrom
claude/voice-on-chat-trigger
May 3, 2026
Merged

feat(voice): trigger /canvas/speak on outbound agent chat when human is in room#1320
itskai-dev merged 2 commits into
mainfrom
claude/voice-on-chat-trigger

Conversation

@itskai-dev

Copy link
Copy Markdown
Collaborator

Summary

Outbound voice loop on canvas — closes the gap where agent chat replies are visually shared (via PR #2839 reply card) but acoustically per-tab (browser-driven /api/tts only plays in the asker's tab).

chatManager.sendMessage now calls triggerVoiceOnChat(message) after eventBus.emitMessagePosted. When all gates pass, it fires-and-forgets a POST to local /canvas/speak — reusing the existing voice_output SSE → voice.output Supabase broadcast path that PR #2840 already wired up to fan out audio to all canvas tabs (asker + peers + big screen).

Lane locked by kai (msg-1777815083752): smallest honest seam, gate on human presence, reuse existing path, no new endpoint/event, browser useVoice stays untouched as fallback this PR.

task: task-1777140469973-770cy4yma

Gates (all required to fire)

  • channel === 'general'
  • from is not 'system' / 'human' / 'user'
  • content passes the speakability filter (no @-only, no commit hashes, no PR/CI status, no code-heavy, no terminal output, ≤350 chars, ≥8 chars after cleaning)
  • listRoomParticipants().length > 0 — at least one human is present in the room (reuses room-presence-store from feat(chat-context): inject ROOM STATUS line when a human is in the room #1316)
  • module-level dedup + 8s global cooldown (mirrors useVoice)

Speakability filter is ported verbatim from apps/web/src/app/presence/use-voice.ts:378-405 so both rails apply the same "no police scanner" bar.

Bar held (per kai)

  • ✅ no new endpoint — reuses /canvas/speak
  • ✅ no new event type — reuses voice_output SSE
  • ✅ no new channel — reuses room:${hostId} Supabase broadcast
  • ✅ no provider rewrite — Kokoro stays primary
  • ✅ no push-bridge widening — voice_output SSE only reaches canvas subscribers
  • ✅ no AGENTS.md changes
  • ✅ browser useVoice untouched in this PR — kept as fallback, no dedupe, no delete

Files

  • src/voice-on-chat.ts (NEW, ~160 LOC) — filter + gate + HTTP self-call trigger
  • src/chat.ts (+5 LOC) — single hook after eventBus.emitMessagePosted
  • tests/voice-on-chat.test.ts (NEW, ~120 LOC) — 18 unit tests for filter + gate

Test plan

  • npx tsc --noEmit — clean
  • npx vitest run tests/voice-on-chat.test.ts — 18/18 passing
  • npx vitest run tests/chat-context-room-active.test.ts tests/chat-dedup.test.ts tests/chat-monotonic-timestamps.test.ts — 28/28 passing (no regressions on chat-adjacent suites)
  • Canonical staging proof on e4e35463-02d7-420d-a00d-65e765ade5a2 — agent posts to #general → human in room hears voice on canvas tab + big-screen tab
  • Empty-room negative — agent posts with no humans → no /canvas/speak call (silent gate)
  • DM negative — agent posts to DM/non-general channel → no fire
  • System negative — system message in #general → no fire

Will not auto-merge. Awaiting team ack + canonical-host proof.

🤖 Generated with Claude Code

itskai-dev and others added 2 commits May 3, 2026 06:41
…is in room

task-1777140469973-770cy4yma — outbound voice loop on canvas

Today canvas has two parallel TTS rails:
  - browser-driven /api/tts plays per-tab via useVoice — fires on every
    qualifying agent reply but has NO room fan-out
  - node-driven /canvas/speak triggers voice_output SSE → voice.output
    Supabase broadcast → handleVoiceOutput on all canvas tabs (PR #2840),
    but only fires for explicit callers (bootstrap intro today)

Result: agent chat replies are visually shared (PR #2839 reply card) but
acoustically per-tab. This closes that gap on the smallest possible seam.

What this PR does:
  - chatManager.sendMessage now calls triggerVoiceOnChat(message) after
    eventBus.emitMessagePosted — fires-and-forgets a POST to local
    /canvas/speak
  - triggerVoiceOnChat gates on:
      • channel === 'general' (not all rooms broadcast voice)
      • from is not 'system' / 'human' / 'user' (humans speak themselves)
      • content passes the speakability filter (no @-only, no commit
        hashes, no PR/CI status, no code-heavy, no terminal output, ≤350
        chars, ≥8 chars after cleaning)
      • listRoomParticipants().length > 0 — only when a human is present
        in the room (reuses room-presence-store from #1316)
      • module-level dedup + 8s global cooldown (mirrors useVoice)
  - speakability filter ported verbatim from
    apps/web/src/app/presence/use-voice.ts so both rails apply the same
    "no police scanner" bar

What this PR does NOT do (per kai msg-1777815083752):
  - no new endpoint
  - no new event type
  - no new channel
  - no provider rewrite (Kokoro stays primary)
  - no push-bridge widening
  - no AGENTS.md changes
  - browser useVoice path stays untouched as fallback — no dedupe, no
    delete (future-deletable; not this PR)

Audio playback is already wired: voice_output SSE → useCanvasStreams
→ window.handleVoiceOutput (asker tab) AND useExpressionChannels
→ roomVoice.publish → voice.output Supabase broadcast → useRoomVoice
→ window.handleVoiceOutput (peer tabs).

Tests: 18 unit tests in tests/voice-on-chat.test.ts cover the
speakability filter (every reject branch + accept branch), the cleaned-
text builder, and the gate decision (channel, sender, speakable, humans
present). Cooldown + dedup state lives in module-level vars stamped by
the trigger; verified end-to-end on canonical staging.

Will not auto-merge. Awaiting team ack + canonical-host proof on
e4e35463-02d7-420d-a00d-65e765ade5a2.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per kai + link in #general (msg-1777816280065 / msg-1777816286111) on PR #1320:
add a one-line decision log so the gate is observable in fly logs without a
behavior change. Cheap canonical-proof unblocker before merge.

- info on willSpeak=true (from + chars + humans count)
- info on non-noisy false reasons (no-humans-in-room / cooldown / not-speakable)
- skip channel-not-general / sender-not-agent — fires on all task-comments
  and system noise; would drown the log

No new dep, no new path. Same 18/18 tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@itskai-dev
itskai-dev merged commit 922d758 into main May 3, 2026
12 checks passed
@itskai-dev
itskai-dev deleted the claude/voice-on-chat-trigger branch May 3, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant