Skip to content

feat(voice): audible recipient ack on inbound /handoff in #general#1332

Merged
itskai-dev merged 1 commit into
mainfrom
claude/voice-ack-on-handoff
May 4, 2026
Merged

feat(voice): audible recipient ack on inbound /handoff in #general#1332
itskai-dev merged 1 commit into
mainfrom
claude/voice-ack-on-handoff

Conversation

@itskai-dev

Copy link
Copy Markdown
Collaborator

Summary

When an agent posts /handoff to=<name> in #general, the named
recipient emits a recipient-authored 1–4 word audible acknowledgment
through the existing /canvas/speak path (Kokoro/ElevenLabs →
voice_output SSE → activeSpeaker rail). Closes the joyless seam where
inbound handoffs were silent on canvas.

Locked behavior (per kai msg-1777921934933):

  • real `/handoff to=` only (strict prefix; cloud SDK typo set not honored on this path)
  • voiceCapable recipient only (`agent_config.settings.voice` non-empty)
  • human present in room (≥1)
  • recipient not already mid-TTS (gate on activeSpeaker)
  • 1–4 word recipient-authored ack via Anthropic `claude-haiku-4-5`
  • silence on any gate-fail or LLM failure (no canned chrome)
  • no chat duplication, no extra canvas UI
  • turn-local dedup (the inbound handoff turn gets ack-only audio; later turns are unaffected because voice-on-chat keys on message content)

Files

  • `src/voice-ack-on-handoff.ts` (new): parse + decide + speak module, wired from `chatManager.sendMessage` right after `triggerVoiceOnChat`
  • `src/active-speakers.ts` (new): `Set`-backed mirror of who is mid-TTS so server-side gates can ask the question without reaching into the `canvasStateMap` closure
  • `src/canvas-interactive.ts` /canvas/speak: marks speaker active on queue, clears on `voice_output` (durationMs+500) or safety net (estimatedMs+8s). Covers chat-driven TTS lifecycle.
  • `src/server.ts` `setActiveSpeaker` / `setActiveSpeakerAudio`: also write to active-speakers store so interactive voice sessions feed the same gate.
  • `src/voice-ack-on-handoff.test.ts` (new): 24 unit tests — parse strictness, gate ordering (channel → sender → parse → self → voiceCapable → humans → activeSpeaker), clamp behavior, and wiring (fast-path skip, gate skip, happy path, silence on empty ackLine, silence when recipient mid-TTS).

Proof bar (canonical staging)

After merge + image roll on `rn-34faba44-wlgkeq`:

  1. Happy path — kai posts `/handoff to=link` from chat with a human in the room → link speaks 1–4 word ack
  2. No human — same handoff with no human in room → silence
  3. Non-voiceCapable recipient — handoff to an agent without `settings.voice` → silence
  4. Recipient mid-TTS — link is currently speaking → silence
  5. Free-text @mention — `@link please look` (not a real handoff) → silence

Test plan

  • `node --import tsx --test src/voice-ack-on-handoff.test.ts` — 24/24 pass locally
  • `tsc --noEmit` clean
  • CI green
  • Roll canonical staging image
  • 5-bar proof on canonical

…ask-1777140469973)

When a real `/handoff to=<name>` lands in #general and the named agent
is on this host, that agent speaks a 1-4 word audible acknowledgment in
its own voice — same /canvas/speak path voice-on-chat already uses, so
the existing voice_output SSE → activeSpeaker rail drives it. No chat
duplication, no extra canvas UI.

Locked behavior:
- real `/handoff to=<name>` only — strict prefix, cloud SDK typo set
  ('/handof', '/andoff', '/andof') is intentionally NOT honored on this
  audible path; embedded mentions and follow-up text are ignored.
- voiceCapable recipient only (agent_config.settings.voice non-empty)
- ≥1 human present in room
- recipient not already mid-TTS (gate on activeSpeaker)
- 1-4 word recipient-authored line via Anthropic claude-haiku-4-5
- silence on any gate-fail or LLM failure — no canned chrome in prod;
  a tiny dev-only fallback pool fires only when ANTHROPIC_API_KEY is
  unset, so dev environments aren't mute.
- turn-local dedup: ack IS the vocal artifact for this turn; the
  existing voice-on-chat content-cache prevents the /handoff line
  itself from also speaking.

Wires into chatManager.sendMessage right after triggerVoiceOnChat.
Adds active-speakers store mirrored from setActiveSpeaker /
setActiveSpeakerAudio in server.ts and from /canvas/speak queue/clear
in canvas-interactive.ts so the gate has truth without depending on
canvasStateMap closure.

Tests: 24 unit tests covering parse, decide-gate ordering, ack
clamping, and trigger wiring (including silent-on-gate-fail and
silent-on-empty-ackLine paths).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@itskai-dev
itskai-dev force-pushed the claude/voice-ack-on-handoff branch from fcb5790 to 67c4fb5 Compare May 4, 2026 19:34
@itskai-dev
itskai-dev merged commit 76427ac into main May 4, 2026
12 checks passed
@itskai-dev
itskai-dev deleted the claude/voice-ack-on-handoff branch May 4, 2026 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant