Skip to content

[Feature]: Agent bus — set window/file audio source via API #56

Description

@rosspeili

One-line summary

Let external agents switch AVATAR’s audio capture to pick app window or audio file through the local agent bus, not only via the Voice gear menu.

Problem / motivation

#6 lets the bus drive stage motion and environment, but lip sync still requires manual Voice panel changes for “capture this window” or “play this file.” Agent workflows (TTS from another app, scripted demos, MCP-driven sessions) need programmatic source selection to stay hands-off.

Existing implementation to align with: Voice panel, useAudioSource, and docs/voice/audio-sources.md.

Proposed solution

Add bus commands (HTTP + WebSocket parity) such as:

  • audio.source.set with { mode: "device" | "window" | "file" | "mic" | "off", ... }
  • For window: accept optional target hints (process name, last-focused) with clear failure when user must pick via OS picker.
  • For file: path or user-library-relative path with validation; respect existing Electron file permissions.
  • Response includes current source + live-dot-relevant status (align with [Feature]: Reactive glass-bar live dot (amplitude + capture status colors) #42 reactive dot states).

Update docs/agents/local-bus.md and Settings → Agents help text. Security: loopback + token only; no remote file URLs without explicit opt-in.

Alternatives considered

  • OS-level virtual cable only — user installs VB-Audio etc.; no API change but worse UX for agents.
  • Mic-only via bus — insufficient for “lip-sync to agent TTS window.”

Primary surface

Voice / lip sync

Constraints you accept

  • Should stay usable offline / local-first
  • Browser-only mode may remain limited vs Electron

Mock / sketch / reference (optional)

Mirror fields already persisted in user settings / voice hooks; reuse runCommand / IPC paths where the Voice panel does today.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:voiceAudio sources, capture, permissions, restartfeatureNew feature or improvement requestfuture:agent-busExternal agent / event bus hooks (roadmap)platform:electronElectron shell, IPC, loopback, luma sampling

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions