🎤 P4 investigation: voice via Nest + Hailo NPU

[adrianwedd]

Per spec P4: investigate voice input via Nest devices; evaluate the Hailo NPU for local whisper/TTS acceleration. Also fold in: Home Assistant sensor integration ideas from retired issues #85/#72.

[adrianwedd]

Deep research complete — voice input + Hailo NPU

Full write-up: docs/research/p4-voice-hailo-2026-07-17.md (commit 021dab9 on adhdo2-p2p3). Synthesized from a 53-source NotebookLM deep-research notebook ("ADHDo P4 — Nest voice input + Hailo NPU whisper/TTS + HA sensors") — available for follow-up queries.

Headlines

1. Hailo-8L Whisper is real and already packaged. Hailo officially ships Whisper tiny/base for Hailo-8/8L, and the community wyoming-hailo-whisper server (plus a HassOS add-on) exposes it over the Wyoming protocol. Reported Pi 5 numbers: 3–4 s per utterance vs 8–10 s CPU-only. Gotchas: decoder embeddings run on CPU, HEFs are English-calibrated (recompiling needs an x86 box), and PCIe gen 3 must be enabled.

2. Nest mics are a dead end — keep Nests output-only. The hardware is locked; the only ways in are drop-in PCB replacements (Onju Voice v2, MiciMike at ~$80 + fiddly disassembly, half-duplex I2S). A $60 Home Assistant Voice PE is cheaper, pre-built, and has the same XMOS DSP. Honest caveat from community testing: all current open satellites underperform Google/Amazon far-field mics in noisy rooms.

3. Pipeline shape: microWakeWord on the satellite → hailo-whisper STT → custom conversation agent bridging to the resident Claude session → Piper TTS (validates the existing choice), with sentence-level streaming as the key latency trick — start speaking the first sentence while generation continues.

4. Sensors (folds in retired #85/#72): mmWave static-presence (Apollo MSR-2 class) to gate nudges on actually-in-the-room; phone alarm-dismissal as the morning-routine trigger; and the community "nagging with an escape hatch" pattern, which matches our nudge design philosophy.

Recommended build

Voice PE satellite + wyoming-hailo-whisper (Whisper base) + Piper + streaming bridge to the resident session; one mmWave sensor in the main workspace. ~3–4 days effort, ~$100 hardware. Per-component estimates in the doc.