feat(voice): always-on Settings toggle + debug panel + i18n (5/8 of #3307)#3344
Merged
M3gA-Mind merged 8 commits intoJun 4, 2026
Merged
Conversation
Run enigo keyboard/mouse on the app main thread via a native-registry executor; enigo's macOS TSMGetInputSourceProperty traps off-thread and crashes the CEF host. Adds mouse/keyboard tools, the main_thread bridge, and downscaled screenshots so the model can see them. Slice 1/7 of tinyhumansai#3307 (was the 'computer control' area).
… loop Adds the Rust-internal automate engine (poll-until-stable settle, playback verification), the AXEnabled diagnostics field + settle primitives on ax_interact, the Music fast-path, and the Windows UIA superset. Exposes launch_platform as pub(crate) so the automate loop can launch apps mid-flow. Slice 2/7 of tinyhumansai#3307 (accessibility/automate engine).
…trator Registers the AutomateTool (multi-step UI flows in one call) and the ax_interact denylist/opt-in plumbing; adds the catalog toggle, tool definition, and orchestrator prompt guidance (automate + screenshot/ mouse/keyboard fallback for Electron apps with empty AX trees). Slice 3/7 of tinyhumansai#3307 (tool wiring + prompts).
Continuous cpal mic → VAD segmenter → STT → agent with no hotkey, opt-in via voice_server.always_on_enabled, 'Hey Tiny' wake word (English-forced STT + fuzzy match), and screen-lock privacy pause. Adds the config schema, live-apply on the settings RPC, start_if_enabled wiring, and a JSON-RPC roundtrip E2E. Slice 4/7 of tinyhumansai#3307 (always-on core).
Surfaces the always-on listening toggle in the reachable Voice panel, adds the VoiceDebugPanel, the voice tauri-command wrapper, and the RPC client method. Adds all voice.debug.* and notch.* i18n keys across the 14 locales (notch keys land here as inert strings; the notch UI that consumes them ships in slice 6). Slice 5/7 of tinyhumansai#3307 (always-on frontend).
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe PR adds an always-on listening mode to the voice settings infrastructure. It extends the ChangesAlways-on listening feature
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
M3gA-Mind
changed the title
This was referenced Jun 4, 2026
Contributor
Author
This was referenced Jun 4, 2026
Take main's versions of already-merged slice-1..4 backend files.
It isn't on main and its 'mouse/keyboard run without an approval prompt' text contradicts the tinyhumansai#3342 ApprovalGate fix. Keep tinyhumansai#3344 a clean UI slice; a corrected desktop-control prompt can land as its own follow-up.
Contributor
Author
Independent review (beyond the CodeRabbit pass)Reviewed the always-on UI slice — the Settings toggle ( Reviewed clean
No correctness issues. LGTM once CI is green. |
coderabbitai
Bot
added
feature
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/components/settings/panels/__tests__/VoicePanel.test.tsx`:
- Around line 114-115: Add behavior tests to VoicePanel.test.tsx that exercise
the new always-on toggle flow: use the existing fixture (which now includes
always_on_enabled) to render the panel, simulate a user click on the DOM element
with test id "voice-always-on-toggle", and assert that the mocked
tauriCommands.openhumanUpdateVoiceServerSettings RPC is called with the expected
payload on toggle-on and toggle-off; also simulate an RPC failure to verify the
UI reverts (rollback) and that the toggle state returns to its prior value.
Update the mocked tauriCommands module in the test to include
openhumanUpdateVoiceServerSettings, and write two specs covering (1) successful
toggle persistence and RPC args, and (2) failed RPC causing a UI revert.
In `@app/src/components/settings/panels/VoicePanel.tsx`:
- Around line 508-521: The onClick handler for toggling
settings.always_on_enabled can run concurrent writes; capture the current value
into a local previous variable, bail if a pending toggle is in progress
(introduce an isTogglingAlwaysOn state flag), set isTogglingAlwaysOn=true,
optimistically update UI via setSettings, call
openhumanUpdateVoiceServerSettings with the new value, and on error rollback by
calling setSettings to restore previous and log the error; finally set
isTogglingAlwaysOn=false in a finally block and ensure the toggle control reads
isTogglingAlwaysOn to disable further clicks while the RPC is pending (refer to
the onClick handler, setSettings, settings.always_on_enabled,
openhumanUpdateVoiceServerSettings).
In `@app/src/lib/i18n/ko.ts`:
- Line 4775: The locale key 'notch.transcribing' currently maps to '변환 중…' but
should use the same STT terminology as the rest of the Korean locale; update the
value for notch.transcribing in app/src/lib/i18n/ko.ts from '변환 중…' to '전사 중…'
so it matches other voice/STT strings (search for the 'notch.transcribing' key
to locate and modify it).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 9091cafc-efca-49c0-b037-62687c7947b7
📒 Files selected for processing (19)
app/src/components/settings/panels/VoiceDebugPanel.tsxapp/src/components/settings/panels/VoicePanel.tsxapp/src/components/settings/panels/__tests__/VoicePanel.test.tsxapp/src/lib/i18n/ar.tsapp/src/lib/i18n/bn.tsapp/src/lib/i18n/de.tsapp/src/lib/i18n/en.tsapp/src/lib/i18n/es.tsapp/src/lib/i18n/fr.tsapp/src/lib/i18n/hi.tsapp/src/lib/i18n/id.tsapp/src/lib/i18n/it.tsapp/src/lib/i18n/ko.tsapp/src/lib/i18n/pl.tsapp/src/lib/i18n/pt.tsapp/src/lib/i18n/ru.tsapp/src/lib/i18n/zh-CN.tsapp/src/services/coreRpcClient.tsapp/src/utils/tauriCommands/voice.ts
…ove notch token CodeRabbit tinyhumansai#3344 + coverage gate: - VoicePanel: behavior tests for the always-on toggle — persists via openhumanUpdateVoiceServerSettings on click, optimistic flip, and revert on RPC failure. - VoiceDebugPanel: new test covering its always-on toggle + save payload. - Move the notch WKWebview core-token block out of this UI slice (it's notch/slice-6 concern) back to main; it lands with the notch PR.
M3gA-Mind
added a commit
to M3gA-Mind/openhuman
that referenced
this pull request
Jun 4, 2026
…ch slice Same stale 'runs without an approval prompt' section as tinyhumansai#3344 — not on main, contradicts the tinyhumansai#3342 ApprovalGate fix. Tracked for a corrected follow-up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Slice 5/8 of #3307 — always-on frontend: Settings toggle, debug panel, i18n.
VoiceDebugPanel, the voice tauri-command wrapper, and the RPC client method.voice.debug.*andnotch.*i18n keys across the 14 locales. (Notch keys land here as inert strings to keep locale parity valid; the notch UI that consumes them ships in slice 6.)Files (19 — 14 are one-line locale additions)
components/settings/panels/{VoicePanel,VoiceDebugPanel}.tsx(+ test),utils/tauriCommands/voice.ts,services/coreRpcClient.ts,lib/i18n/*.ts(14).Summary by CodeRabbit
New Features
Internationalization
Tests