feat(notch): always-visible macOS notch status pill (6/8 of #3307)#3345
Merged
Conversation
Run enigo keyboard/mouse on the app main thread via a native-registry executor; enigo's macOS TSMGetInputSourceProperty traps off-thread and crashes the CEF host. Adds mouse/keyboard tools, the main_thread bridge, and downscaled screenshots so the model can see them. Slice 1/7 of tinyhumansai#3307 (was the 'computer control' area).
… loop Adds the Rust-internal automate engine (poll-until-stable settle, playback verification), the AXEnabled diagnostics field + settle primitives on ax_interact, the Music fast-path, and the Windows UIA superset. Exposes launch_platform as pub(crate) so the automate loop can launch apps mid-flow. Slice 2/7 of tinyhumansai#3307 (accessibility/automate engine).
…trator Registers the AutomateTool (multi-step UI flows in one call) and the ax_interact denylist/opt-in plumbing; adds the catalog toggle, tool definition, and orchestrator prompt guidance (automate + screenshot/ mouse/keyboard fallback for Electron apps with empty AX trees). Slice 3/7 of tinyhumansai#3307 (tool wiring + prompts).
Continuous cpal mic → VAD segmenter → STT → agent with no hotkey, opt-in via voice_server.always_on_enabled, 'Hey Tiny' wake word (English-forced STT + fuzzy match), and screen-lock privacy pause. Adds the config schema, live-apply on the settings RPC, start_if_enabled wiring, and a JSON-RPC roundtrip E2E. Slice 4/7 of tinyhumansai#3307 (always-on core).
Surfaces the always-on listening toggle in the reachable Voice panel, adds the VoiceDebugPanel, the voice tauri-command wrapper, and the RPC client method. Adds all voice.debug.* and notch.* i18n keys across the 14 locales (notch keys land here as inert strings; the notch UI that consumes them ships in slice 6). Slice 5/7 of tinyhumansai#3307 (always-on frontend).
Transparent NSPanel + WKWebView anchored at the top-centre of the primary screen showing live Ready/Listening/Processing state; automate streams step progress to it via the overlay:attention socket bridge. macOS only; no-op elsewhere. Slice 6/7 of tinyhumansai#3307 (notch status pill).
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds a macOS-only floating notch panel (NSPanel + WKWebView) that injects core RPC URL and bearer token into the webview and renders a bundled NotchApp React pill UI which connects to core via Socket.IO and displays activity events. ChangesmacOS Notch Activity Indicator Feature
Sequence DiagramsequenceDiagram
participant TauriRuntime
participant NSPanel
participant WKWebView as WKWebView (NotchApp)
participant CoreProcess
participant SocketIO
TauriRuntime->>NSPanel: Create transparent panel at top-center
TauriRuntime->>WKWebView: Load NotchApp (dev URL or bundled)
WKWebView->>WKWebView: Page loads, listens for notch:core-url
TauriRuntime->>TauriRuntime: Spawn injection timer
loop Poll until ready
TauriRuntime->>CoreProcess: Check OPENHUMAN_CORE_RPC_URL env
TauriRuntime->>CoreProcess: Get current bearer token
end
TauriRuntime->>WKWebView: evaluateJavaScript inject globals + dispatch notch:core-url
WKWebView->>SocketIO: Connect with injected token
CoreProcess->>SocketIO: Emit dictation/transcription/state/attention
SocketIO->>WKWebView: dictation:toggle, transcription, etc.
WKWebView->>WKWebView: Update pill mode/text/styling
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
This was referenced Jun 4, 2026
M3gA-Mind
changed the title
This was referenced Jun 4, 2026
Contributor
Author
6 tasks
…notch # Conflicts: # app/src-tauri/src/lib.rs # docs/voice-automate-plan.md # docs/voice-system-actions.md # src/openhuman/accessibility/app_fastpaths/fastpaths_tests.rs # src/openhuman/accessibility/app_fastpaths/music.rs # src/openhuman/accessibility/automate.rs # src/openhuman/agent_registry/agents/orchestrator/agent.toml # src/openhuman/config/ops.rs # src/openhuman/config/schemas.rs # src/openhuman/tools/impl/browser/screenshot.rs # src/openhuman/tools/impl/computer/main_thread.rs # src/openhuman/voice/always_on.rs
…ch slice Same stale 'runs without an approval prompt' section as tinyhumansai#3344 — not on main, contradicts the tinyhumansai#3342 ApprovalGate fix. Tracked for a corrected follow-up.
Contributor
Author
Independent review (beyond the CodeRabbit pass)Reviewed the notch slice — the native Reviewed clean
No correctness issues. LGTM once CI is green. |
coderabbitai
Bot
added
feature
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
app/src/index.css (1)
57-91: ⚡ Quick winAdd a reduced-motion escape hatch for the new notch animations.
notch-pill-in,notch-bar, andnotch-dotall animate here, but the existingprefers-reduced-motionoverride later in this file only disables the command-palette/help motion. The notch will still pulse continuously for users who have reduced-motion enabled.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/src/index.css` around lines 57 - 91, The new notch animations notch-pill-in, notch-bar, and notch-dot currently run even when users set prefers-reduced-motion; add a prefers-reduced-motion media query that targets these keyframes/elements and disables or minimizes their animation (e.g., set animation: none or animation-duration: 0.001s and remove transforms/opacity transitions) so the notch no longer pulses for reduced-motion users; update CSS selectors that apply these animations (where notch-pill-in, notch-bar, notch-dot are referenced) to respect the prefers-reduced-motion override.app/src/notch/NotchApp.tsx (1)
181-255: ⚡ Quick winReplace
console.*diagnostics with a namespaceddebuglogger.These new
console.debug/console.warncalls bypass the app's dev-only logging convention and will emit raw runtime details in production consoles. Please route them through adebug('notch')namespace instead. As per coding guidelines, "Inapp/src, use namespaceddebuglogs (e.g. from thedebugnpm package) with dev-only detail for development diagnostics."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/src/notch/NotchApp.tsx` around lines 181 - 255, Replace raw console calls with a namespaced debug logger: add "import debug from 'debug'" at the top and create a logger (e.g. const dbg = debug('notch')); then change every console.debug(...) and console.warn(...) in NotchApp.tsx (notably inside the socket.on handlers for 'dictation:toggle', 'dictation:transcription', 'companion:state_changed', 'overlay:attention', the socket.connect() log, and the catch block) to use dbg(...) (format messages and include err with %O for the catch). Keep the same text payloads and trimming logic; only swap out console.* for the dbg(...) calls so logs follow the app’s namespaced dev-only convention.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/notch/NotchApp.tsx`:
- Around line 214-227: The code currently falls back to the raw agentState
string when mapping text (modeMap/textMap and setState), exposing untranslated
backend values; update the logic in NotchApp (the modeMap/textMap and the
setState call that reads agentState) to never render raw agentState: add
explicit mappings for any expected companion states and change the default
fallback to a translated generic string using useT() (e.g. t('notch.unknown',
'Unknown') or t('notch.status.unknown', 'Status unknown')); ensure both the mode
and text branches use the translation fallback so all user-visible strings go
through useT().
- Around line 164-180: The connectSocket async initializer uses a local disposed
flag that only flips in an internal disposer which isn't wired into the effect
cleanup, so if unmount or route changes happen while connectCoreSocket() is
resolving the socket can still attach listeners and call setState; fix by
returning or exposing the disposer from connectSocket and ensuring callers (the
effect, the preload path and the "notch:core-url" handler) call that disposer on
cleanup/unmount (or store the disposer on a ref and call it in the useEffect
cleanup), ensuring disposed is set and any pending socket is disconnected and
prevented from setting socketRef.current or dispatching state after teardown
(refer to connectSocket, connectCoreSocket, socketRef.current and the effect
handler that registers "notch:core-url").
---
Nitpick comments:
In `@app/src/index.css`:
- Around line 57-91: The new notch animations notch-pill-in, notch-bar, and
notch-dot currently run even when users set prefers-reduced-motion; add a
prefers-reduced-motion media query that targets these keyframes/elements and
disables or minimizes their animation (e.g., set animation: none or
animation-duration: 0.001s and remove transforms/opacity transitions) so the
notch no longer pulses for reduced-motion users; update CSS selectors that apply
these animations (where notch-pill-in, notch-bar, notch-dot are referenced) to
respect the prefers-reduced-motion override.
In `@app/src/notch/NotchApp.tsx`:
- Around line 181-255: Replace raw console calls with a namespaced debug logger:
add "import debug from 'debug'" at the top and create a logger (e.g. const dbg =
debug('notch')); then change every console.debug(...) and console.warn(...) in
NotchApp.tsx (notably inside the socket.on handlers for 'dictation:toggle',
'dictation:transcription', 'companion:state_changed', 'overlay:attention', the
socket.connect() log, and the catch block) to use dbg(...) (format messages and
include err with %O for the catch). Keep the same text payloads and trimming
logic; only swap out console.* for the dbg(...) calls so logs follow the app’s
namespaced dev-only convention.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 68119d50-6f73-4c34-94b0-f3437c94fcdf
📒 Files selected for processing (6)
app/src-tauri/src/lib.rsapp/src-tauri/src/notch_window.rsapp/src/index.cssapp/src/main.tsxapp/src/notch/NotchApp.tsxapp/src/services/coreRpcClient.ts
…mount CodeRabbit tinyhumansai#3345: - notch_window.rs: log only the page source *kind* and file basenames, never absolute bundle paths (they contain /Users/<login>/… PII). - NotchApp.tsx: capture connectSocket()'s disposer and call it on effect cleanup / on a new core-url, so a still-resolving connectCoreSocket can't attach listeners or setState after teardown.
Clears the diff-cover gate for tinyhumansai#3345: - NotchApp.test.tsx: drives a mock Socket.IO through all handlers (dictation toggle/transcription, companion state, overlay attention), both bootstrap paths (preloaded global + notch:core-url event), and the ready/listening/transcribing/thinking/speaking/attention render modes. - coreRpcClient: cover the host-injected __OPENHUMAN_NOTCH_CORE_TOKEN__ fast-path in getCoreRpcToken.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Slice 6/8 of #3307 — always-visible macOS notch status pill.
automatestreams step progress to it via theoverlay:attentionsocket bridge.Files (5)
app/src-tauri/src/notch_window.rs,app/src-tauri/src/lib.rs(notch show/hide + startup),app/src/notch/NotchApp.tsx,app/src/main.tsx,app/src/index.css.Summary by CodeRabbit
New Features
Style
Enhancement
Tests