Skip to content

Use the native npm Computer Use bundle with unified macOS permission onboarding - #486

Merged
syzsunshine219 merged 6 commits into
MemTensor:computer_usefrom
memory-lee:codex/native-npm-computer-use
Sep 20, 2026
Merged

syzsunshine219 merged 6 commits into
MemTensor:computer_usefrom
memory-lee:codex/native-npm-computer-use

Conversation

@memory-lee

@memory-lee memory-lee commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

When a macOS Computer Use operation needs permissions, Memmy presents one panel with both Accessibility and Screen Recording settings. The operation waits for explicit continuation and a fresh permission check. Returning from System Settings no longer triggers another screen-capture probe or repeated system prompts.

  • Use the unmodified open-computer-use@0.3.5 npm bundle, removing vendored Swift sources, patches, and custom native builds while preserving the published helper name, identity, and signature.

  • Add a draggable card for the exact active helper when it is missing from the macOS permission list, with a copy-path fallback. Validate the IPC sender and block dragging during checks or failed helper suspension.

  • Keep the permission window in its existing position when opening Settings; it remains available as a drag source without moving to the right.

  • Pause only the owned helper during setup, reconnect after permission changes, and never replay an operation that may already have executed.

  • Provide passive desktop observation through get_screen_state, avoiding unintended Finder/application activation. This separate screen-capture path uses Memmy's own recording permission.

  • Update native bundle installation, packaging checks, CI, and permission regression coverage. Ignore the local signing-material directory.

  • Synchronize project and derived package version metadata to 1.1.6.

Validation

  • 207 targeted Vitest regression tests passed across permissions, MCP, runner behavior, screen capture, and packaging. Local server/socket tests were run outside the restricted sandbox.
  • 9 development helper installation tests passed.
  • Desktop shell and Agent production TypeScript checks passed.
  • Package version verification and diff whitespace checks passed.

Remaining validation and limits

  • Native drag-and-drop into System Settings and the complete first-grant flow need final installed-package acceptance testing.
  • Signed DMG/notarization validation is blocked by the supplied P12 certificate/password mismatch. No signing credentials or local workspace configuration are included.
  • macOS-triggered helper relaunch after manually revoking permission can still show the upstream native onboarding window; this PR does not change the upstream executable.
  • Windows/Linux runtime execution was not retested locally.

@syzsunshine219
syzsunshine219 merged commit 632ce48 into MemTensor:computer_use Sep 20, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants