feat: refine background browser workflow

Author: neriousy

CONTEXT.md

@@ -57,11 +57,14 @@ generation ID so stale disconnects and late work cannot mutate a newer generatio
 Browser Session owns explicit page lifecycle through Host Chrome and Browser Control. It observes pages created through
 the shared runtime API, but it does not implement popup adoption. New windows requested by page content are not a second
 tab-creation policy. Browser Control uses `open` for ordinary navigation and reserves `new_tab` for additional pages.
-Browser Control and Host Chrome new-tab actions are both append-only.
+Browser Control and Host Chrome new-tab actions are both append-only. Browser Control changes logical selection and
+page state in the background but never activates Host Chrome; `/browser`, the tab strip, and browser-mode keybindings
+are the explicit foreground boundary.
 
-Closing the final tracked page closes Host Chrome while Browser Session prepares a hidden replacement welcome page.
-Presentation code renders that internal page as **New tab** with an empty address and never displays its data URL in
-Host Chrome. Browser Control and DevTools may still report the underlying runtime URL.
+Closing the final tracked page closes Host Chrome and leaves Browser Session empty; it does not invent a hidden
+replacement target. The next explicit `/browser`, `open`, or `new_tab` operation creates the next page. A plain
+`/browser` creates the local welcome page on demand, renders it as **New tab** with an empty address, and can retry a
+failed page creation on the next invocation.
 
 ## Runtime And Presentation
 
@@ -75,6 +78,10 @@ agent but do not continuously send raw frames to Kitty. Presentation remains out
 Electron offscreen paint → raw RGBA file → Kitty image → terminal placement
 ```
 
+Host Chrome foreground navigation waits for the exact selected Browser Surface to finish viewport reconciliation and
+attach its Presentation before sending `goto`. Browser Control navigation does not wait for Presentation because agent
+work remains offscreen and must not activate the Browser workspace.
+
 The fixed Electron main/GPU/network/profile/DevTools cost is therefore paid once per Browser Generation, page renderer
 cost scales with the number and complexity of Shared Browser Pages, and active Presentation cost scales with viewport
 pixels and frame rate. The plugin defaults to 30 fps: 960×600 RGBA is about 66 MiB/s and 1920×1080 is about 237 MiB/s
@@ -89,7 +96,8 @@ first-run package download, Chrome installer, browser cache, or system-Chrome di
 
 The two MCP roles stay separate:
 
-- `opencode-browser-control` owns tracked tab lifecycle, selection, visibility, and stable `tabId` policy.
+- `opencode-browser-control` owns background tracked-tab lifecycle, logical selection, and stable `tabId` policy. It
+  does not expose visibility mutations.
 - `opencode-browser-devtools` inspects and operates the same live pages through Chrome DevTools MCP.
 
 Use Browser Control for `new_tab` and `close_tab`; letting a DevTools client invent page lifecycle would bypass Browser

README.md

@@ -88,13 +88,16 @@ Restart `opencode2` after changing the package or its options.
 
 ## Use
 
-Run `/browser` or choose **Open browser tab** from the command palette. Run
-`/browser https://example.com` to navigate and show the browser in one step.
+`/browser` reveals the currently selected browser tab when one exists; it does not create another tab. When the browser
+is empty, it creates the local **New tab** page on demand. `/browser https://example.com` reuses the selected tab and
+starts navigation only after its terminal surface has the final viewport. Use
+`/browser-new` or `/browser-new https://example.com` when you explicitly want an additional tab. Both commands are
+available from the slash-command picker and command palette.
 
 | Action                      | Keyboard or UI                                           |
 | --------------------------- | -------------------------------------------------------- |
 | Focus the address           | `Ctrl-L` / `Cmd-L`                                       |
-| New browser tab             | `Ctrl-T` / `Cmd-T`, or `+`                               |
+| New browser tab             | `/browser-new [url]`, `Ctrl-T` / `Cmd-T`, or `+`         |
 | Close selected browser tab  | `Ctrl-W` / `Cmd-W`, or the tab's `×`                     |
 | Next / previous browser tab | `Ctrl-Tab` / `Ctrl-Shift-Tab`                            |
 | Reload or stop loading      | `Ctrl-R` / `Cmd-R`, or the toolbar button                |
@@ -107,8 +110,10 @@ behavior there; `Cmd-W` still closes the page.
 
 Returning to OpenCode hides the browser without closing Electron or its pages, so reopening is fast. **Remove browser
 tabs from strip** also hides every browser page item while keeping the runtime ready; `/browser` restores them.
-Closing the final tracked page closes the browser workspace. The session prepares an internal welcome page so reopening
-stays fast; Host Chrome presents it as **New tab** with an empty address instead of exposing its internal data URL.
+Closing the final tracked page closes the browser workspace and leaves no hidden replacement target. The Electron
+runtime stays ready; the next explicit `/browser`, `open`, or `new_tab` operation creates the next page. A plain
+`/browser` creates the local welcome page on demand and presents it as **New tab** with an empty address instead of
+exposing its internal data URL.
 
 Popup windows are deliberately denied. Use `+` or the controller's `new_tab` tool when a new tracked page is
 required.
@@ -132,16 +137,19 @@ The plugin automatically registers two local MCP servers:
 
 | MCP server                  | Responsibility                                                                        |
 | --------------------------- | ------------------------------------------------------------------------------------- |
-| `opencode-browser-control`  | Show, hide, navigate, create, list, select, and close tracked browser tabs.           |
+| `opencode-browser-control`  | Navigate, create, list, select, and close tracked browser tabs in the background.     |
 | `opencode-browser-devtools` | Inspect and operate the same live Electron pages through `chrome-devtools-mcp@1.6.0`. |
 
-The control server exposes eight tools: `show`, `open`, `new_tab`, `list_tabs`, `select_tab`, `close_tab`,
-`hide`, and `status`.
+The control server exposes six tools: `open`, `new_tab`, `list_tabs`, `select_tab`, `close_tab`, and `status`. Agent
+operations never select the Browser workspace. They can add a browser item to the terminal tab strip and update its
+loading state, but OpenCode stays active until the user runs `/browser` or selects that browser tab.
 
 For reliable agent automation:
 
-1. Use `open` for an ordinary single-page request; it reuses the selected tab, including the initial welcome tab.
-2. Use `new_tab` only when the user requests an additional tab or the task must keep multiple pages open concurrently.
+1. Use `open` for an ordinary single-page request; it reuses the selected tab in the background, including the initial
+   welcome tab.
+2. Use `new_tab` only when the user requests an additional tab or the task must keep multiple pages open concurrently;
+   the new page is also created and selected in the background.
 3. Read its stable logical `tabId`, current CDP `targetId`, URL, and title.
 4. Call Chrome DevTools MCP `list_pages` and match the live page by URL/title.
 5. Use Chrome DevTools MCP's numeric `pageId` for page-scoped DevTools operations.
@@ -159,6 +167,10 @@ retrying the same timed-out call.
 The plugin launches the installed Chrome DevTools MCP entry with Node.js. It does not use `npx`, download an MCP
 package during activation, discover system Chrome, or create a second browser runtime.
 
+DevTools screenshots default to WebP at quality 80 and are bounded to 640×640 during capture. The bound keeps even an
+explicit PNG capture below Chrome DevTools MCP's inline-image threshold, so screenshot results remain typed image
+attachments that OpenCode can render in the session feed instead of opaque temporary-file messages.
+
 If the plugin becomes active during an existing agent turn, begin a new turn so OpenCode can refresh its MCP tool
 catalog.
 
@@ -177,7 +189,8 @@ OpenCode Host UI
 
 Background pages remain available to the user and agent, idle their offscreen paint rate at 1 fps, and do not send
 frames to Kitty. Changing terminal geometry updates the selected page viewport and placement without stretching a stale
-frame.
+frame. Explicit foreground URL commands wait for that viewport and Presentation to be ready before navigation; agent
+navigation remains immediate and offscreen.
 
 OpenCode V2 currently exposes session tabs rather than arbitrary native plugin tabs. The plugin therefore renders a
 composite strip in the global app slot: OpenCode sessions and logical browser pages remain independently owned, while

bun.lock

@@ -1,7 +1,7 @@
 {
   "$schema": "https://json.schemastore.org/package.json",
   "name": "@neriousy/opencode-browser",
-  "version": "0.4.2",
+  "version": "0.5.0",
   "description": "An Electron browser surface for the OpenCode v2 TUI",
   "keywords": [
     "opencode",
@@ -65,7 +65,7 @@
     "@opentui/solid": "0.4.5",
     "@types/bun": "latest",
     "@types/node": "^24.0.0",
-    "opentui-browser": "0.3.1",
+    "opentui-browser": "0.3.2",
     "oxfmt": "0.41.0",
     "solid-js": "1.9.12",
     "typescript": "^5"

package.json

@@ -4,6 +4,7 @@ import { Schema } from "effect"
 import { ElectronBrowser, type ElectronPage } from "opentui-browser"
 import { createServer } from "node:http"
 import { createRequire } from "node:module"
+import { DEVTOOLS_SCREENSHOT_ARGUMENTS } from "../src/mcp.js"
 
 const require = createRequire(import.meta.url)
 const devtoolsMcpEntry = require.resolve("chrome-devtools-mcp/build/src/bin/chrome-devtools-mcp.js")
@@ -12,15 +13,21 @@ const McpTextContent = Schema.Struct({
   type: Schema.Literal("text"),
   text: Schema.String,
 })
+const McpImageContent = Schema.Struct({
+  type: Schema.Literal("image"),
+  data: Schema.String,
+  mimeType: Schema.String,
+})
 const McpContent = Schema.Array(Schema.Unknown)
 const decodeMcpContent = Schema.decodeUnknownSync(McpContent)
 const isMcpTextContent = Schema.is(McpTextContent)
+const isMcpImageContent = Schema.is(McpImageContent)
 
 const marker = `opencode-browser-mcp-${crypto.randomUUID()}`
 const server = createServer((_request, response) => {
   response.writeHead(200, { "content-type": "text/html; charset=utf-8" })
   response.end(
-    `<!doctype html><title>OpenCode Browser MCP Smoke</title><script>globalThis.__mcpMarker=${JSON.stringify(marker)}</script>`,
+    `<!doctype html><style>html,body{margin:0}canvas{display:block}</style><title>OpenCode Browser MCP Smoke</title><canvas width="1200" height="1200"></canvas><script>globalThis.__mcpMarker=${JSON.stringify(marker)};const canvas=document.querySelector("canvas");const context=canvas.getContext("2d");const image=context.createImageData(canvas.width,canvas.height);let value=0x12345678;for(let offset=0;offset<image.data.length;offset+=4){value^=value<<13;value^=value>>>17;value^=value<<5;image.data[offset]=value;image.data[offset+1]=value>>>8;image.data[offset+2]=value>>>16;image.data[offset+3]=255}context.putImageData(image,0,0)</script>`,
   )
 })
 
@@ -47,6 +54,7 @@ try {
       "--experimentalPageIdRouting",
       "--no-usage-statistics",
       "--no-performance-crux",
+      ...DEVTOOLS_SCREENSHOT_ARGUMENTS,
     ],
     stderr: "pipe",
   })
@@ -55,11 +63,11 @@ try {
     stderr += String(chunk)
   })
 
-  client = new Client({ name: "opencode-browser-smoke", version: "0.4.0" })
+  client = new Client({ name: "opencode-browser-smoke", version: "0.5.0" })
   await client.connect(transport)
 
   const listedTools = await client.listTools()
-  for (const required of ["list_pages", "evaluate_script"]) {
+  for (const required of ["list_pages", "evaluate_script", "take_screenshot"]) {
     if (!listedTools.tools.some((tool) => tool.name === required)) {
       throw new Error(`chrome-devtools-mcp did not expose ${required}\n${stderr}`)
     }
@@ -85,13 +93,25 @@ try {
     throw new Error(`DevTools evaluated the wrong page:\n${evaluationText}`)
   }
 
+  const screenshot = await client.callTool({
+    name: "take_screenshot",
+    arguments: { pageId, fullPage: true },
+  })
+  if (screenshot.isError) throw new Error(textContent(screenshot.content))
+  const image = decodeMcpContent(screenshot.content).find(isMcpImageContent)
+  if (!image) throw new Error(`DevTools did not return an inline screenshot:\n${textContent(screenshot.content)}`)
+  const screenshotBytes = Buffer.from(image.data, "base64").byteLength
+  if (image.mimeType !== "image/webp") throw new Error(`Unexpected screenshot type: ${image.mimeType}`)
+  if (screenshotBytes >= 2_000_000) throw new Error(`Screenshot exceeded the inline response limit: ${screenshotBytes}`)
+
   console.log(
     JSON.stringify({
       chromeDevtoolsMcp: "1.6.0",
       pageId,
       targetId: page.targetId,
       url: pageUrl,
       marker,
+      screenshot: { mimeType: image.mimeType, bytes: screenshotBytes },
     }),
   )
 } finally {

scripts/smoke-devtools-mcp.ts

@@ -281,10 +281,13 @@ async function runHostRuntimeSmoke(application: string) {
     const tab = probe.createBrowserTabState()
     const session = probe.createBrowserSession(browser, page, page.url)
     const initialTabId = session.snapshot().selectedTabId
-    const temporary = await session.newTab("https://temporary.example")
+    const temporary = await session.newTab({ url: "https://temporary.example" })
     if (session.snapshot().selectedTabId !== temporary.tabId || temporary.page.targetId === page.targetId) {
       throw new Error("Packed BrowserSession did not select a newly created page")
     }
+    if (temporary.page.url !== "https://temporary.example") {
+      throw new Error("Packed BrowserSession did not forward the new page URL")
+    }
     if (!(await session.closeTab(temporary.tabId)) || session.snapshot().selectedTabId !== initialTabId) {
       throw new Error("Packed BrowserSession did not reconcile a closed page")
     }