Skip to content

feat(core): optional intent label on interaction tools — audit/HITL traceability (BrowserMCP adoption A-2) #894

@shaun0927

Description

@shaun0927

Tier: core (additive; existing tool surface preserved)
PR target: develop

Background

Once #831 ("formalize read_page refs as canonical interaction entrypoint") lands, the typical interaction call shape becomes:

interact { tabId: "t1", ref: "ax_42", action: "click" }
form_input { tabId: "t1", ref: "ax_77", value: "alice@example.com" }

A ref like ax_42 is opaque. The downstream consumers of every interaction call —

  • oc_journal (src/tools/journal.ts)
  • Ralph S7 HITL escalation context (src/utils/ralph/)
  • Trace JSONL (src/core/trace/storage.ts)

— end up storing only { ref: "ax_42", action: "click" }. The original query (natural-language description of the target) is no longer required when a ref is supplied, so the agent's intent disappears from the audit trail.

(Note: oc_evidence_bundle (src/core/contracts/evidence-bundle.ts) is intentionally NOT in this list. Its current shape has no context object — surfacing intent there would require a separate evidence-bundle schema change and is deferred to a follow-up.)

browsermcp (Apache-2.0) makes the human label a required, separate arg on every interaction tool:

// browsermcp/src/tools/snapshot.ts
{ element: "Submit button", ref: "ax_42" } // element used only for logging

The effect is two-fold:

  1. The agent self-validates intent before mutating the DOM ("am I really clicking Submit?").
  2. Logs and HITL contexts are human-readable without resolving the ref back to a snapshot.

OpenChrome already has the substrate (oc_journal, evidence bundle, Ralph HITL) — it just doesn't carry the label. This issue adds a single optional field; no behavioural change for callers that omit it.

Proposed Implementation

  1. Add intent?: string (max 120 chars) to the inputSchema of these tools (under src/tools/):
    • interact.ts (name: 'interact')
    • form-input.ts (name: 'form_input')
    • fill-form.ts (name: 'fill_form')
    • drag-drop.ts (name: 'drag_drop')
    • file-upload.ts (name: 'file_upload')
  2. Tool description: append a single line to each tool's top-level description:
    'Pass intent="..." (≤120 chars) to label this action in audit logs.'
  3. Semantics:
    • The intent value is never used for element resolution. It is purely descriptive.
    • When intent is provided by the caller, it is included in: the trace TraceEvent for that call (new top-level field intent), oc_journal recent entry (new top-level field intent), and the Ralph S7 HITL escalation payload (new field intent).
    • When intent is omitted, the trace/journal/HITL records do not emit an intent key at all (i.e., absent ≠ empty string). This keeps the on-disk format byte-identical to v1.11.0 when the field is not used.
    • Empty string and strings > 120 chars: schema rejects with a clear INVALID_INTENT error code. No silent truncation. No DOM action is performed when validation fails.
  4. Hint engine integration (src/hints/): no new hints in this issue; an intent-missing hint may be added later as a follow-up if we observe the field being skipped on high-stakes calls.
  5. No CLI flag, no env var, no settings file change.

Acceptance Criteria

  • intent?: string field added to all 5 tools' inputSchemas with max-length 120 enforcement.
  • Tool descriptions updated with the single-line guidance.
  • Trace JSONL writer (src/core/trace/storage.ts) records intent as a top-level field on TraceEvent for the relevant tool calls only when the caller provided it.
  • oc_journal (kind recent) surfaces intent in the entry shape only when present.
  • Ralph S7 HITL escalation payload includes intent only when present.
  • Validation: empty "" and >120 chars produce INVALID_INTENT error; omitting the field is allowed and produces v1.11.0-identical output.
  • Backward-compatibility test: omitting intent produces a byte-identical response to v1.11.0 for the same input (snapshot test on a frozen fixture).
  • npm run build && npm test green.
  • PR targets develop.

Real verification (post-merge, via openchrome MCP)

A reproducer script at scripts/verify/A2-intent-passthrough.mjs codifies the following steps so any reviewer can re-run them.

  1. Build and launch: npm run build && node dist/index.js.
  2. mcp__openchrome__navigate to https://httpbin.org/forms/post.
  3. mcp__openchrome__read_page mode=ax → collect refs for custname, custtel, custemail.
  4. Three calls in sequence:
    • mcp__openchrome__form_input { ref: <custname>, value: "Alice", intent: "fill customer name" }
    • mcp__openchrome__form_input { ref: <custtel>, value: "555-0100", intent: "fill phone" }
    • mcp__openchrome__form_input { ref: <custemail>, value: "alice@example.com" } ← intent omitted
  5. mcp__openchrome__oc_journal { kind: "recent", limit: 5 } → response MUST contain three form_input entries; the first two have an intent field with the supplied value; the third entry has no intent key at all (absence-vs-empty test).
  6. Negative path: mcp__openchrome__interact { tabId: <t>, query: "Submit", intent: "" } → returns INVALID_INTENT error; no DOM action performed (verify via oc_journal: zero new interact entries after this call).
  7. Negative path: same with intent of length 200 → returns INVALID_INTENT. No truncation. Zero new journal entries.
  8. Trace inspection: read ~/.openchrome/traces/<sessionId>/*.jsonl; the first two form_input event lines contain "intent":"..."; the third event line MUST NOT contain the key "intent" (verified by jq 'has("intent")').
  9. Backward-compat path: rerun the entire sequence omitting intent on every call; resulting JSONL is byte-identical to a checked-in v1.11.0 fixture for the same sequence (tests/fixtures/intent/v1.11.0-trace.jsonl).

Pass criteria: steps 4–9 all succeed; step 5 readouts match expected (including the absence test); steps 6–7 produce error without side-effects; step 9 produces byte-identical output to fixture.

Out of scope (deferred)

  • Surfacing intent in oc_evidence_bundle — requires a separate change to EvidenceBundleSnapshot (src/core/contracts/evidence-bundle.ts has no context object today). File as a follow-up after this lands and we have real journal data to inform the bundle's schema.
  • An intent-missing hint-engine rule (separate follow-up if usage data justifies it).
  • Backfilling intent on computer/act (higher-level orchestration tools — their high-level natural-language input already serves this role).
  • Backfilling intent on javascript_tool (free-form code; intent would duplicate the code body).
  • LLM-side prompt updates to teach agents to populate intent (host concern, not server).

Dependencies

Effort

XS (~0.5–1 dev-day). Five schema additions, one trace-writer change, one journal-shape change, Ralph HITL payload field, snapshot tests.

References

Revision history

  • 2026-05-12 r1: Initial draft.
  • 2026-05-12 r2: Critic-driven revision.
    • Removed false claims about oc_evidence_bundle.context.intent and the includeIntent parameter — neither exists; evidence-bundle integration moved to Out of scope as a follow-up.
    • Clarified absent-vs-empty semantics for trace/journal records: omitted intent produces no key, not an empty string.
    • Added a byte-identity backward-compat test against a checked-in v1.11.0 fixture.

OpenChrome 실검증 체크리스트

2026-05-14 재검증 완료. 최신 origin/develop 코드, targeted Jest/lint, OpenChrome CLI 실호출, localhost fixture 산출물로 직접 확인 가능한 항목만 close 근거로 사용했다.

검증 대상

검증 증거

  • npm run build 통과.
  • npm run lint:tier 통과: 521 modules / 1239 dependencies, no dependency violations.
  • npm run lint:tool-schemas 통과: 82 baselined violations, 0 new.
  • targeted Jest 통과: 38 passed / 1 skipped suites, 436 passed / 1 skipped tests.
  • OpenChrome CLI 실호출: oc_connection_health connected, localhost fixture navigate 성공.
  • OpenChrome tools/list introspection에서 관련 default 또는 pilot-gated tool surface 존재 확인.
  • 대표 bounded diagnostic 호출이 구조화된 성공/오류 응답을 반환함을 확인.

이슈별 코드/테스트 근거

  • 관련 구현/문서/테스트 파일이 최신 트리에 존재하고 targeted 검증에 포함됨:
    • tests/tools/intent.test.ts
    • src/utils/ralph/hitl-escalation.ts
    • src/tools/interact.ts
    • src/tools/form-input.ts
    • src/tools/fill-form.ts

산출물

  • 증거 로그: .omx/reverify-evidence/targeted-jest.log
  • 증거 로그: .omx/reverify-evidence/lint-tier.log
  • 증거 로그: .omx/reverify-evidence/lint-tool-schemas.log
  • 증거 로그: .omx/reverify-evidence/openchrome-live-smoke.log

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1P1 highenhancementNew feature or requestobservabilityObservabilityreliabilityReliability and stability improvement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions