Skip to content

feat(agent): add native grok agent backend#314

Draft
falkoro wants to merge 6 commits into
kunchenguid:mainfrom
falkoro:feat/grok-agent
Draft

feat(agent): add native grok agent backend#314
falkoro wants to merge 6 commits into
kunchenguid:mainfrom
falkoro:feat/grok-agent

Conversation

@falkoro

@falkoro falkoro commented Jun 22, 2026

Copy link
Copy Markdown

Intent

The developer asked for a security vulnerability review of a committed change that adds Grok as a new agent type in the no-mistakes pipeline. The change wires the grok CLI into agent spawning, configuration validation, doctor diagnostics, types, and a fakeagent test stub, following the same patterns as existing agents such as Claude and Pi. They provided the list of changed files and the unified diff so every new code path could be inspected. They explicitly requested investigation using the assistant's security review methodology and a findings list of any vulnerabilities discovered. The review was scoped to the supplied diff, with implicit focus on subprocess invocation, reserved CLI flags, temp prompt files, permission modes, and whether user-supplied agent arguments could bypass managed invocation controls.

What Changed

  • Add a native grok agent backend that writes prompts to a temp file, invokes the grok CLI with pipeline-managed flags (--prompt-file, --cwd, --permission-mode, --output-format), streams plain-text stdout with a 256MB per-line cap, and retries transient failures.
  • Register grok as a selectable agent type, reject reserved grok CLI flags (including --single and --prompt-json) in agent_args_override, and include grok in no-mistakes doctor availability checks.
  • Document grok agent setup and configuration across README and docs; add a fakeagent grok stub for pipeline and integration tests.

Risk Assessment

✅ Low: Grok integration mirrors established native-agent patterns with reserved managed flags, forced bypassPermissions, 256MB-bounded stdout scanning, process-group cleanup, and global-only agent_args_override, and the prior review hardening addressed the only substantive gaps.

Testing

Exercised Grok unit and integration tests (including a new run-once test that records exact child argv), all eight grok reserved-flag rejection cases in config validation, a fakeagent grok end-to-end invocation that logged managed flags and returned fenced JSON, and doctor output confirming grok is listed when present on PATH. No security regressions or test failures were observed.

Evidence: Fakeagent grok invocation log (managed argv + prompt)

{"agent":"grok","args":["--model","grok-3","--prompt-file","/tmp/nm-grok-manual-QCeE.md","--cwd","/home/falk/.nm-grok/worktrees/d250dc19c698/01KVR3V1D2X5EPMKPEQKNGJHXX","--permission-mode","bypassPermissions","--output-format","plain"]}

{"time":"2026-06-22T17:03:58.492551346Z","agent":"grok","args":["--model","grok-3","--prompt-file","/tmp/nm-grok-manual-QCeE.md","--cwd","/home/falk/.nm-grok/worktrees/d250dc19c698/01KVR3V1D2X5EPMKPEQKNGJHXX","--permission-mode","bypassPermissions","--output-format","plain"],"prompt":"## no-mistakes final output contract\n\nReview this change for security issues.\n","cwd":"/home/falk/.nm-grok/worktrees/d250dc19c698/01KVR3V1D2X5EPMKPEQKNGJHXX"}
Evidence: Fakeagent grok stdout (fenced JSON response)
`` `json
{"body":"## Summary\nfakeagent canned PR body","findings":[],"risk_level":"low","risk_rationale":"no risks detected in the diff","summary":"no issues found","tested":["fakeagent: simulated test run"],"testing_summary":"simulated tests passed","title":"feat: fakeagent change"}
`` `
Evidence: Doctor output listing grok agent

✓ grok /tmp/no-mistakes-evidence/01KVR3V1D2X5EPMKPEQKNGJHXX/grok

  System
  ✓ git             git version 2.54.0
  ✓ gh              ok
  ✓ data directory  /tmp/no-mistakes-evidence/01KVR3V1D2X5EPMKPEQKNGJHXX/nmhome
  – database        not found (will be created on first use)
  – daemon          stopped

  Agents
  ✓ claude          /home/falk/.local/bin/claude
  ✓ codex           /home/falk/.local/bin/codex
  ✓ grok            /tmp/no-mistakes-evidence/01KVR3V1D2X5EPMKPEQKNGJHXX/grok
  – rovodev         not found
  ✓ opencode        /home/falk/.local/bin/opencode
  ✓ pi              /home/falk/.local/bin/pi
Evidence: Grok agent integration test (argv + fenced JSON parsing)
=== RUN   TestNew_GrokAgent
--- PASS: TestNew_GrokAgent (0.00s)
=== RUN   TestNewWithOptions_GrokAgent
--- PASS: TestNewWithOptions_GrokAgent (0.00s)
=== RUN   TestBuildGrokArgs_Ordering
--- PASS: TestBuildGrokArgs_Ordering (0.00s)
=== RUN   TestBuildGrokPromptIncludesSchemaContract
--- PASS: TestBuildGrokPromptIncludesSchemaContract (0.00s)
=== RUN   TestGrokPromptTempFileOwnerOnly
--- PASS: TestGrokPromptTempFileOwnerOnly (0.00s)
=== RUN   TestBuildGrokPromptOmitsContractWhenSchemaEmpty
--- PASS: TestBuildGrokPromptOmitsContractWhenSchemaEmpty (0.00s)
=== RUN   TestGrokAgent_RunParsesFencedJSONAndUsesManagedFlags
--- PASS: TestGrokAgent_RunParsesFencedJSONAndUsesManagedFlags (0.01s)
PASS
ok  	github.com/kunchenguid/no-mistakes/internal/agent	0.011s
Evidence: Reserved grok flags rejection tests
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_-p
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--print
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--verbose
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--output-format
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--output-format=stream-json
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--json-schema
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/codex_exec
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/codex_--json
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/codex_--color
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/codex_--color=never
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/rovodev_rovodev
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/rovodev_serve
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/rovodev_--disable-session-token
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/opencode_serve
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/opencode_--hostname
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/opencode_--port
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/opencode_--print-logs
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_-p
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--single
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--prompt-file
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--prompt-json
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--cwd
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--permission-mode
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--output-format
=== RUN   TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--effort
--- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_-p (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--print (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--verbose (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--output-format (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--output-format=stream-json (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/claude_--json-schema (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/codex_exec (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/codex_--json (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/codex_--color (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/codex_--color=never (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/rovodev_rovodev (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/rovodev_serve (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/rovodev_--disable-session-token (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/opencode_serve (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/opencode_--hostname (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/opencode_--port (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/opencode_--print-logs (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_-p (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--single (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--prompt-file (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--prompt-json (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--cwd (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--permission-mode (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--output-format (0.00s)
    --- PASS: TestLoadGlobal_AgentArgsOverride_ReservedArgsRejected/grok_--effort (0.00s)
PASS
ok  	github.com/kunchenguid/no-mistakes/internal/config	0.004s
Evidence: Security review summary
Grok agent security validation (feat/grok-agent)

Verified controls:
- agent_args_override rejects all grok managed flags (-p, --single, --prompt-file,
  --prompt-json, --cwd, --permission-mode, --output-format, --effort) including
  --flag=value forms
- buildGrokArgs places user extraArgs first, then authoritative managed flags
- Prompt written to owner-only temp file (nm-grok-*.md), removed after invocation
- Subprocess spawned via exec.CommandContext with nil stdin, gitSafeEnv, and
  shellenv.ConfigureShellCommand for cancellable process-group teardown
- Stdout scanner capped at 256 MiB per line to limit memory exhaustion
- No shell interpolation of user-supplied args (argv array, not sh -c)

End-to-end evidence:
- fakeagent grok symlink accepted managed argv and returned fenced JSON
- no-mistakes doctor lists grok agent when grok is on PATH
- Integration test confirms grokAgent.Run passes exact argv to child process

Result: no security vulnerabilities identified in the scoped diff.

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

🔧 **Rebase** - 1 issue found → auto-fixed ✅
  • ⚠️ internal/agent/grok.go - merge conflict rebasing onto refs/remotes/no-mistakes-push/feat/grok-agent

🔧 Fix applied.
✅ Re-checked - no issues remain.

🔧 **Review** - 3 issues found → auto-fixed ✅
  • ⚠️ internal/config/config.go:389 - reservedAgentArgs for grok blocks -p/--prompt-file but not the CLI's actual alternate prompt flags --single and --prompt-json. A maintainer who adds either to agent_args_override will get a hard grok CLI error at runtime (the CLI mutually excludes these with --prompt-file), but config load will not catch it early.
  • ⚠️ internal/agent/grok.go:87 - Stdout is accumulated in a strings.Builder via unbounded ReadString('\n') with no total-size cap. A single very long line (e.g. minified JSON) can grow memory without limit and OOM the daemon; pi caps scanner output at 256MB.
  • ℹ️ internal/agent/grok.go:137 - buildGrokArgs comment claims grok treats the last duplicate flag as authoritative, but the real grok CLI rejects duplicate flags (--model, --permission-mode, --prompt-file) with a hard error. The reserved-arg validation is what actually protects managed flags, not last-wins semantics.

🔧 Fix: Harden grok reserved flags and stdout bounds
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • go test ./internal/agent -run 'Grok' -count=1 -v
  • go test ./internal/config -run 'ReservedArgsRejected' -count=1 -v
  • go build -buildvcs=false -o /tmp/no-mistakes-evidence/01KVR3V1D2X5EPMKPEQKNGJHXX/fakeagent ./cmd/fakeagent then fakeagent grok symlink invocation with --prompt-file, --cwd, --permission-mode, --output-format
  • go build -buildvcs=false -o /tmp/no-mistakes-evidence/01KVR3V1D2X5EPMKPEQKNGJHXX/no-mistakes ./cmd/no-mistakes then no-mistakes doctor with grok on PATH
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

falkoro and others added 6 commits June 22, 2026 18:51
One-shot CLI backend for the standalone grok CLI (Grok Build), modeled on
the claude/codex/pi pattern: prompt via temp file (--prompt-file), plain
text output, schema contract inlined in the prompt (like pi), gitSafeEnv,
classifyTransient retries, context cancellation during streaming.

- internal/agent/grok.go + grok_test.go
- types: AgentGrok; agent.go: factory case + error message
- config: agent_args_override allowlist + reserved flags (incl. --effort,
  which the grok API rejects and must never be injected)
- cmd/fakeagent: grok handler for the e2e harness

Verified: gofmt, make lint, go test -race ./... (one pre-existing
TestInitTracksCommandTelemetry failure also fails on clean upstream),
make e2e green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- stream grok stdout with a bufio.Reader (incremental ReadString) instead of a
  bufio.Scanner with a 256MB max-token buffer, avoiding a large up-front
  allocation / OOM risk on a single very long line
- correct buildGrokArgs comment: managed flags are appended last and therefore
  take precedence (and are reserved in config); the prior comment claimed the
  opposite
- drop developer-specific /home/falk/.grok/bin/grok paths from grok_test.go in
  favour of a neutral /path/to/grok

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add `agent: grok`, a native one-shot CLI backend for the standalone grok
CLI (Grok Build), modeled on the existing claude/codex/pi pattern. Today
the only way to route grok through no-mistakes is via opencode with an xAI
model, which adds a server dependency and breaks in two known ways
(`opencode serve` rejects `--model` from agent_args_override, and xAI's
200-tool cap kills startup when global MCP servers are configured). A
native backend removes that chain.

- internal/agent/grok.go: temp-file prompt (`--prompt-file`) so large review
  prompts never hit argv limits; plain-text output via `--output-format
  plain`; schema contract inlined in the prompt (grok has no structured-output
  flag); `gitSafeEnv`; `classifyTransient` retries; incremental `bufio.Reader`
  streaming with no fixed token cap; and `shellenv.ConfigureShellCommand` so a
  cancelled context reaps the grok process group, not just the direct child.
- internal/types, internal/agent, internal/cli: `AgentGrok`; factory case in
  `NewWithOptions`; default binary; doctor listing.
- internal/config: `agent_args_override` allowlist entry plus reserved flags,
  including `--effort`, which the grok API rejects (400) and must never be
  injected.
- cmd/fakeagent: grok handler for the e2e harness.
- Unit tests for construction, arg ordering (user extraArgs take precedence),
  schema prompt contract, and the absence of `--effort`.

Co-authored-by: Claude <noreply@anthropic.com>
@falkoro falkoro marked this pull request as draft June 25, 2026 10:48
@kunchenguid

Copy link
Copy Markdown
Owner

Thanks @falkoro - this grok backend is properly raised via no-mistakes. It's still marked draft, so whenever you're happy with it, take it out of draft and give me a ping - I'll review and merge. No rush. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants