Skip to content

feat(agent): add GitHub Copilot CLI agent backend#318

Merged
kunchenguid merged 4 commits into
kunchenguid:mainfrom
kevinamick:feat/github-copilot-agent
Jun 26, 2026
Merged

feat(agent): add GitHub Copilot CLI agent backend#318
kunchenguid merged 4 commits into
kunchenguid:mainfrom
kevinamick:feat/github-copilot-agent

Conversation

@kevinamick

Copy link
Copy Markdown
Contributor

Intent

Add the GitHub Copilot CLI as a first-class no-mistakes agent at full parity with the existing backends (subprocess agent running 'copilot -p' with JSONL output and inlined schema, plus a ~/.copilot/session-state transcript intent reader), wired into auto-detection, config, reserved flags, doctor, and docs. Re-running only push/pr/ci to open the PR and watch CI after switching to a GitHub account permitted to create the PR.

What Changed

  • Added a copilot agent backend (internal/agent/copilot.go) that runs copilot -p with JSONL output and an inlined response schema, plus a matching ~/.copilot/session-state transcript intent reader (internal/intent/reader_copilot.go).
  • Wired the new AgentCopilot type into auto-detection probe order, default binary mapping, agent_args_override with reserved flags (-p, --prompt, --output-format, --no-color), the doctor agent check, and the auto agent factory.
  • Updated configuration defaults, AGENTS.md, the intent package doc comment, and the docs site to list Copilot CLI alongside the existing Claude, Codex, Rovo Dev, OpenCode, and Pi backends.

Risk Assessment

✅ Low: The change is additive and well-bounded, faithfully mirrors the existing pi/claude agent and reader patterns with comprehensive unit tests and accurate docs, all call sites are wired, and the only prior actionable finding was correctly resolved with no new issues introduced.

Testing

Baseline go build ./... is clean. Ran the targeted copilot agent and intent-reader suites plus the agent/config/types packages: every copilot-specific test passes, including two that spawn a real fake copilot process and exercise the full copilot -p JSONL streaming/parse path and non-zero-exit error surfacing, and reader tests that discover/load ~/.copilot/session-state transcripts with cwd/time filtering and synthetic-message exclusion. For reviewer-visible product evidence I added a portable in-process test that renders the actual no-mistakes doctor CLI report and confirms copilot is now a first-class Agents entry detected on PATH (captured as a transcript artifact). Two failing tests (internal/cli TestRoot*/telemetry and internal/intent symlink) are pre-existing, reproduced, environment-only issues (git 2.54 bare-repo config behavior; Windows symlink privilege) that the additive Copilot change does not touch. Transient build output (bin/) was removed; the new test file is intentionally kept.

Evidence: Rendered `no-mistakes doctor` report — copilot listed and detected as a first-class agent

Agents – claude not found – codex not found – rovodev not found – opencode not found – pi not found ✓ copilot <bin>\copilot.cmd (detected on PATH, same probe as the other agents)

=== RUN   TestDoctorListsCopilotAgent
    doctor_copilot_test.go:38: rendered `no-mistakes doctor` report:
          System
          ✓ git             git version 2.54.0.windows.1
          ✓ gh              ok
          ✓ data directory  C:\Users\kamick\AppData\Local\Temp\TestDoctorListsCopilotAgent740455664\001
          – database        not found (will be created on first use)
          – daemon          stopped
        
          Agents
          – claude          not found
          – codex           not found
          – rovodev         not found
          – opencode        not found
          – pi              not found
          ✓ copilot         C:\Users\kamick\AppData\Local\Temp\TestDoctorListsCopilotAgent740455664\002\copilot.cmd
--- PASS: TestDoctorListsCopilotAgent (0.17s)
PASS
ok  	github.com/kunchenguid/no-mistakes/internal/cli	3.483s
Evidence: Copilot agent + transcript-reader test transcript (incl. real-subprocess copilot -p JSONL run)
=== RUN   TestCopilotAgent_BuildArgs
--- PASS: TestCopilotAgent_BuildArgs (0.00s)
=== RUN   TestCopilotAgent_BuildArgs_ExtraArgsFirst
--- PASS: TestCopilotAgent_BuildArgs_ExtraArgsFirst (0.00s)
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-all-tools
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-all
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/yolo
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-tool
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-tool-eq
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/excluded-tools
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/available-tools
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/deny-tool
=== RUN   TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-all-paths
--- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-all-tools (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-all (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/yolo (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-tool (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-tool-eq (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/excluded-tools (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/available-tools (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/deny-tool (0.00s)
    --- PASS: TestCopilotAgent_BuildArgs_UserPermissionSuppressesDefault/allow-all-paths (0.00s)
=== RUN   TestCopilotAgent_BuildArgs_UserAskUserSuppressesDefault
--- PASS: TestCopilotAgent_BuildArgs_UserAskUserSuppressesDefault (0.00s)
=== RUN   TestBuildCopilotPrompt_InlinesSchema
--- PASS: TestBuildCopilotPrompt_InlinesSchema (0.00s)
=== RUN   TestBuildCopilotPrompt_NoSchemaIsUnchanged
--- PASS: TestBuildCopilotPrompt_NoSchemaIsUnchanged (0.00s)
=== RUN   TestParseCopilotEvents_FinalMessageAndUsage
--- PASS: TestParseCopilotEvents_FinalMessageAndUsage (0.00s)
=== RUN   TestParseCopilotEvents_CapturesErrorEvent
--- PASS: TestParseCopilotEvents_CapturesErrorEvent (0.00s)
=== RUN   TestParseCopilotEvents_SkipsMalformedAndSessionLines
--- PASS: TestParseCopilotEvents_SkipsMalformedAndSessionLines (0.00s)
=== RUN   TestCopilotAgent_RunParsesJSONOutput
--- PASS: TestCopilotAgent_RunParsesJSONOutput (0.19s)
=== RUN   TestCopilotAgent_RunReportsErrorOnNonZeroExit
--- PASS: TestCopilotAgent_RunReportsErrorOnNonZeroExit (0.18s)
PASS
ok  	github.com/kunchenguid/no-mistakes/internal/agent	1.172s
=== RUN   TestCopilotReader_DiscoversAndLoadsRealMessages
--- PASS: TestCopilotReader_DiscoversAndLoadsRealMessages (0.31s)
=== RUN   TestCopilotReader_FiltersByCWD
--- PASS: TestCopilotReader_FiltersByCWD (0.61s)
=== RUN   TestCopilotReader_TimeWindow
--- PASS: TestCopilotReader_TimeWindow (0.25s)
=== RUN   TestCopilotReader_NoHomeNoCrash
--- PASS: TestCopilotReader_NoHomeNoCrash (0.00s)
=== RUN   TestCopilotReader_SkipsSessionWithoutStartEvent
--- PASS: TestCopilotReader_SkipsSessionWithoutStartEvent (0.29s)
PASS
ok  	github.com/kunchenguid/no-mistakes/internal/intent	4.601s
testing: warning: no tests to run
PASS
ok  	github.com/kunchenguid/no-mistakes/internal/config	2.777s [no tests to run]
- Outcome: ⚠️ 3 infos across 1 run (18m38s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 3 issues found → auto-fixed ✅
  • ⚠️ internal/agent/copilot.go:145 - copilotUserSetPermissionMode treats --available-tools/--excluded-tools as permission flags, so supplying only one of them via agent_args_override suppresses the default --allow-all-tools. Those flags restrict the available tool set, not the approval policy; without an approval grant the non-interactive -p run can block or fail on tool-approval prompts, because the always-added --no-ask-user only disables the ask_user tool, it does not auto-approve tool execution. Confirm whether --allow-all-tools should still be appended when the user supplied only tool-set (not approval) flags.
  • ℹ️ internal/agent/copilot.go:248 - parseCopilotEvents only accumulates OutputTokens; copilotEventData declares no input/cache token fields and the terminal result event's usage object (present in fixtures, e.g. {"premiumRequests":1}) is ignored. Every other agent records InputTokens/cache tokens, so copilot usage/cost reporting will undercount. Likely acceptable given Copilot's premium-request billing model, but it is an intentional-looking asymmetry worth a quick confirm.
  • ℹ️ internal/agent/copilot.go:96 - The error message captured from error/assistant.abort events (copilotErr) is only surfaced when waitErr != nil or exitCode != 0. If copilot reports an error/abort but still exits 0 with no non-empty assistant.message content, finalizeTextResult returns a generic "copilot returned no text output" and the real reason is discarded. There is also no delta-accumulation fallback (unlike pi's finalText), so a final answer delivered only through assistant.message_delta would be lost. Consider surfacing copilotErr (and/or accumulated deltas) when lastMessage is empty.

🔧 Fix: Restrict copilot allow-all-tools suppression to approval grants
✅ Re-checked - no issues remain.

⚠️ **Test** - 3 infos
  • ℹ️ internal/cli/root_test.go - Pre-existing, environment-only failures unrelated to the Copilot change: internal/cli TestRoot* and TestInitTracksCommandTelemetry fail because git 2.54.0.windows.1 returns fatal: not in a git directory (exit 128) for git config receive.advertisePushOptions true against a bare repo (invoked by internal/gate and internal/daemon during init). Reproduced directly: git init --bare &lt;dir&gt;; git -C &lt;dir&gt; config receive.advertisePushOptions true => exit 128. The Copilot change touches none of gate/daemon/git/init, so these are not regressions; flagged for reviewer awareness so CI noise isn't mistaken for a Copilot defect.
  • ℹ️ internal/intent/disambiguator_test.go:234 - Pre-existing, environment-only failure unrelated to the Copilot change: internal/intent TestAgentDisambiguatorPreservesPreexistingIgnoredSymlink fails with A required privilege is not held by the client because creating a symlink on this Windows host requires Developer Mode/elevated privileges. All Copilot reader tests in the same package pass. Flagged for awareness only.
  • ℹ️ internal/cli/doctor_copilot_test.go - new test file written by agent: internal/cli/doctor_copilot_test.go
  • go build ./... (clean)
  • go test -run &#39;Copilot&#39; -v ./internal/agent ./internal/intent ./internal/config — all copilot agent + transcript-reader tests pass, including TestCopilotAgent_RunParsesJSONOutput and TestCopilotAgent_RunReportsErrorOnNonZeroExit which spawn a real fake copilot subprocess and parse its JSONL end-to-end
  • go test ./internal/agent ./internal/config ./internal/types (green: agent registration, defaultBinary, auto-detect probe order, agent_args_override/reserved flags, AgentCopilot const)
  • Added and ran new portable test go test -run &#39;TestDoctorListsCopilotAgent&#39; -v ./internal/cli — renders the real no-mistakes doctor report and asserts copilot appears in Agents and is detected at its on-PATH binary
  • go test -run &#39;TestDoctor&#39; -v ./internal/cli (doctor telemetry test passes)
  • Reproduced git init --bare &lt;dir&gt;; git -C &lt;dir&gt; config receive.advertisePushOptions true => fatal: not in a git directory (exit 128) to confirm the cli TestRoot*/telemetry failures are git 2.54 environment issues, not regressions
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

kevinamick and others added 4 commits June 23, 2026 14:05
Add the GitHub Copilot CLI as a first-class agent at parity with the
existing backends. New copilot agent runs `copilot -p` with JSONL
output, streams assistant deltas, and inlines the JSON schema into the
prompt. Wire it into auto-detection, agent_args_override, reserved
flags, doctor, and the default config. Add a copilot intent transcript
reader over ~/.copilot/session-state and register it. Update docs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The fix step requests schema-constrained output, but Copilot 1.0.x is
non-deterministic about honoring the JSON-only output contract: it often
emits the schema JSON in an earlier assistant message and then closes with
a prose summary (e.g. "Now I've applied all four fixes..."). The previous
code kept only the last assistant.message, so extraction had nothing to
recover and the whole step failed with an undiagnosable error:

    copilot output parse: invalid character 'N' looking for beginning of value

parseCopilotEvents now collects every non-empty assistant.message, and
finalizeCopilotResult tries each newest-first against the schema, returning
the first that parses. finalizeTextResult also embeds a trimmed output
snippet in parse-failure errors so future failures are debuggable.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@thecrockster

Copy link
Copy Markdown

Please @kunchenguid can you add support for Copilot CLI

@kunchenguid
kunchenguid merged commit aedb3a1 into kunchenguid:main Jun 26, 2026
9 checks passed
@kunchenguid

Copy link
Copy Markdown
Owner

Thanks @kevinamick - merged! 🎉 A clean, full-parity Copilot backend: the arg-array subprocess wiring (through ConfigureShellCommand), the intent-reader, and the trust-boundary handling were all spot on. Really appreciate the thorough work and docs.

@thecrockster

Copy link
Copy Markdown

Excellent! Thank you both @kevinamick and @kunchenguid

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants