[copilot-cli-research] Copilot CLI Deep Research - 2026-08-03

[github-actions]

Analysis Date: 2026-08-03
Repository: github/gh-aw
Scope: 269 total workflows, 98 using engine: {id: copilot, ...} block form (33 using shorthand engine: copilot)

---

📊 Executive Summary

Research Topic: Copilot CLI Optimization Opportunities
Key Findings:

1. --share flag is used in exactly 1 workflow (this research workflow) despite being available for conversation sharing/tracking.
2. 5 custom agent files (create-safe-output-type, custom-engine-implementation, grumpy-reviewer, interactive-agent-designer, w3c-specification-writer) are defined under .github/agents/ but referenced by zero workflows — unchanged since the prior research run.
3. --autopilot/max-continuations is used in only 11 of 98 copilot-block workflows, leaving most workflows to rely on a single non-continuing run even where multi-step tasks are common.
4. copilot-sdk: true adoption is strong (68 workflows) and growing, suggesting a broad shift toward SDK mode — worth confirming CLI-only mode is still intentionally used elsewhere.
5. Network/firewall configuration (network:) appears in only 42/98 copilot workflows; only 1 workflow references firewall: explicitly, meaning most rely on defaults rather than explicit allowlists.

Primary Recommendation: Clean up orphaned agent files (delete or wire them into workflows) and evaluate --share adoption for workflows with complex/long-running interactions where conversation replay would aid debugging.

This is a repeat analysis (previous run: 30732964595, 2026-08-02). The orphaned-agent-file finding is a recurring, unresolved item across two consecutive research runs — it's a low-effort cleanup that keeps being deferred.

---

Critical Findings

🔴 High Priority Issues

• 5 unreferenced custom agent files sitting in .github/agents/ for at least 2 research cycles with no workflow using them via engine.agent. These represent either dead code or missed integration opportunities.
• Low network:/firewall explicit configuration (42/98, ~43%) — many copilot workflows run with default network settings rather than reviewed allowlists, which is a security posture gap worth auditing individually.

🟡 Medium Priority Opportunities

• --autopilot/max-continuations underused (11/98) — workflows doing multi-step repo analysis or iterative fixes could benefit from continuation support instead of single-shot runs.
• --share flag essentially unused (1/269) — no visibility into whether teams want conversation-sharing for debugging failed runs.
• Most bare: true usage (14 workflows) suppresses custom instructions; confirm this is intentional per-workflow rather than copy-paste.

---

View Full Analysis

1️⃣ Current State Analysis

View Copilot CLI Capabilities Inventory

Copilot CLI Capabilities Inventory (from pkg/workflow/copilot_engine*.go)

• CLI flags emitted by gh-aw: --disable-builtin-mcps (always), --no-ask-user (autonomous runs, v1.0.19+), --agent <name> (via engine.agent), --autopilot --max-autopilot-continues N (via max-continuations), --add-dir (multiple, for /tmp, cache dirs), --allow-all-paths (when edit tool enabled), --no-custom-instructions (bare mode), --headless --no-auto-update --port (SDK mode), --log-level all --log-dir.
• Engine config options: id: copilot, version, model, agent, bare, copilot-sdk, driver (custom SDK driver — JS/Python/Go/Ruby, inline or file), args (raw passthrough), env, max-tool-denials (SDK-only), harness sub-keys mapped to GH_AW_HARNESS_* env vars.
• MCP support: RenderMCPConfig builds JSON MCP config (copilot_mcp.go); GitHub MCP toolsets used by 64/98, github: tool block by 72/98.
• Sandbox: AWF (Agentic Workflow Firewall) integration for max-turns; firewall enablement toggles HOME path handling for ARC/DinD topology.
• Not found in code: --share is not referenced anywhere in copilot_engine_execution.go — it appears only in this research prompt's example text, meaning it may not be an actual supported CLI flag in the installed Copilot CLI version, or it's unmapped to any engine.* config. This should be verified against the actual copilot binary help output before recommending it further.

View Usage Statistics

Usage Statistics

• Total Workflows: 269
• Copilot (shorthand engine: copilot): 33
• Copilot (block id: copilot): 98
• agent: sub-field usage: 84/98
• copilot-sdk: true usage: 68/98
• bare: true usage: 14/98
• args: (raw CLI passthrough) usage: 5 workflows (copilot-pr-merged-report.md, daily-cli-performance.md, eslint-miner.md, jsweep.md, smoke-copilot.md)
• network: config usage: 42/98
• safe-outputs: usage: 86/98
• cache-memory: usage: 25/98
• Timeout distribution: 20min (25), 30min (23), 15min (18), 10min (14), 5min (6), 45min (6), 60min (3), 120min (2), others (long tail)

---

2️⃣ Feature Usage Matrix

Feature Category	Available Features	Used	Not Used / Underused	Usage Rate
CLI Flags	--disable-builtin-mcps, --no-ask-user, --agent, --autopilot, --add-dir, --allow-all-paths, --no-custom-instructions	--disable-builtin-mcps/--no-ask-user (auto-applied), --agent (84), --allow-all-paths (auto), --no-custom-instructions via bare (14)	--autopilot/continuations (11/98 only)	Autopilot: ~11%
Engine Config	agent, bare, copilot-sdk, driver, args, env, max-tool-denials, harness, model, version	agent (84), copilot-sdk (68), bare (14), args (5)	driver (custom SDK driver) rarely used; explicit version pinning rare	Custom driver: near 0%
MCP Servers	GitHub MCP, custom JSON MCP config	GitHub MCP (72)	Custom third-party MCP servers rare	Low custom MCP adoption
Network Config	network: allowlist, firewall:	network: (42)	firewall: explicit (1)	~43% explicit network config
Custom Agents	.github/agents/*.agent.md (9 total)	4 referenced (adr-writer, contribution-checker, technical-doc-writer, ci-cleaner)	5 orphaned (see below)	44% dead

---

3️⃣ Missed Opportunities

View High Priority Opportunities

🔴 High Priority

Opportunity 1: Orphaned custom agent files

• What: 5 files in .github/agents/ (create-safe-output-type.agent.md, custom-engine-implementation.agent.md, grumpy-reviewer.agent.md, interactive-agent-designer.agent.md, w3c-specification-writer.agent.md) are not referenced by any workflow's engine.agent field.
• Why It Matters: Dead configuration accumulates maintenance risk (docs drift, unclear intent) and confuses contributors about which agents are "live."
• Where: .github/agents/*.agent.md
• How to Implement: Either (a) wire each into a workflow via engine: {id: copilot, agent: <name>}, or (b) remove the file if no longer needed, or (c) document them explicitly as "templates/examples" if intentionally unused.
• Example:

  engine:
    id: copilot
    agent: w3c-specification-writer

Opportunity 2: Low explicit network/firewall configuration

• What: Only 42/98 copilot-block workflows set network:, and just 1 sets firewall: explicitly.
• Why It Matters: Workflows running with default network access have a larger attack surface for prompt-injection-driven exfiltration attempts.
• Where: Audit the 56 copilot workflows without network: for whether they touch untrusted content (issues/PRs/external data) and need an allowlist.
• How to Implement: Add network: {allowed: [...]} scoped to only the domains each workflow needs (e.g., api.github.com, package registries used).

View Medium Priority Opportunities

🟡 Medium Priority

Opportunity 3: Underused autopilot/continuations

• What: max-continuations (which triggers --autopilot --max-autopilot-continues) is set in only 11/98 workflows.
• Why It Matters: Multi-step workflows (e.g., large refactors, iterative issue triage) may silently stop after one pass without continuation support, requiring manual re-triggering.
• Where: Workflows doing large-scope analysis (e.g., jsweep.md, eslint-miner.md) could benefit if they aren't already using it.
• How to Implement:

  engine:
    id: copilot
    max-continuations: 3

Opportunity 4: --share flag unexplored

• What: Not referenced in the Go engine code (copilot_engine_execution.go), suggesting gh-aw doesn't currently map any config to it, and only this research workflow's prompt text mentions it.
• Why It Matters: If Copilot CLI does support --share for conversation URLs, it could aid debugging failed runs; if it doesn't exist in the installed CLI version, prompt/doc text referencing it should be corrected to avoid confusion.
• How to Implement: Verify via copilot --help in the CLI version gh-aw installs; if valid, consider exposing via engine.args or a dedicated config key; if invalid, remove references from this workflow's prompt.

Opportunity 5: Custom SDK driver (engine.driver) underused

• What: gh-aw supports fully custom Copilot SDK drivers (JS/Python/Go/Ruby, inline or file-based) but adoption appears minimal outside smoke-test workflows.
• Why It Matters: Advanced workflows needing bespoke tool-approval logic or telemetry hooks aren't leveraging this extensibility point.
• Where: Consider for workflows needing custom sandboxing or non-standard tool-call interception.

View Low Priority Opportunities

🟢 Low Priority

Opportunity 6: Explicit version pinning

• What: Very few workflows pin an explicit engine.version; most rely on "latest," which the code notes matters for flag compatibility (e.g., --no-ask-user requires v1.0.19+).
• Why It Matters: Silent CLI upgrades could introduce behavior changes; pinning aids reproducibility for critical workflows.
• Where: High-stakes workflows (release automation, security checks) are best candidates.

Opportunity 7: args: passthrough usage is minimal and inconsistent

• What: Only 5 workflows use engine.args for raw CLI passthrough, each seemingly for different one-off needs.
• Why It Matters: If common patterns emerge (e.g., always adding a specific --add-dir), consider promoting to first-class config instead of ad hoc args.

Opportunity 8: Timeout tuning

• What: Timeout values cluster around 10–30 minutes with some outliers up to 120 minutes; no clear correlation to workflow complexity was found in this pass.
• Why It Matters: Right-sizing timeouts reduces wasted CI minutes on workflows that fail fast, and avoids premature cutoffs on genuinely long-running ones.

Opportunity 9: bare: true review

• What: 14 workflows suppress custom instructions via bare: true; worth spot-checking whether all of these are intentional (e.g., isolated test/smoke workflows) vs. accidental copy-paste from a template.

---

4️⃣ Specific Workflow Recommendations

View Workflow-Specific Recommendations

Orphaned agents: create-safe-output-type, custom-engine-implementation, grumpy-reviewer, interactive-agent-designer, w3c-specification-writer

• Current State: Defined in .github/agents/*.agent.md, zero workflow references across two consecutive research runs.
• Recommended Changes: Decide fate — wire up, archive, or document as templates.
• Expected Benefits: Reduced repo clutter and clearer contributor guidance on active vs. reference agents.

jsweep.md, eslint-miner.md (use args: for custom CLI flags)

• Current State: Use raw engine.args passthrough for one-off CLI needs.
• Recommended Changes: Confirm these couldn't be satisfied by first-class config (bare, agent, max-continuations) to reduce brittleness against future CLI flag changes.

---

5️⃣ Trends & Insights

View Historical Trends

Comparing to previous analysis (run 30732964595, 2026-08-02):

• Orphaned agent list is unchanged — same 5 files, still unreferenced after a full cycle. This is now a recurring, unaddressed finding.
• copilot-sdk: true continues strong adoption (68 this run vs. 99 block-form workflows tracked differently in prior counts — direct comparison method differed slightly between runs; recommend standardizing the counting query for future runs).
• --share usage remains at 1 (the research workflow itself) — no organic adoption signal.
• Note: exact workflow/engine counts differ slightly from the prior run due to methodology differences (prior run counted id: copilot across all workflows including detection-only stanzas differently). Future runs should use a single canonical query, stored in this memory file, for apples-to-apples trend tracking.

---

6️⃣ Best Practice Guidelines

1. Retire or wire up orphaned agent files promptly: An agent file with zero workflow references for more than one research cycle should be actioned (used or removed) rather than left indefinitely.
2. Default to explicit network: allowlists for workflows processing untrusted content: Especially those triggered by issues/PR comments from external actors.
3. Reserve bare: true for workflows that genuinely don't need repo custom instructions (e.g., pure smoke tests), and document why in a comment when used elsewhere.

---

---

7️⃣ Action Items

Immediate Actions (this week):

• Decide disposition (use/archive/remove) for the 5 orphaned agent files.
• Verify whether --share is a real Copilot CLI flag; correct or remove misleading references in this workflow's own prompt if not.

Short-term (this month):

• Audit the ~56 copilot workflows lacking network: config for exposure to untrusted input; add allowlists where warranted.
• Evaluate max-continuations for multi-step analysis workflows (e.g., jsweep.md, eslint-miner.md).

Long-term (this quarter):

• Standardize a single canonical counting methodology for future Copilot CLI research runs (store the exact grep/query set in repo-memory notes) to enable precise trend comparisons.
• Consider promoting common engine.args passthrough patterns to first-class engine config options if repeated needs emerge.

---

View Supporting Evidence & Methodology

📚 References

• Copilot Engine implementation: pkg/workflow/copilot_engine.go, pkg/workflow/copilot_engine_execution.go, pkg/workflow/copilot_mcp.go
• Engine documentation: docs/src/content/docs/reference/engines.md
• Custom agent files: .github/agents/*.agent.md
• Previous research: repo-memory branch memory/copilot-cli-research (copilot-cli-research/latest.json, run 30732964595)

Research Methodology

Static analysis via grep/find over .github/workflows/*.md frontmatter and pkg/workflow/copilot_*.go source files. Counted CLI flags emitted unconditionally vs. conditionally in buildCopilotArgs; cross-referenced engine.agent values against .github/agents/*.agent.md filenames; compared network/firewall/safe-outputs frontmatter presence across all id: copilot workflows. No workflow runs were triggered as part of this analysis.

---

Generated by Copilot CLI Deep Research (Run: 30785495408)

Generated by 🔬 Copilot CLI Deep Research Agent · auto · 35 AIC · ⌖ 6.19 AIC · ⊞ 10.6K · ◷

• expires on Aug 3, 2026, 8:58 PM UTC-08:00

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #50172

View newer issue

---

This action was performed automatically by the Copilot CLI Deep Research Agent workflow.