🔍 Claude Code User Documentation Review - 2026-03-25 #22905
Closed
Replies: 1 comment
-
|
This discussion has been marked as outdated by Claude Code User Documentation Review. A newer discussion is available at Discussion #23121. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Executive Summary
As a developer who uses Claude Code and actively avoids GitHub Copilot, I reviewed the core gh-aw documentation to assess whether it is accessible and actionable for non-Copilot users. The overall picture is positive: Claude is treated as a first-class engine, authentication is clearly documented, and a meaningful set of workflow examples (34 of 178) use Claude. However, several issues create Copilot-centric impressions — most notably in the architecture diagrams, the
how-they-work.mdxintroduction, and the README — that could cause early-stage confusion.Key Finding: Claude Code users can successfully adopt gh-aw, but will encounter minor friction from Copilot-centric wording and incomplete engine parity documentation in a few key places.
Persona Context
I reviewed this documentation as a developer who:
Question 1: Onboarding Experience
Can a Claude Code user understand and get started with gh-aw?
Short answer: Yes, with minor friction.
The Quick Start guide (
quick-start.mdx) is the strongest point: prerequisites immediately listAnthropic Claudealongside GitHub Copilot and OpenAI Codex. Step 2 of the wizard explicitly says "Select an AI Engine — Choose between Copilot, Claude, or Codex," and the corresponding secret names (ANTHROPIC_API_KEY,OPENAI_API_KEY,COPILOT_GITHUB_TOKEN) are all called out.However, the README and
how-they-work.mdx— the first two documents many users encounter — do not convey this multi-engine support clearly.Specific Issues Found:
README.md, entire Overview section): Zero mention of Claude, Codex, or any engine alternatives. A developer landing here first will have no idea that non-Copilot options exist before clicking into deeper docs.how-they-work.mdx, line 26: "Workflows support GitHub Copilot (default), Claude by Anthropic, and Codex." Gemini is now documented inengines.mdas a supported engine but is completely absent from this introduction page.how-they-work.mdx, line 27: Copilot is labeled the default without explaining what that means practically (i.e., thatengine:can simply be set toclaudeto switch).Recommended Fixes:
how-they-work.mdxto mention Gemini alongside the other three engines.how-they-work.mdxclarifying that non-Copilot engines are equally supported and just need a different secret.Question 2: Inaccessible Features for Non-Copilot Users
What features or steps don't work without Copilot?
Features That Require Copilot (or are Copilot-only):
engine.agent(custom agent files,.github/agents/)engines.md, feature tablemax-continuations(autopilot multi-run mode)engines.md, feature tableCOPILOT_GITHUB_TOKENPAT requirementauth.mdxFeatures That Work Without Copilot (engine-agnostic):
edit:,bash:,github:,web-fetch:,playwright:,cache-memory:,repo-memory:,qmd:,agentic-workflows:gh aw init,gh aw compile,gh aw run,gh aw trial,gh aw logs,gh aw auditmax-turnsis actually Claude-only (a Claude advantage over other engines)api-targetandANTHROPIC_BASE_URLwork for ClaudeMissing Documentation:
engine.agentfor Claude users who want custom behavior profiles (the answer appears to be: write detailed system context in the workflow markdown itself, but this isn't stated).Question 3: Documentation Gaps and Assumptions
Where does the documentation assume Copilot usage?
Copilot-Centric Language Found In:
docs/src/content/docs/introduction/architecture.mdx, line 181 — The AWF diagram renders a box labeledCOPILOT["Copilot CLI"]inside the "AI Agent Process" subgraph, hardcoding Copilot as the canonical agent in a security architecture diagram that applies to all engines.docs/src/content/docs/introduction/architecture.mdx, line 252 — The MCP Gateway diagram showsAGENT["Agent container\nCopilot CLI + MCP client\n172.30.0.20"], reinforcing a Copilot-specific mental model of the container architecture.docs/src/content/docs/reference/auth.mdx, lines 47-49 — The CLI example under "Adding secrets using the CLI" shows onlyCOPILOT_GITHUB_TOKENas the example, before the reader reaches the individual engine sections.Missing Alternative Instructions:
Copilot Custom Configurationsection inengines.md).engine.agentfor Claude workflows.Severity-Categorized Findings
🚫 Critical Blockers (Score: 0/10 — None Found)
No critical blockers were identified. Claude users can complete the full quick-start guide without a Copilot subscription.
Obstacle 1: Architecture Diagrams Are Copilot-Specific
Impact: A Claude Code user reading the security architecture page will see "Copilot CLI" hardcoded in all network/container diagrams, undermining confidence that gh-aw works properly with Claude.
Current State:
architecture.mdxlines 181 and 252 render "Copilot CLI" as a named node inside the agent container and firewall diagrams. The configuration example beneath the firewall diagram also usesengine: copilotwithout a note that other engines apply the same pattern.Why It's Problematic: Security architecture is important for enterprise evaluation. A diagram labeling the agent as "Copilot CLI" when you're using Claude creates doubt about whether the documented safeguards actually apply.
Suggested Fix: Replace
COPILOT["Copilot CLI"]withENGINE["AI Engine\n(Copilot / Claude / Codex / Gemini)"]and update the MCP Gateway diagram to use genericAGENT["Agent container\nAI Engine + MCP client"]language. Add a note under the configuration example: "Replaceengine: copilotwithengine: claude,codex, orgeminidepending on your chosen AI engine."Affected Files:
docs/src/content/docs/introduction/architecture.mdxObstacle 2: README Has Zero Non-Copilot Visibility
Impact: The README is the entry point for most developers discovering gh-aw. It mentions Copilot implicitly (via the "Guardrails" section referencing Copilot-specific links in the community section) but never mentions Claude, Codex, or multi-engine support anywhere in the prose sections.
Current State: The README sections Overview, Guardrails, and Documentation contain no mention of supported AI engines. A developer who does not use Copilot reading only the README would have no indication this tool is relevant to them.
Suggested Fix: Add to the Overview section: "gh-aw supports GitHub Copilot, Claude (Anthropic), Codex (OpenAI), and Gemini (Google) as AI engines — no Copilot subscription required to get started."
Affected Files:
README.md💡 Minor Confusion Points (Score: 5/10)
how-they-work.mdxlists "GitHub Copilot, Claude by Anthropic, and Codex" butengines.mdnow documents Gemini as a 4th engine. File:docs/src/content/docs/introduction/how-they-work.mdxengines.mdandhow-they-work.mdxsay Copilot is the "default" without clarifying that the default only applies whenengine:is omitted — which implies Copilot is preferred or better. File:docs/src/content/docs/reference/engines.md,how-they-work.mdxauth.mdxCLI example snippet under "Adding secrets using the CLI" showsCOPILOT_GITHUB_TOKENas the only example before explaining other options exist. File:docs/src/content/docs/reference/auth.mdx, line 48engine.agent?" explanation for Claude: Copilot has a dedicatedCopilot Custom Configurationsection and a link to "Copilot Agent Files". Claude users wanting equivalent custom-prompt behavior have no comparable guide. File:docs/src/content/docs/reference/engines.mdmax-continuationslisted only as Copilot-only without explaining Claude's alternative approach to extended agentic runs (max-turns). The docs note them in the table but don't explain the conceptual difference. File:docs/src/content/docs/reference/engines.mdEngine Comparison Analysis
Available Engines
Based on my review, gh-aw supports these engines:
Rating Scale: ⭐⭐⭐⭐⭐ (Excellent) to ⭐ (Poor/Missing)
Notes on Claude's score: Claude setup docs in
auth.mdxare clear and complete. The-1star from perfect comes from the missing "custom agent equivalent" guide and the architecture diagram issue. Claude is a genuinely well-supported second option.Tool Availability Analysis
Engine-Agnostic Tools (work with any engine):
edit:— file editinggithub:— GitHub API toolsetsbash:— shell commandsweb-fetch:— web content fetchingplaywright:— browser automationcache-memory:— persistent cross-run memoryrepo-memory:— repository-scoped memoryqmd:— documentation vector searchagentic-workflows:— workflow introspectionmcp-servers:custom integrationsEngine-Specific Tools/Features:
engine.agent— Copilot only (custom agent files)max-continuations— Copilot only (autopilot multi-run)max-turns— Claude only (iteration limit per run)web-search:— Codex has native support; others require MCPUnclear/Undocumented:
edit:tool configurationAuthentication Requirements
Current Documentation
Quick Start guide covers authentication for:
COPILOT_GITHUB_TOKEN) — detailed instructions with deep-link to PAT creationANTHROPIC_API_KEY) — clear instructions with link to platform.claude.comOPENAI_API_KEY) — clear instructions with link to platform.openai.comGEMINI_API_KEY) — clear instructions with link to aistudio.google.comMissing for Claude Users
platform.claude.com/docs/en/get-startedin auth docs currently has a generic URL that could be more specific (e.g., directly to API key creation)Secret Names
COPILOT_GITHUB_TOKENANTHROPIC_API_KEYOPENAI_API_KEYGEMINI_API_KEYAll four are well-documented in
auth.mdx.Example Workflow Analysis
Workflow Count by Engine
Quality of Examples
Copilot Examples:
Extensive and varied. 85 explicit Copilot workflows cover a wide range of use cases from PR triage to security scanning to automated daily reports.
Claude Examples:
34 workflows is a solid set. However, they are not surfaced prominently in documentation — a user reading the Quick Start would add the
daily-repo-statusworkflow using the engine wizard, and wouldn't know to look at these examples. Adding a "Claude Engine Cookbook" or similar page linking to key Claude workflow examples would significantly improve discoverability.Codex Examples:
17 workflows; reasonable representation. Similar discoverability issue.
Gemini Examples:
Zero workflow examples despite Gemini being documented as a supported engine in
engines.md. This is a gap — at minimum, one example Gemini workflow would validate that the engine integration actually works.Recommended Actions
Priority 1: Critical Documentation Fixes
docs/src/content/docs/introduction/architecture.mdxREADME.mdPriority 2: Major Improvements
how-they-work.mdxto include Gemini — Add Gemini to the engine list in the "AI Engines" section and verify the list stays in sync withengines.md. File:docs/src/content/docs/introduction/how-they-work.mdxmax-turns, model selection viaengine.model) in the same way Copilot users configureengine.agent. File:docs/src/content/docs/reference/engines.md.github/workflows/orengines.mdPriority 3: Nice-to-Have Enhancements
engine: claude(orcodex,gemini) is all that's needed to switch engines."Positive Findings
What Works Well
engines.mdis excellent — showsmax-turnsas Claude-only,max-continuationsas Copilot-only, making tradeoffs clear--engine claudeacross all relevant commands (gh aw new,gh aw run,gh aw validate,gh aw add,gh aw secrets bootstrap)gh aw new --engine claudegenerates correct Claude frontmatter automaticallyadd-wizardinteractive setup handles engine selection and secret configuration for all enginesapi-targetandANTHROPIC_BASE_URLfor Claude proxy support is well-documentedConclusion
Can Claude Code Users Successfully Adopt gh-aw?
Answer: Yes, with modest effort
The core documentation — quick start, authentication, engines reference — provides everything a Claude Code user needs to get started. Authentication is clear, the wizard handles engine selection, and there are 34 production workflow examples using Claude. A motivated developer who follows the Quick Start guide can be running their first Claude-powered agentic workflow in under 10 minutes.
The friction points are real but shallow: the README and
how-they-work.mdxintroduction create an initial "is this just for Copilot users?" impression that the Quick Start then dispels. The architecture diagrams, while technically accurate, paint a Copilot-specific picture of the runtime that could cause doubt. And the absence of any Gemini examples (despite engine documentation existing) is a credibility gap.The single biggest ROI fix is updating the two architecture diagrams from "Copilot CLI" to "AI Engine" — this is a 5-minute change that eliminates the most visually prominent Copilot-only signal in the docs.
Overall Assessment Score: 7.5/10
Breakdown:
max-continuationsandengine.agentare real Copilot advantages)Appendix: Files Reviewed
Complete List of Documentation Files Analyzed
README.mddocs/src/content/docs/setup/quick-start.mdxdocs/src/content/docs/introduction/how-they-work.mdxdocs/src/content/docs/introduction/architecture.mdxdocs/src/content/docs/reference/tools.mddocs/src/content/docs/setup/cli.mddocs/src/content/docs/reference/engines.mddocs/src/content/docs/reference/auth.mdx.github/workflows/*.md(178 workflow files analyzed for engine distribution)References:
Beta Was this translation helpful? Give feedback.
All reactions