-
Notifications
You must be signed in to change notification settings - Fork 324
[cli-consistency] CLI Consistency Issues - 2026-03-31 #23733
Description
Summary
Automated CLI consistency inspection found 13 inconsistencies in command help text and documentation that should be addressed for better user experience and documentation clarity.
Breakdown by Severity
- High: 0
- Medium: 5 (misleading/inconsistent behavior)
- Low: 8 (minor polish/documentation gaps)
Issue Categories
- Redundant JSON flags (2 commands) —
audit diff,audit report - Inconsistent flag units (2 commands) —
trial,logs - Misleading flag naming/description (2 commands) —
upgrade,add - Unexplained jargon (1 command) —
logs - Documentation/CLI discrepancies (3 areas) —
add,remove,initdocs - Minor polish (5 items) —
init,pr,new,audit,logs
Inspection Details
- Total Commands Inspected: 36 (all top-level commands + subcommands with
--help) - Commands with Issues: 12
- Date: 2026-03-31
- Method: Executed all CLI commands with
--helpflags, analyzed actual output, and cross-referenced againstdocs/src/content/docs/setup/cli.md
Findings Summary
✅ No issues found in these areas:
- Flag naming consistency across most commands (all use standard
-h,-v,-r,-e,-j,-o,-d) - Example completeness (all commands have 3+ examples)
- Command descriptions match their actual listed behavior
github.github.comdocumentation URLs are intentional (confirmed inrelease.mdnetwork allowlist)- Global flags (
--banner,-v/--verbose) are consistent across all commands - Short flag conventions are consistent (no conflicts found)
- Redundant JSON flags in
audit diffandaudit report - Inconsistent timeout units between
trial(minutes) andlogs(seconds) - Misleading
upgrade --no-fixflag name - DIFC acronym unexplained in
logshelp - Remote wildcard example in docs not reflected in CLI help
- Several minor documentation wording issues
Detailed Findings
1. Redundant --format json and -j/--json flags
Commands Affected: audit diff, audit report
Priority: Medium
Type: Redundant flags / Inconsistency
Current Output (from ./gh-aw audit diff --help):
Flags:
--format string Output format: pretty, markdown, json (default "pretty")
-h, --help Show help for gh aw audit diff
-j, --json Output results in JSON format
```
**Current Output** (from `./gh-aw audit report --help`):
```
Flags:
--format string Output format: markdown, pretty, json (default "markdown")
-h, --help Show help for gh aw audit report
-j, --json Output results in JSON format
```
**Issue**: Both `audit diff` and `audit report` expose two overlapping ways to request JSON output: `--format json` and `-j/--json`. Users can't tell if they behave identically or if there's a subtle difference. The sibling commands `audit <run-id>` and `logs` only have `-j/--json` (no `--format` flag).
Additionally, the default format differs between these two sibling commands (`pretty` vs `markdown`), which can surprise users.
**Suggested Fix**: Remove the `-j/--json` shorthand from `audit diff` and `audit report` (keeping `--format`), or remove `--format` and keep only `-j/--json` (consistent with other commands). Document why the defaults differ or unify them.
---
#### 2. Inconsistent timeout units: `trial` (minutes) vs `logs` (seconds)
**Commands Affected**: `gh aw trial`, `gh aw logs`
**Priority**: Medium
**Type**: Inconsistency
**Current Output** (from `./gh-aw trial --help`):
```
--timeout int Execution timeout in minutes (default 30)
```
**Current Output** (from `./gh-aw logs --help`):
```
--timeout int Download timeout in seconds (0 = no timeout)
```
**Issue**: The `--timeout` flag uses **minutes** in `trial` but **seconds** in `logs`. Users who switch between commands could easily pass the wrong magnitude — e.g., passing `300` intending 5 minutes would set a 300-minute timeout in `trial` or a 5-minute timeout in `logs`.
**Suggested Fix**: Standardize on one unit (seconds is more standard for CLI tools) and update the flag description and implementation accordingly. At minimum, make the unit explicit in each description (currently `logs` says "in seconds" but `trial` doesn't say "in minutes" as prominently).
---
#### 3. Misleading `upgrade --no-fix` flag name
**Commands Affected**: `gh aw upgrade`
**Priority**: Medium
**Type**: Misleading flag name
**Current Output** (from `./gh-aw upgrade --help`):
```
--no-fix Skip codemods, action updates, and workflow compilation (only update agent files)
```
**Other related flags:**
```
--no-actions Skip updating GitHub Actions versions
--no-compile Skip recompiling workflows (do not modify lock files)
```
**Issue**: The flag `--no-fix` has a name that implies "skip only the `fix` codemods step", but its actual behavior is to skip **three steps**: codemods, action updates, and workflow compilation. Those last two steps have their own dedicated flags (`--no-actions`, `--no-compile`). The naming creates a false mental model where users might use `--no-fix` thinking it only skips codemods, unknowingly also skipping action updates and compilation.
**Suggested Fix**: Either rename `--no-fix` to `--agent-only` (reflecting that it only updates agent files) or `--no-fix-no-actions-no-compile`, or update the description to clarify: "Skip codemods, action updates, and compilation (implies --no-actions and --no-compile; only updates agent files)".
---
#### 4. Unexplained "DIFC" acronym in `logs` help
**Commands Affected**: `gh aw logs`
**Priority**: Medium
**Type**: Unexplained jargon
**Current Output** (from `./gh-aw logs --help`):
```
# Content filtering
gh aw logs --filtered-integrity # Filter logs with DIFC integrity-filtered items in gateway logs
...
--filtered-integrity Filter to runs with DIFC integrity-filtered items in the gateway logs
Issue: "DIFC" is not defined anywhere in the help text. Most users won't know what DIFC (Data Information Flow Control) means. The flag name --filtered-integrity doesn't help either, as it doesn't include the acronym.
Suggested Fix: Expand the acronym in the flag description: "Filter to runs with DIFC (Data Information Flow Control) integrity-filtered items in the gateway logs", or use plain language: "Filter to runs where tool calls were blocked by integrity checks".
5. Remote wildcard example in docs not documented in CLI help
Commands Affected: gh aw add (docs vs CLI)
Priority: Medium
Type: Documentation/CLI discrepancy
Current docs (docs/src/content/docs/setup/cli.md, line 163):
gh aw add "githubnext/agentics/ci-*" # Add multiple with wildcards
```
**Current CLI help** (from `./gh-aw add --help`):
```
Workflow specifications:
- Three parts: "owner/repo/workflow-name[`@version`]" (implicitly looks in workflows/ directory)
- Four+ parts: "owner/repo/workflows/workflow-name.md[`@version`]" (requires explicit .md extension)
- GitHub URL: "https://github.com/owner/repo/blob/branch/path/to/workflow.md"
- Local file: "./path/to/workflow.md" (adds a workflow from local filesystem)
- Local wildcard: "./*.md" or "./dir/*.md" (adds all .md files matching pattern)
- Version can be tag, branch, or SHA (for remote workflows)
```
**Issue**: The documentation shows remote wildcard syntax (`"githubnext/agentics/ci-*"`), but the CLI help only documents "Local wildcard" support. Users who read the docs and try remote wildcards may get unexpected results, or the CLI help is missing this documented capability.
**Suggested Fix**: If remote wildcards are supported, add a "Remote wildcard" entry to the Workflow specifications in `add --help`. If not supported, remove the example from the docs.
---
#### 6. Docs wording error: "logs `.gitignore`" in `init` description
**Commands Affected**: `docs/src/content/docs/setup/cli.md` line 131
**Priority**: Low
**Type**: Typo/wording error
**Current docs:**
```
Initialize repository for agentic workflows. Configures `.gitattributes`, creates the dispatcher agent file
(`.github/agents/agentic-workflows.agent.md`), and logs `.gitignore`.
```
**Issue**: "logs `.gitignore`" is not meaningful. The `init --help` output doesn't mention `.gitignore` at all. This reads like it should say "updates `.gitignore`" or the `.gitignore` reference should be removed.
**Suggested Fix**: Remove "and logs `.gitignore`" or replace with accurate wording based on what `init` actually does to `.gitignore` (if anything).
---
#### 7. `init --codespaces` shows confusing default value in help
**Commands Affected**: `gh aw init`
**Priority**: Low
**Type**: Confusing flag display
**Current Output** (from `./gh-aw init --help`):
```
--codespaces string[=" "] Create devcontainer.json for GitHub Codespaces...
```
**Issue**: The `string[=" "]` display shows a space character as the optional-flag default. This is Cobra's rendering of a flag that accepts an optional string value, but `[=" "]` (equals a space) is confusing and looks like a display artifact to most users.
**Suggested Fix**: Consider using a more meaningful default like `[="current"]` or restructuring the flag description to avoid the confusing display.
---
#### 8. Inconsistent "PR" vs "pull request" terminology
**Commands Affected**: `gh aw pr`, `gh aw pr transfer`
**Priority**: Low
**Type**: Terminology inconsistency
**Current Output** (from `./gh-aw pr --help`):
```
Available subcommands:
• transfer - Transfer a PR from one repository to another
```
**Current Output** (from `./gh-aw pr transfer --help`):
```
Transfer a pull request from one repository to another.
```
**Issue**: The parent `pr` command uses the abbreviation "PR" in its subcommand listing, while the `pr transfer` subcommand uses the full phrase "pull request". Minor but inconsistent.
**Suggested Fix**: Use "pull request" consistently in all help text (the full form is already used in the subcommand and is more accessible to new users).
---
#### 9. `new --interactive / -i` exists but `init` lacks `-i` short flag
**Commands Affected**: `gh aw new`, `gh aw init`
**Priority**: Low
**Type**: Flag inconsistency
**Current Output** (from `./gh-aw new --help`):
```
-i, --interactive Launch interactive workflow creation wizard
```
**Current Output** (`init` has no `-i` flag):
```
(no -i flag; interactive is the default mode)
```
**Issue**: `new` has a `-i/--interactive` flag for its interactive mode. `init` also has an interactive mode (it's the default), but no `-i` flag. While `init`'s interactive mode is the default (so `-i` isn't strictly needed), the asymmetry may confuse users who expect `-i` to work on both commands.
**Suggested Fix**: Add `-i/--interactive` to `init` as an alias for the default interactive mode (no-op when interactive is already default, but provides consistent discoverability).
---
#### 10. `add --name/-n` flag description omits "first workflow only" limitation
**Commands Affected**: `gh aw add`
**Priority**: Low
**Type**: Incomplete flag description
**Current Output** (from `./gh-aw add --help`):
Body text (mentions limitation):
```
The -n flag allows you to specify a custom name for the workflow file (only applies to the first workflow when adding multiple).
```
Flags table (omits limitation):
```
-n, --name string Specify name for the added workflow (without .md extension)
```
**Issue**: The body text describes the `-n` limitation (only first workflow) but the flags table description doesn't. Users relying on `--help` flags listing alone won't see the "first workflow only" caveat.
**Suggested Fix**: Update the flag description: `Specify name for first workflow when adding multiple (without .md extension)`.
---
#### 11. `remove` docs lack `--keep-orphans` documentation
**Commands Affected**: `gh aw remove` (docs)
**Priority**: Low
**Type**: Documentation gap
**Current docs** (`docs/src/content/docs/setup/cli.md`, line 514-520):
```
#### `remove`
Remove workflows (both `.md` and `.lock.yml`).
\`\`\`bash wrap
gh aw remove my-workflow
\`\`\`
```
**CLI help** (from `./gh-aw remove --help`) also documents:
```
--keep-orphans Skip removal of orphaned include files that are no longer referenced by any workflow
```
And examples showing pattern removal: `gh aw remove test- # Remove all workflows starting with 'test-'`
**Issue**: The docs for `remove` are very sparse — they show only a single example and don't mention: (1) pattern/prefix matching, (2) orphaned include file cleanup behavior, or (3) the `--keep-orphans` flag.
**Suggested Fix**: Expand the `remove` docs section to match the CLI help, adding pattern examples and documenting `--keep-orphans`.
---
#### 12. `logs --summary-file` has awkward dual-parenthetical description
**Commands Affected**: `gh aw logs`
**Priority**: Low
**Type**: Minor wording
**Current Output** (from `./gh-aw logs --help`):
```
--summary-file string Path to write the summary JSON file relative to output directory (use empty string to disable) (default "summary.json")
```
**Issue**: The flag description has two back-to-back parentheticals: `(use empty string to disable) (default "summary.json")`. This is readable but slightly awkward.
**Suggested Fix**: Combine: `Path to write the summary JSON file relative to output directory (default "summary.json"; use empty string "" to disable)`.
---
#### 13. `audit diff` and `audit report` have different default `--format` values
**Commands Affected**: `gh aw audit diff`, `gh aw audit report`
**Priority**: Low
**Type**: Inconsistency between sibling commands
**Current Output** (from `./gh-aw audit diff --help`):
```
--format string Output format: pretty, markdown, json (default "pretty")
```
**Current Output** (from `./gh-aw audit report --help`):
```
--format string Output format: markdown, pretty, json (default "markdown")Issue: audit diff defaults to pretty while audit report defaults to markdown. The different defaults may be intentional (diff is for terminal inspection, report is for piping to files), but this isn't explained to users. Additionally, the options are listed in different orders (pretty, markdown, json vs markdown, pretty, json), which might suggest different precedence/preference.
Suggested Fix: If the different defaults are intentional, add a comment in each explaining why: e.g., (default "pretty" — suitable for terminal inspection) vs (default "markdown" — suitable for piping to files or $GITHUB_STEP_SUMMARY). Normalize the option ordering to match.
Generated by CLI Consistency Checker · ◷
- expires on Apr 2, 2026, 1:44 PM UTC