Agent Persona Exploration - 2026-03-29

[github-actions[bot]]

Persona Overview

• Agent: developer.instructions (agentic-workflows)
• Scenarios Tested: 6 (across 5 personas)
• Average Quality Score: 4.83/5.0
• Run: §23700701560

Key Findings

• Consistently high quality: All 6 scenarios scored ≥ 4.6/5.0 — the agent produces production-ready workflow configurations across diverse automation types
• Trigger selection is a strength: Every scenario received 5/5 for trigger appropriateness, including the nuanced choice of workflow_run over pull_request for post-CI artifact analysis
• Security practices are deeply embedded: strict mode, network firewall (AWF), minimal permissions, scoped bash tool lists, and the noop guard appear in every response
• Prompt structure is consistent: All responses follow a Context → Numbered Steps → Output template → Error handling → Noop guard pattern
• Minor gaps in tool scoping: Two scenarios had slightly broad bash glob patterns or top-level permission grants that could be tightened

Top Patterns

1. Trigger by workflow type: PR automation → pull_request + paths: + concurrency cancel-in-progress; Post-CI artifact work → workflow_run; Scheduled reports → schedule cron + workflow_dispatch for testing
2. Tools consistently scoped: GitHub toolsets named explicitly ([pull_requests, repos], [actions, issues], [default, discussions]); bash limited to named command families (go:*, git:*, curl, jq) never "*"
3. Safe-output hygiene: max: 1 + hide-older-comments: true / close-older-discussions: true applied universally to prevent accumulation; noop guard called out as required in every response

View High-Quality Responses (Top 3)

be-schema — DB Migration Reviewer (5.0/5.0)
Perfect across all dimensions. Standouts: (1) concurrency group keyed to PR number cancels stale runs on rapid pushes; (2) 7-category safety check taxonomy (transactions, destructive ops, rollback, FK indexes, NOT NULL without default, lock-hazardous ops, irreversible transforms); (3) add-labels: allowed: [needs-migration-review] as a strict guardrail — agent cannot apply arbitrary labels.

do-incident — Deployment Incident Reporter (5.0/5.0)
Correctly handles the hard cross-repo write case: scoped PAT (OPS_ISSUES_WRITE_TOKEN) with fine-grained issues:write only on the ops repo; close-older-key tied to run ID prevents duplicate incidents on retries; flake vs regression detection via querying recent run history; label creation shell scripts provided in setup checklist.

pm-digest — Weekly Product Digest (5.0/5.0)
Clean scheduled workflow design: close-older-discussions: true keeps the "Product Updates" category tidy automatically; three-phase gather/classify/draft structure prevents the common partial-data drafting failure; classification priority ordering (breaking → high → medium → low → internal) is explicit and deterministic.

View Areas for Improvement

fe-bundle — Bundle Size Analyzer (4.6/5.0)
bash: ["gh api:*", "gh run:*", ...] is a somewhat broad glob. More precise alternatives: bash: ["gh:*"] or individual commands like bash: ["unzip", "jq", "node", "mkdir", "cat"] plus a scoped github toolset for API access. The two-workflow architecture justification is excellent, but the bash allowlist reduces security score.

qa-coverage — PR Coverage Guard (4.6/5.0)
permissions: pull-requests: write appears in the top-level job permissions rather than being exclusively managed by the safe-outputs subsystem. In this case it may be technically required (git operations during worktree setup could need it), but the response doesn't clarify why — leaving room for ambiguity about whether safe-outputs alone would suffice.

Recommendations

1. Document workflow_run as the canonical pattern for post-CI analysis in .github/aw/create-agentic-workflow.md. The agent correctly chose it over pull_request for the bundle-size scenario (artifact availability, fork safety), but this decision tree isn't visible to users without asking. A short "When to use workflow_run vs pull_request" section would surface this proactively.

2. Add a cross-repo write pattern guide to .github/aw/github-agentic-workflows.md. The incident reporter correctly used a fine-grained PAT + target-repo in create-issue, but this is a non-obvious pattern. A canonical example of scoped-PAT cross-repo issue/PR creation would reduce friction for DevOps scenarios.

3. Tighten bash tool scope guidance: The agent sometimes uses command-family globs (gh api:*) when a more restrictive allowlist (individual commands + scoped GitHub toolset) would be safer. A note in the documentation recommending specific commands over family globs would improve the security baseline of generated workflows.

---

View Full Scenario Scores

Scenario	Persona	Trigger	Tools	Security	Prompt	Completeness	Avg
DB Migration Reviewer	Backend Eng	5	5	5	5	5	5.0
Deployment Incident Reporter	DevOps Eng	5	5	5	5	5	5.0
Weekly Product Digest	Product Mgr	5	5	5	5	5	5.0
Weekly Cost Digest	DevOps Eng	5	4	5	5	5	4.8
PR Coverage Guard	QA Tester	5	4	4	5	5	4.6
Bundle Size Analyzer	Frontend Dev	5	4	4	5	5	4.6

References:

• §23700701560

AI generated by Agent Persona Explorer · history