[prompt-clustering] Copilot Agent Prompt Clustering Analysis — 2026-03-27

[github-actions[bot]]

Summary

Analysis Period: Last 30 days (2026-02-25 to 2026-03-27)
Total Tasks Analyzed: 411
Clusters Identified: 7
Overall Success Rate: 76% (311 merged / 411 total)

Metric	Value
Total PRs analyzed	411
Successfully merged	311 (76%)
Closed without merge	64
Currently open	36
Clusters identified	7

Cluster Analysis Details

Cluster Overview

#	Theme	Tasks	Success Rate	Top Keywords
1	Updates & Dependency Management	119	76%	update, version, cli, mcp
2	Bug Fixes via Issue Resolution	74	84%	failure, error, issue, workflow
3	Safe Outputs & Copilot Feature Dev	58	66%	issue, copilot, agent, comments
4	CI/CD Failure Investigation	50	86%	fix, workflow, logs, failing
5	Agentic Workflow Architecture	43	74%	agentic, workflows, pull request
6	Feature Planning & Prompts	40	65%	plan, discussion, context
7	Safe-Outputs Infrastructure	27	74%	safe outputs, steps, upload

Detailed Cluster Analysis

Updates & Dependency Management

• Size: 119 tasks (29.0% of total)
• Success Rate: 76% (90 merged, 18 closed without merge, 11 open)
• Top Keywords: update, github, version, cli, mcp, review
• Description: Dependency updates, version bumps, CLI/MCP config upgrades
• Example PRs: #18268, #18313, #18319, #18332

---

Bug Fixes via Issue Resolution

• Size: 74 tasks (18.0% of total)
• Success Rate: 84% (62 merged, 7 closed without merge, 5 open)
• Top Keywords: job, failure, error, issue, workflow, step
• Description: Fixing specific bugs/failures identified via issues, typically involving workflow/step errors
• Example PRs: #18251, #18258, #18266, #18318

---

Safe Outputs & Copilot Feature Dev

• Size: 58 tasks (14.1% of total)
• Success Rate: 66% (38 merged, 15 closed without merge, 5 open)
• Top Keywords: issue, section, copilot, workflow, issue copilot, agent
• Description: Developing safe-output capabilities, copilot integrations, and issue-based features
• Example PRs: #18257, #18327, #18364, #18370

---

CI/CD Failure Investigation

• Size: 50 tasks (12.2% of total)
• Success Rate: 86% (43 merged, 6 closed without merge, 1 open)
• Top Keywords: job, fix, workflow, logs identify, actions workflow, analyze workflow
• Description: Analyzing failing GitHub Actions workflows and implementing root-cause fixes
• Example PRs: #18450, #18725, #18728, #18735

Sample prompt: "Fix the failing GitHub Actions workflow Integration: CLI Update Command. Analyze the workflow logs, identify the root cause of the failure, and implement a fix."

---

Agentic Workflow Architecture

• Size: 43 tasks (10.5% of total)
• Success Rate: 74% (32 merged, 5 closed without merge, 6 open)
• Top Keywords: remove, agentic, workflows, agentic workflows, pull, pull request
• Description: Core architecture changes to workflow orchestration, PR flows, and engine interfaces
• Example PRs: #18320, #18536, #18796, #18802

---

Feature Planning & Prompt Engineering

• Size: 40 tasks (9.7% of total)
• Success Rate: 65% (26 merged, 10 closed without merge, 4 open)
• Top Keywords: plan, discussion github, discussion, issue, context, github
• Description: Planning features, creating prompts/reference docs, expanding capabilities
• Example PRs: #18271, #18272, #18273, #18274

---

Safe-Outputs Infrastructure

• Size: 27 tasks (6.6% of total)
• Success Rate: 74% (20 merged, 3 closed without merge, 4 open)
• Top Keywords: safe, safe output, output, outputs, safe outputs, support
• Description: Building safe-output tooling, steps injection, upload/LFS support
• Example PRs: #18460, #18626, #18647, #18701

Sample prompt: "Add a safe-output.steps field that allows users to inject steps into the safe-output jobs. These steps must run after checking out the repository and setting up the action, and before configuring or running the agent."

Key Findings & Recommendations

Key Findings

1. CI/CD Failure Investigation has the highest success rate (86%): Tasks that ask the agent to analyze failing GitHub Actions workflows, identify root causes, and implement fixes are the most reliably completed. The agent excels at structured debugging workflows.

2. Bug Fixes via Issues is also high-performing (84%): Structured issue-based bug fix requests (with clear issue titles, descriptions, and context) achieve strong merge rates. The structured <issue_title> + <issue_description> prompt format helps the agent understand scope.

3. Feature Planning & Prompts has the lowest merge rate (65%): Open-ended planning, prompt engineering, and feature expansion tasks are harder — likely due to more subjective review standards and larger scope uncertainty.

4. Updates & Dependency Management is the largest cluster (29% of all tasks): Routine dependency/version bump tasks dominate the workload with 76% success, indicating these are well-understood patterns but can fail when there are breaking changes.

5. Safe Outputs & Copilot Feature Dev (66% merge rate): Feature development through issue-based copilot prompts has more variability — larger feature scope leads to more iteration/review cycles.

Recommendations

1. Prefer structured issue prompts for bug fixes: The bug-fix cluster's 84% success rate correlates with well-structured <issue_title> + <issue_description> prompt patterns. Apply this format more broadly across task types.

2. Decompose planning tasks into concrete sub-tasks: The 65% success rate on planning/prompt-engineering tasks suggests these benefit from being split into more targeted, actionable sub-tasks before assignment.

3. CI failure investigation is a strength — lean into it: With 86% success, automated CI failure detection and agent assignment is high-ROI. Consider expanding coverage to more workflow types.

4. Monitor open PRs in Agentic Architecture cluster: 6 open PRs (14%) in the architecture cluster suggest larger, more complex changes that may need additional review time or scope reduction.

Full PR Data Table (First 80 PRs by cluster)

PR	Title	Cluster Theme	Outcome
#18460	Add safe-outputs.steps for injecting custom steps	Safe-Outputs Infrastructure	🟡 Open
#18626	feat: Git LFS support for upload-asset safe output	Safe-Outputs Infrastructure	🔴 Closed
#18647	fix: include noop in safe-output tools prompt	Safe-Outputs Infrastructure	🟡 Open
#18701	Auto-detect OIDC/vault actions in safe-outputs	Safe-Outputs Infrastructure	✅ Merged
#18754	Fix missing cross-repo and auth properties in safe outputs	Safe-Outputs Infrastructure	✅ Merged
#18834	Align title-prefix tool description constraints	Safe-Outputs Infrastructure	✅ Merged
#18914	feat: expose safe-output results as workflow_call outputs	Safe-Outputs Infrastructure	✅ Merged
#18989	Add set-issue-type safe output type	Safe-Outputs Infrastructure	✅ Merged
#18993	Add concurrency-group support to safe_outputs job	Safe-Outputs Infrastructure	✅ Merged
#19552	Display secrecy and integrity fields in safe output steps	Safe-Outputs Infrastructure	✅ Merged
#19620	Set default max to 1 for safe outputs (original)	Safe-Outputs Infrastructure	🔴 Closed
#19621	Enterprise: replace hardcoded github.com with GITHUB_SERVER_URL	Safe-Outputs Infrastructure	✅ Merged
#19965	feat(add): fetch dispatch-workflow dependencies	Safe-Outputs Infrastructure	✅ Merged
#20114	Fix safe output handler to gracefully ignore custom safe outputs	Safe-Outputs Infrastructure	✅ Merged
#20115	Rename safe-inputs to mcp-scripts	Safe-Outputs Infrastructure	✅ Merged
#20507	Merge agent job artifacts into single agent artifact	Safe-Outputs Infrastructure	✅ Merged
#21285	docs: add taint metadata to safe outputs specification	Safe-Outputs Infrastructure	🔴 Closed
#21329	fix: activate GitHub App configuration in shared workflows	Safe-Outputs Infrastructure	✅ Merged
#21415	Add daily-safe-output-integrator agentic workflow	Safe-Outputs Infrastructure	✅ Merged
#21578	Add support for temporary ids in assign milestone	Safe-Outputs Infrastructure	🟡 Open
#21582	Add support for defined custom safe-output scripts	Safe-Outputs Infrastructure	✅ Merged
#21752	feat: mount custom GitHub Actions as safe output tools	Safe-Outputs Infrastructure	🟡 Open
#21806	Inject sanitizeContent into safe-output script wrapper	Safe-Outputs Infrastructure	✅ Merged
#21836	Comment out unconfigured app credentials in activation	Safe-Outputs Infrastructure	✅ Merged
#21902	Bump absolute maximum for update_issue and update_discussion	Safe-Outputs Infrastructure	✅ Merged
#21903	Skip write permissions for staged safe output handlers	Safe-Outputs Infrastructure	✅ Merged
#18320	Add schema path heuristic for misplaced frontmatter fields	Agentic Workflow Architecture	✅ Merged
#18536	Improve fallback issue instructions when PR creation fails	Agentic Workflow Architecture	✅ Merged
#18796	Remove SupportsFirewall from agentic engine interface	Agentic Workflow Architecture	✅ Merged
#18838	Rename supportsLLMGateway bool to llmGatewayPort	Agentic Workflow Architecture	✅ Merged
#18842	feat: label agent failure issues with agentic-workflows	Agentic Workflow Architecture	✅ Merged
#18891	feat: add target, target-repo, and allowed-repos to resolve	Agentic Workflow Architecture	✅ Merged
#18942	feat: add footer: false support to add-comment	Agentic Workflow Architecture	✅ Merged
#19016	Remove --create-pull-request and --push flags from add	Agentic Workflow Architecture	🟡 Open
#19079	Remove inline reporting guidelines from 33 workflows	Agentic Workflow Architecture	✅ Merged
#19117	add command: remove --push flag, require interactive	Agentic Workflow Architecture	✅ Merged
#19142	Detect triggers: keyword misuse in compile	Agentic Workflow Architecture	✅ Merged
#19220	Add fetch option to checkout: for cross-repo branches	Agentic Workflow Architecture	✅ Merged
#19308	Add workflow_dispatch operation input and unified run	Agentic Workflow Architecture	✅ Merged
#19316	Remove copilot-requests feature flag from smoke tests	Agentic Workflow Architecture	✅ Merged
#19336	Remove copilot-requests: true feature from all agentic	Agentic Workflow Architecture	🟡 Open
#19344	Add history link to add_comment, update_issue, update_discussion	Agentic Workflow Architecture	✅ Merged
#19374	Disable non-operation jobs in agentic maintenance	Agentic Workflow Architecture	✅ Merged
#19741	Add /ace slash command workflow	Agentic Workflow Architecture	✅ Merged
#19958	Add protected-files support to runtimes	Agentic Workflow Architecture	✅ Merged
#20349	docs: add self-contained and Copilot debugging prompts	Agentic Workflow Architecture	✅ Merged
#20732	feat: add daily-function-namer agentic workflow	Agentic Workflow Architecture	✅ Merged
#20933	docs: require Copilot coding agent to always create PRs	Agentic Workflow Architecture	✅ Merged
#20935	Add excluded-files field to create-pull-request	Agentic Workflow Architecture	✅ Merged
#20987	Optimize qmd-docs workflows: instruct models explicitly	Agentic Workflow Architecture	✅ Merged
#18268	Use GFM Alert syntax for discussion-to-issue fallback warning	Updates & Dependency Management	✅ Merged
#18313	Update Claude Code 2.1.52→2.1.56 and Copilot CLI 0.0.415→0.0.417	Updates & Dependency Management	✅ Merged
#18319	deps: update golang.org/x/tools from v0.41.0 to v0.42.0	Updates & Dependency Management	✅ Merged
#18345	Reformat footer install message with hyperlinked label	Updates & Dependency Management	✅ Merged
#18375	Add Daily Documentation Healer workflow	Updates & Dependency Management	✅ Merged
#18430	Add URL redirects for all historical docs file renames	Updates & Dependency Management	✅ Merged
#18504	Emit default environment variables for GitHub Enterprise	Updates & Dependency Management	✅ Merged
#18515	Update CLI versions: Claude Code 2.1.59, Codex 0.105.0	Updates & Dependency Management	✅ Merged
#18251	fix(SEC-005): remove false-positive cross-repo pattern matches	Bug Fixes via Issue Resolution	✅ Merged
#18258	refactor: semantic function clustering — eliminate duplicates	Bug Fixes via Issue Resolution	✅ Merged
#18266	docs: document intentional exclusion of head_commit.id	Bug Fixes via Issue Resolution	✅ Merged
#18318	fix: clarify audit report on missing_tool vs missing_data	Bug Fixes via Issue Resolution	✅ Merged
#18257	feat(safe-outputs): add duplicate state_reason to close-issue	Safe Outputs & Copilot Feature Dev	✅ Merged
#18327	fix(safe-outputs): preserve custom XML tags, escape known HTML/SVG	Safe Outputs & Copilot Feature Dev	✅ Merged
#18364	fix(scout): normalize report headers to h3	Safe Outputs & Copilot Feature Dev	✅ Merged
#18450	fix: correct TestUpdateCommand_NoMergeFlag assertion	CI/CD Failure Investigation	✅ Merged
#18725	fix: use GetActionPin dynamically in tests	CI/CD Failure Investigation	✅ Merged
#18271	Expand language ecosystem inference for network.allowed	Feature Planning & Prompts	✅ Merged
#18272	Add visual regression reference prompt	Feature Planning & Prompts	✅ Merged
#18273	feat: add test-coverage prompt with artifact-reading guidance	Feature Planning & Prompts	✅ Merged
#18274	feat: add perf-benchmark prompt for performance testing	Feature Planning & Prompts	✅ Merged

References: Workflow Run §23664944049

AI generated by Copilot Agent Prompt Clustering Analysis · history

• expires on Mar 28, 2026, 8:22 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-28T20:22:54.195Z.

Closed by Workflow