[prompt-analysis] Copilot PR Prompt Analysis - 2026-03-24

[github-actions[bot]]

Summary

Analysis Period: Last 30 days (1,000 PRs from copilot/* branches)
Total PRs: 1,000 | Merged: 832 (83.2%) | Closed: 168 (16.8%) | Open: 0

⬆️ Trend: Merge rate improved from 82.5% (2026-03-23) → 83.2% today (+0.7pp). Closed PRs dropped from 173 → 168.

---

Prompt Category Success Rates

Category	Total	Merged	Closed	Success Rate
📚 Documentation	39	38	1	97.4%
🐛 Bug Fix	407	359	48	88.2%
♻️ Refactoring	63	56	7	88.9%
📦 Dependency Update	38	31	7	81.6%
✨ Feature Addition	189	154	35	81.5%
🧪 Testing	4	3	1	75.0%
🚧 WIP ([WIP] tagged)	52	17	35	32.7%

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt body: ~914 words (median: ~518 words)
• 64.5% of merged PRs are reviewed and merged within 1 hour of creation; 80.3% within 2 hours
• 51.4% use conventional commit prefixes (fix:, feat:, docs:, etc.)
• Title verbs: add (149), fix (111), update (48), remove (30), improve (17)
• Non-WIP PRs succeed at 86.0% — a strong baseline

Example merged prompts:

1. PR #22582 – Fix go/unsafe-quoting (CWE-78/89/94) in expression deprecation warning → Merged in < 1h

   "CodeQL flagged user-controlled expression content being embedded inside single-quoted substrings in the activation output..."

2. PR #22577 – Add aw_context parsing from aw_info.json to logs and audit JSON output → Merged

   "aw_info.json already contains a context field (written by generate_aw_info.cjs when triggered via workflow_dispatch)..."

3. PR #22310 – docs(sandbox): remove Custom AWF Configuration → Merged (documentation: 97.4% success rate)

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Slightly longer bodies (avg 987 words vs 914 for merged — over-engineering may signal complexity)
• [WIP] tag is the strongest predictor of closure: 67.3% closure rate vs 14.0% for non-WIP
• Closed PRs more frequently involve bash, lock, engine, inference, node keywords — suggesting infrastructure/tooling changes that hit conflicts or get superseded
• Feature PRs (81.5%) lag bug fix PRs (88.2%) — suggesting features require more context/spec work

Example closed prompts:

1. PR #22571 – Verify aw_context caller metadata parsing in generate_aw_info.cjs → Closed

   Verification/investigation task — may have been superseded or resolved separately

2. PR #22570 / #22569 – Bump/Upgrade GitHub MCP server to v0.32.0 → Closed (both)

   Two competing PRs for the same dependency bump — only one can merge; classic race condition

3. PR #22518 – Fix systemic secrets-outside-env zizmor findings in workflow compiler → Closed

   "Systemic" changes touching many files may get superseded by targeted fixes

Key Insights

• 🚩 [WIP] is a high-risk signal: WIP-tagged PRs close 67% of the time. The tag signals uncertainty to reviewers and the agent itself. Avoid [WIP] unless the PR is intentionally exploratory.

• ⚡ Speed matters: 80% of merged PRs are merged within 2 hours — this repo moves fast. Prompts that generate focused, clean changes succeed quickly. Over-scoped changes (touching many files) slow review and invite closure.

• 🔁 Duplicate PRs get closed: At least 2 closed PRs (Upgrade GitHub MCP server to v0.32.0 #22569, Bump default GitHub MCP server image to v0.32.0 #22570) appear to be competing implementations of the same MCP server version bump. When multiple agents tackle the same task simultaneously, only one survives.

• 📚 Documentation wins: 97.4% success rate for documentation PRs — clear, bounded changes with obvious correctness are fast to approve.

Recommendations

1. DO: Use precise, scoped prompts — "Fix the X in Y" outperforms "Investigate/Verify Z"
2. DO: Apply conventional commit prefixes (fix:, feat:, docs:) — 51.4% of merged PRs use them; they signal intent clearly
3. DO: Target documentation and bug fix work — both have >88% success rates
4. AVOID: Tagging PRs as [WIP] — this 2-in-3 closure predictor should only be used for explicitly exploratory work
5. AVOID: Duplicate dependency bump PRs — coordinate to ensure only one agent handles each version update

Historical Trends (Last 7 Days)

Date	Total	Merged	Closed	Success Rate
2026-03-16	1,000	803	196	80.3%
2026-03-17	1,000	812	185	81.2%
2026-03-18	1,000	818	179	81.8%
2026-03-19	1,000	816	180	81.6%
2026-03-21	1,000	821	177	82.1%
2026-03-22	1,000	825	174	82.5%
2026-03-23	1,000	825	173	82.5%
2026-03-24	1,000	832	168	83.2%

📈 Trend: Steady improvement over the past week (+2.9pp from 80.3% → 83.2%). The reduction in closed PRs (196 → 168) suggests fewer failed attempts and better prompt targeting.

Full Category Breakdown with WIP

Category	Merged	Closed	Total	Rate
Documentation	38	1	39	97.4%
Bug Fix	359	48	407	88.2%
Refactoring	56	7	63	88.9%
Other	174	34	208	83.7%
Dependency Update	31	7	38	81.6%
Feature Addition	154	35	189	81.5%
Testing	3	1	4	75.0%
WIP	17	35	52	32.7%

Merge Speed Distribution

Time Window	Count	% of Merged
< 1 hour	537	64.5%
1–2 hours	131	15.8%
2–24 hours	159	19.1%
> 24 hours	5	0.6%

Median time to merge: 0.7 hours
Average time to merge: 1.9 hours

References:

• §23473576087

AI generated by Copilot PR Prompt Pattern Analysis · history

• expires on Mar 25, 2026, 4:53 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-25T04:53:20.533Z.

Closed by Workflow