[audit] Agentic Workflow Audit — 2026-03-27

[github-actions[bot]]

Daily audit of agentic workflow runs for the 24-hour period ending 2026-03-27T21:05Z.

Summary

Metric	Value
Total runs (completed)	14
✅ Success	8 (57%)
❌ Failure	6 (43%)
🔄 In Progress	3
Total tokens	11,338,941
Estimated cost	$5.68

Note: 5 of 6 "failed" runs had agents that concluded successfully — failures occurred in post-agent infrastructure steps (detection, push_repo_memory, cache), not the AI agents themselves. Only 1 run (Metrics Collector) had no agent output.

---

📊 Trend Charts

Workflow Health:

Success rate dropped to 57% today (compared to 71% on 2026-02-24), driven primarily by recurring threat detection model format failures and infrastructure issues in post-agent steps.

Token & Cost:

Today's cost of $5.68 for 11.3M tokens is similar to the Feb-24 baseline ($3.26 for 128M tokens — significantly higher token count). The Sergo Go Expert workflow consumed the most tokens at 4.2M (76 turns), followed by Static Analysis Report at 1.4M (31 turns).

---

❌ Failed Runs — Root Cause Analysis

View All 6 Failed Runs

1. Daily Workflow Updater — Detection Auth Failure

• Run: §23667136119 | Engine: copilot | Turns: 16 | Tokens: 618,858
• Agent Result: ✅ Success — created PR to update 4 actions (codeql-action, gh-aw-actions/setup, stale-repos, setup-ruby)
• Failure cause: Detection step authentication failure: Authentication failed (Request ID: 0C51:9FAA7:C50D3A:DD0350:69C6EFE5) — transient API auth error
• Impact: PR creation safe output pending; workflow blocked from posting

2. Auto-Triage Issues — Detection Format Failure (Run 1)

• Run: §23662084001 | Engine: copilot | Turns: 20 | Tokens: 1,002,208
• Agent Result: ✅ Success — labeled issues
• Failure cause: No THREAT_DETECTION_RESULT found in detection log — Copilot detection model did not produce expected structured output format
• Impact: Agent outputs blocked by failed detection

3. Auto-Triage Issues — Detection Format Failure (Run 2)

• Run: §23667067977 | Engine: claude | Turns: 3 | Tokens: 105,059
• Agent Result: ✅ Success — added mcp + security labels to issue DIFC proxy does not pass GITHUB_SERVER_URL to container — breaks GHEC integrity filtering #23274 (COLLABORATOR-filed bug)
• Failure cause: Same detection format failure as above
• Impact: Recurrence within same day

4. Daily DIFC Integrity-Filtered Events Analyzer — Missing Cache Artifact

• Run: §23665157709 | Engine: copilot | Turns: 0
• Agent Result: ✅ Success — created discussion with 746-event analysis, uploaded 4 charts
• Failure cause: update_cache_memory step: Artifact not found for name: cache-memory — cache artifact expired
• Impact: Cache update skipped; agent output successfully delivered

5. PR Triage Agent — Local Action Missing Post-Checkout

• Run: §23660968453 | Engine: copilot | Turns: 11 | Tokens: 529,818
• Agent Result: ✅ Success (noop) — no fork PRs to triage, posted to no-op runs issue #21483
• Failure cause: push_repo_memory step: Can't find 'action.yml' under actions/setup — after sparse checkout switches to memory branch, local action is unavailable
• Impact: Repo memory write failed; workflow marked failure despite successful agent noop

6. Metrics Collector - Infrastructure Agent — File Pattern Mismatch

• Run: §23663993174 | Engine: copilot
• Agent Result: ❌ No safe outputs — agent ran but produced no output visible to the workflow
• Failure cause: push_repo_memory wrote agent-performance-latest.md but the memory filter is metrics/** — file was skipped (0 files copied)
• Impact: No metrics data written to memory; infrastructure state not persisted

---

✅ Successful Runs

Workflow	Engine	Tokens	Turns
Step Name Alignment	claude	939,534	21
Static Analysis Report	claude	1,384,721	31
Copilot Agent Prompt Clustering Analysis	claude	1,148,195	23
Sergo - Serena Go Expert	claude	4,241,516	76
Daily Documentation Updater	claude	1,369,032	35
AI Moderator (×3)	codex	—	—

---

🔍 Key Issues & Recommendations

Issue 1: Recurring Threat Detection Format Failures ⚠️ HIGH

Pattern: 2/6 failures from No THREAT_DETECTION_RESULT found in detection log

• The Copilot model used for threat detection is not reliably producing the expected structured format THREAT_DETECTION_RESULT:{...}
• Affects all Copilot-engine workflows that use threat detection
• Recommendation: Add retry logic for detection step; improve detection prompt to more forcefully specify output format; consider fallback detection path

Issue 2: Detection Authentication Failures ⚠️ MEDIUM

Pattern: Transient API auth failure during detection container startup

• Recommendation: Add retry with exponential backoff for auth failures in detection step

Issue 3: Metrics Collector File Pattern Mismatch 🔧 MEDIUM

Pattern: Workflow writes agent-performance-latest.md but memory config filters metrics/**

• Recommendation: Update Metrics Collector to write output under metrics/ subdirectory, or adjust the memory filter pattern to match the agent's actual output path

Issue 4: Cache Memory Artifact Expiry 🔧 LOW

Pattern: update_cache_memory step fails when artifact doesn't exist yet

• Recommendation: Add if-no-files-found: warn or conditional step to handle missing cache artifact gracefully on first run

Issue 5: actions/setup Missing After Memory Branch Checkout 🔧 LOW

Pattern: push_repo_memory step switches to memory branch, losing local action definitions

• Recommendation: Ensure the push_repo_memory step pre-caches required actions before switching branches

---

References

• §23667136119 — Daily Workflow Updater (detection auth failure)
• §23662084001 — Auto-Triage Issues (detection format failure)
• §23663993174 — Metrics Collector (file pattern mismatch)

AI generated by Agentic Workflow Audit Agent · history

• expires on Mar 28, 2026, 9:20 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-28T21:20:49.099Z.

Closed by Workflow