Make threat-detect binary install step continue-on-error in warn mode - #52400
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
PR TriageCategory: chore | Risk: low | Priority: low | Score: 31/100 Score breakdown: impact 15/50, urgency 8/30, quality 8/20 CI: unknown (draft) Recommended action: CI hardening chore, still draft/blocked. Automated triage — run 52400
|
PR TriageCategory: chore · Risk: low · Priority: low Makes threat-detect binary install step continue-on-error in warn mode. Mostly bulk lock.yml regeneration from a one-line source change; low value/urgency.
|
There was a problem hiding this comment.
Pull request overview
Makes threat-detect installation failures non-fatal in warn mode while preserving strict and expression-configured behavior.
Changes:
- Applies detection
continue-on-errorpolicy to binary installation. - Adds default warn-mode coverage.
- Regenerates 104 affected workflows.
Show a summary per file
| File | Description |
|---|---|
pkg/workflow/threat_detection_steps.go |
Applies failure policy to installation. |
pkg/workflow/threat_detection_isolation_test.go |
Tests default warn mode. |
.github/workflows/docs-noob-tester.lock.yml |
Regenerates tolerant install step. |
.github/workflows/ci-coach.lock.yml |
Regenerates tolerant install step. |
.github/workflows/commit-changes-analyzer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/ab-testing-advisor.lock.yml |
Regenerates tolerant install step. |
.github/workflows/agentic-token-trend-audit.lock.yml |
Regenerates tolerant install step. |
.github/workflows/copilot-session-insights.lock.yml |
Regenerates tolerant install step. |
.github/workflows/breaking-change-checker.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-elixir-credo-snippet-audit.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-agentrx-trace-optimizer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-cli-tools-tester.lock.yml |
Regenerates tolerant install step. |
.github/workflows/blog-auditor.lock.yml |
Regenerates tolerant install step. |
.github/workflows/cloclo.lock.yml |
Regenerates tolerant install step. |
.github/workflows/q.lock.yml |
Regenerates tolerant install step. |
.github/workflows/copilot-pr-prompt-analysis.lock.yml |
Regenerates tolerant install step. |
.github/workflows/issue-monster.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-astrostylelite-markdown-spellcheck.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-code-metrics.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-multi-device-docs-tester.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-choice-test.lock.yml |
Regenerates tolerant install step. |
.github/workflows/duplicate-code-detector.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-evals-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-byok-ollama-test.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-file-diet.lock.yml |
Regenerates tolerant install step. |
.github/workflows/changeset.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-formal-spec-verifier.lock.yml |
Regenerates tolerant install step. |
.github/workflows/test-quality-sentinel.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-hippo-learn.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-assign-issue-to-user.lock.yml |
Regenerates tolerant install step. |
.github/workflows/copilot-pr-merged-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/deep-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/impeccable-skills-reviewer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/pr-code-quality-reviewer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/cli-version-checker.lock.yml |
Regenerates tolerant install step. |
.github/workflows/github-remote-mcp-auth-test.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-doc-healer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-experiment-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/avenger.lock.yml |
Regenerates tolerant install step. |
.github/workflows/copilot-opt.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-regulatory.lock.yml |
Regenerates tolerant install step. |
.github/workflows/craft.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-model-inventory.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-news.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-performance-summary.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-reliability-review.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-doc-updater.lock.yml |
Regenerates tolerant install step. |
.github/workflows/agent-performance-analyzer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-token-consumption-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-mcp-concurrency-analysis.lock.yml |
Regenerates tolerant install step. |
.github/workflows/audit-workflows.lock.yml |
Regenerates tolerant install step. |
.github/workflows/api-consumption-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-cli-performance.lock.yml |
Regenerates tolerant install step. |
.github/workflows/constraint-solving-potd.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-graft-intelligence.lock.yml |
Regenerates tolerant install step. |
.github/workflows/chaos-pr-bundle-fuzzer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/ci-doctor.lock.yml |
Regenerates tolerant install step. |
.github/workflows/contribution-check.lock.yml |
Regenerates tolerant install step. |
.github/workflows/pr-sous-chef.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-compiler-quality.lock.yml |
Regenerates tolerant install step. |
.github/workflows/design-decision-gate.lock.yml |
Regenerates tolerant install step. |
.github/workflows/archivx-agentic-workflows-analyzer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-function-namer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-observability-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/typist.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-ambient-context-optimizer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/prompt-clustering-analysis.lock.yml |
Regenerates tolerant install step. |
.github/workflows/code-scanning-fixer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/aw-failure-investigator.lock.yml |
Regenerates tolerant install step. |
.github/workflows/architecture-guardian.lock.yml |
Regenerates tolerant install step. |
.github/workflows/github-mcp-structural-analysis.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-architecture-diagram.lock.yml |
Regenerates tolerant install step. |
.github/workflows/copilot-agent-analysis.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-agent-of-the-day-blog-writer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/example-workflow-analyzer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-credit-limit-test.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-compiler-threat-spec-optimizer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/pr-description-caveman.lock.yml |
Regenerates tolerant install step. |
.github/workflows/deployment-incident-monitor.lock.yml |
Regenerates tolerant install step. |
.github/workflows/eslint-monster.lock.yml |
Regenerates tolerant install step. |
.github/workflows/copilot-cli-deep-research.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-awf-spec-compiler-surfacing.lock.yml |
Regenerates tolerant install step. |
.github/workflows/ponytail-reviewer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-cache-strategy-analyzer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-max-ai-credits-test.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-caveman-optimizer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-model-resolution.lock.yml |
Regenerates tolerant install step. |
.github/workflows/cli-consistency-checker.lock.yml |
Regenerates tolerant install step. |
.github/workflows/detection-analysis-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/agent-job-health.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-issues-report.lock.yml |
Regenerates tolerant install step. |
.github/workflows/artifacts-summary.lock.yml |
Regenerates tolerant install step. |
.github/workflows/agent-persona-explorer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-geo-optimizer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/approach-validator.lock.yml |
Regenerates tolerant install step. |
.github/workflows/mattpocock-skills-reviewer.lock.yml |
Regenerates tolerant install step. |
.github/workflows/archie.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-testify-uber-super-expert.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-aw-cross-repo-compile-check.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-community-attribution.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-repo-chronicle.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-rendering-scripts-verifier.lock.yml |
Regenerates tolerant install step. |
.github/workflows/copilot-pr-nlp-analysis.lock.yml |
Regenerates tolerant install step. |
.github/workflows/auto-triage-issues.lock.yml |
Regenerates tolerant install step. |
.github/workflows/claude-code-user-docs-review.lock.yml |
Regenerates tolerant install step. |
.github/workflows/daily-fact.lock.yml |
Regenerates tolerant install step. |
Review details
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 106/106 changed files
- Comments generated: 1
- Review effort level: Balanced
| if !strings.Contains(installStepBlock, "continue-on-error: true") { | ||
| t.Error("Install threat-detect binary step must set continue-on-error: true in warn mode") |
There was a problem hiding this comment.
Added strict-mode and expression-mode test cases (TestExternalDetectorInstallStepContinueOnErrorStrictMode, TestExternalDetectorInstallStepContinueOnErrorExpressionMode) that assert the install step directly, in a248f03.
|
✅ Ponytail Reviewer completed successfully!
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. No ADR enforcement needed: PR #52400 does not have the 'implementation' label and has only 44 new lines of code in business logic directories (threshold: 100).
|
|
✅ PR Code Quality Reviewer completed the code quality review.
|
|
✅ Test Quality Sentinel completed test quality analysis.
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
There was a problem hiding this comment.
Request changes
This change weakens the strict-mode detection contract: the external detector install step now inherits the warn-mode continue-on-error behavior in every generated workflow, so a failed threat-detect download can be reported as success even when safe-outputs.threat-detection.continue-on-error: false is configured.
The blocking issue
The compiler already preserves strict vs warn behavior for the conclusion step, and buildInstallThreatDetectStep has tests/comments describing the same split. But the generated lockfiles in this PR all add continue-on-error: true unconditionally on the install step. That means a strict workflow cannot actually enforce binary availability anymore: the install step failure is swallowed before the detection job reaches the logic that is supposed to block safe outputs. Please add coverage for the strict and expression branches and make sure the recompiled workflows reflect the configured continue-on-error value instead of hard-coding true.
🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 13.1 AIC · ⌖ 4.56 AIC · ⊞ 6.5K
Comment /review to run again
There was a problem hiding this comment.
Ponytail review: one finding.
net: -8 lines possible.
Generated by ✂️ Ponytail Reviewer for #52400 · auto · 27.2 AIC · ⌖ 3.4 AIC · ⊞ 6.8K
Comment /ponytail to run again
| // treat a failed/absent binary as a non-fatal detection failure via | ||
| // GH_AW_DETECTION_CONTINUE_ON_ERROR. Without continue-on-error on this install step, a | ||
| // transient download failure (e.g. a GitHub Releases CDN blip) would still mark this step — | ||
| // and therefore the whole detection job — as `failure`, even though the workflow logic |
There was a problem hiding this comment.
L519-524: yagni: third copy of the continue-on-error resolution block (data.SafeOutputs.ThreatDetection lookup). Extract a resolveContinueOnError(data) (bool, string) helper used by all three buildStep functions.
There was a problem hiding this comment.
Extracted resolveThreatDetectionContinueOnError and reused it across all three build*Step functions in 4e7627b.
PR Triage
Automated triage by PR Triage Agent.
|
PR TriageCategory: Score breakdown
Recommended action:
|
|
@copilot merge main and recompile |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Merged main and recompiled all |
|
@copilot Quick triage for maintainer-ready follow-up: Please refresh the branch if needed, address the remaining maintainer-facing follow-up, and run the Outstanding review items (newest first):
Failed checks from the compact candidate set:
Branch update was requested automatically for this run when GitHub allows it.
|
…ntinue-on-error Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
PR TriageCategory: chore | Risk: medium | Score: 55/100
Recommended action:
|
|
@copilot Quick triage for maintainer-ready follow-up: Please refresh the branch if needed, address the remaining maintainer-facing follow-up, and run the Outstanding review items (newest first):
Failed checks from the compact candidate set:
Branch update was requested automatically for this run when GitHub allows it.
|
|
@copilot Quick triage for maintainer-ready follow-up: Please refresh the branch if needed, address any remaining maintainer-facing follow-up, and run the Outstanding review items (newest first):
Failed checks from the compact candidate set:
Branch update was requested automatically for this run when GitHub allows it.
|
PR Triage
Adds continue-on-error tolerance to threat-detect installer across 104 recompiled lockfiles. CI passing. Note: existing labels on this PR are conflicting (multiple pr-priority/pr-action values) — recommend manual label cleanup. Automated triage — see [PR Triage Report] for full context.
|
|
🎉 This pull request is included in a new release. Release: |
A GitHub Releases CDN blip caused the
evals/detectionjob installers for the AWF firewall and threat-detect binaries to fail across multiple PR-gate workflows within a 25-minute window. The threat-detect binary download already had retry-with-backoff (curl --retry 5 --retry-delay 10 --retry-max-time 180), but a hard failure after retries still marked the wholedetectionjob asfailure— even though the job's own logic (GH_AW_DETECTION_CONTINUE_ON_ERROR) already treats a missing threat-detect binary as non-fatal in warn mode.Compiler change
buildInstallThreatDetectStepnow takes*WorkflowDataand emitscontinue-on-error: true(or the configured expression) for the "Install threat-detect binary" step, reusing the same continue-on-error resolution already applied to the detection conclusion/analysis steps:This keeps the install step's failure semantics consistent with the job's existing tolerance policy: in warn mode a transient download failure no longer flips the job conclusion to
failure; in strict mode (continue-on-error: false) the step still fails the job as before.Test coverage
threat_detection_isolation_test.go: added an assertion inTestExternalDetectorPathverifying the install step carriescontinue-on-error: truein the default warn mode.Generated workflows
.lock.ymlfiles; 104 workflows usingfeatures: gh-aw-detection: truepick up the newcontinue-on-error: trueline on this step.Run: https://github.com/github/gh-aw/actions/runs/31871584565> Generated by 👨🍳 PR Sous Chef · gpt54 · 11.5 AIC · ⌖ 5.77 AIC · ⊞ 8.7K · ◷