🏥 Safe Output Health Report - November 16, 2025

[github-actions[bot]]

🏥 Safe Output Health Report - November 16, 2025

Executive Summary

Analysis of safe output job health over the last 24 hours reveals a significant degradation in create_pull_request reliability, with all 3 PR creation jobs failing (100% failure rate). A new critical issue has emerged: PR reviewer request permission errors are causing complete job failures even when pull requests are created successfully.

Period: Last 24 hours (November 15-16, 2025)

Key Metrics:

• Runs Analyzed: 33
• Safe Output Jobs Executed: 9
• Safe Output Jobs Failed: 3 (33.3% failure rate)
• Error Clusters Identified: 3
• Critical Issues: 1 (new)

Trend Alert: 🔴 DEGRADED - Success rate dropped from 100% (Nov 15) to 66.7% (Nov 16)

Safe Output Job Statistics

Job Type	Total Executions	Failures	Success Rate	Status
create_pull_request	3	3	0%	🔴 Critical
create_issue	5	0	100%	✅ Healthy
create_discussion	1	0	100%	✅ Healthy
TOTAL	9	3	66.7%	🟡 Degraded

Comparison with Previous Day (Nov 15)

Metric	Nov 15	Nov 16	Change
Total Jobs	11	9	-18%
Failed Jobs	0	3	+300%
Success Rate	100%	66.7%	-33.3pp

---

Error Clusters

Cluster 1: PR Reviewer Request Permission Error 🔴 CRITICAL

Impact: High - Causes complete job failure even when PR is created successfully

• Count: 2 occurrences
• Affected Jobs: create_pull_request
• Affected Workflows: Tidy
• Error Pattern: Resource not accessible by personal access token (HTTP 403)
• API Endpoint: /repos/{owner}/{repo}/pulls/{pull_number}/requested_reviewers

Sample Error:

gh: Resource not accessible by personal access token (HTTP 403)
{"message":"Resource not accessible by personal access token","documentation_url":"https://docs.github.com/rest/pulls/review-requests#request-reviewers-for-a-pull-request","status":"403"}
##[error]Process completed with exit code 1.

Root Cause:
The create_pull_request safe output job attempts to request reviewers using gh pr edit --add-reviewer, but the GitHub Actions token or personal access token lacks the necessary permissions to add reviewers via the GitHub API. The PR creation succeeds, but the subsequent reviewer request fails, causing the entire job to fail with exit code 1.

Impact:

• Pull request is created successfully
• Reviewer request fails
• Entire job marked as failed, causing workflow failure
• False negative - the primary operation (PR creation) succeeded

Affected Runs:

• §19366739905 - Tidy workflow (Nov 14, 13:56 UTC)
• §19390353878 - Tidy workflow (Nov 15, 13:14 UTC)

---

Cluster 2: Git Push Workflow Permission Error

Impact: Medium - Has effective fallback (creates issue instead of PR)

• Count: 1 occurrence
• Affected Jobs: create_pull_request
• Affected Workflows: Q
• Error Pattern: refusing to allow a GitHub App to create or update workflow .github/workflows/*.md without workflows permission

Sample Error:

! [remote rejected] q-shorter-description-396c03b9c56b1447 -> q-shorter-description-396c03b9c56b1447 
  (refusing to allow a GitHub App to create or update workflow `.github/workflows/q.md` without `workflows` permission)
error: failed to push some refs to 'https://github.com/githubnext/gh-aw.git'
##[error]Git push failed: The process '/usr/bin/git' failed with exit code 1
##[warning]Git push operation failed - creating fallback issue instead of pull request

Root Cause:
GitHub App token used by the workflow lacks the workflows permission scope, preventing push of commits that modify workflow files in .github/workflows/. This is a security feature to prevent unauthorized workflow modifications.

Impact:

• Git push fails
• Fallback mechanism creates issue instead ([q] Shorten Q workflow description to 2 paragraphs #3987)
• Overall workflow run marked as success (fallback worked)
• Good error handling - user gets output via issue

Affected Runs:

• §19375050197 - Q workflow (Nov 14, 19:14 UTC) - Created fallback issue [q] Shorten Q workflow description to 2 paragraphs #3987

---

Cluster 3: Buffer Deprecation Warning

Impact: Low - No functional impact

• Count: 9 occurrences
• Affected Jobs: All safe output job types
• Affected Workflows: All
• Error Pattern: DeprecationWarning: Buffer() is deprecated

Sample Warning:

(node:11746) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. 
Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.

Root Cause:
The actions/download-artifact@v4 action uses the deprecated Buffer() constructor in Node.js, triggering deprecation warnings.

Impact:

• No functional impact on safe output jobs
• Warning noise in logs
• May become an error in future Node.js versions

---

Root Cause Analysis

API Permission Issues

The most critical finding is that the create_pull_request safe output job has brittle error handling when dealing with GitHub API permission errors. Specifically:

1. PR Creation Succeeds: The core operation (creating the pull request) completes successfully
2. Reviewer Request Fails: A secondary operation (adding reviewers) fails due to insufficient permissions
3. Job Fails Entirely: The entire job is marked as failed, even though the primary objective was accomplished

This creates a false negative situation where workflows appear to have failed, but the user's intended outcome (PR creation) actually succeeded.

Permission Scope Requirements

The GitHub REST API endpoint /repos/{owner}/{repo}/pulls/{pull_number}/requested_reviewers requires:

• Write access to the repository
• Specific permissions to manage pull request reviewers
• Personal access tokens may not have these permissions by default
• GitHub Actions tokens have limited reviewer management capabilities

Workflow Permission Protection

The workflow file modification protection is working as designed:

• Prevents unauthorized modification of workflow files (security feature)
• Fallback issue creation provides good UX
• This is not a bug, but an expected security constraint

---

Recommendations

Critical Issues (Immediate Action Required)

1. Make Reviewer Requests Optional in create_pull_request

Priority: 🔴 Critical

Problem: Reviewer request failures cause complete job failure even when PR is created successfully

Root Cause: No error handling for optional reviewer requests

Recommended Action:

• Wrap reviewer request logic in try-catch or exit code check
• Log warning if reviewer request fails, but continue job execution
• Mark job as successful if PR is created (core operation)

Code Location: create_pull_request safe output script

Expected Outcome:

• PR creation succeeds → Job succeeds (with warning about reviewer request)
• Users get their PRs reliably
• Reviewer requests become "best effort" rather than required

Affected Workflows: Tidy (and any other workflow using create_pull_request)

---

2. Document Token Permission Requirements

Priority: 🟠 High

Problem: Users may not know that reviewer requests require additional permissions

Recommended Action:

• Add documentation to safe output guide about required permissions
• Add configuration option to disable reviewer requests: disable_reviewer_requests: true
• Consider detecting permission issues and auto-disabling reviewer requests

Expected Outcome:

• Users understand permission requirements
• Can opt out of reviewer requests if permissions unavailable
• Better user experience with clear expectations

---

Process Improvements

3. Continue Monitoring Workflow Permission Fallback

Priority: 🟢 Medium

Problem: None - fallback is working well

Recommended Action:

• Continue monitoring the git-push-workflow-permission error pattern
• Ensure fallback issue creation continues to work reliably
• Document this as expected behavior for workflow file modifications

Expected Outcome:

• Users understand this is a security feature, not a bug
• Fallback mechanism provides good UX

---

4. Update actions/download-artifact Dependency

Priority: 🟡 Low

Problem: Deprecation warnings create log noise

Recommended Action:

• Update actions/download-artifact to latest version (v5 if available)
• Or wait for upstream fix in v4
• Track issue in actions/download-artifact repository

Expected Outcome:

• Cleaner logs
• Future-proof against Node.js breaking changes

---

Work Item Plans

Work Item 1: Fix create_pull_request Reviewer Request Error Handling

Type: Bug Fix
Priority: 🔴 Critical
Estimated Effort: Small (1-2 hours)

Description:
The create_pull_request safe output job fails entirely when reviewer requests fail due to permission errors, even though the pull request creation succeeds. This creates false negatives where workflows appear failed but actually accomplished their primary goal.

Acceptance Criteria:

• Reviewer request failures do not cause job failure
• PR creation success results in job success
• Reviewer request failures are logged as warnings
• Job summary shows PR was created successfully even if reviewer request failed
• Tests confirm behavior with and without reviewer permissions

Technical Approach:

1. Locate reviewer request code in create_pull_request script
2. Wrap gh pr edit --add-reviewer in conditional logic
3. Check exit code and log warning if fails
4. Continue job execution regardless of reviewer request outcome
5. Update job output to distinguish between PR creation and reviewer request status

Example Code Pattern:

# After PR creation succeeds
if [ -n "$REVIEWERS" ]; then
  if gh pr edit "$PR_NUMBER" --add-reviewer "$REVIEWERS" 2>/dev/null; then
    echo "✓ Reviewers added successfully"
  else
    echo "⚠ Warning: Could not add reviewers (insufficient permissions)"
    echo "::warning::Reviewer request failed, but PR was created successfully"
  fi
fi

# Ensure job succeeds if PR was created
exit 0

Files to Modify:

• .github/actions/create-pull-request/action.yml (or equivalent script location)

Dependencies: None

Testing Plan:

• Test with token that has reviewer permissions → reviewers added
• Test with token lacking permissions → warning logged, job succeeds
• Verify job output correctly shows PR URL regardless of reviewer status

---

Work Item 2: Add Configuration for Reviewer Requests

Type: Enhancement
Priority: 🟠 High
Estimated Effort: Medium (3-4 hours)

Description:
Add workflow configuration option to disable reviewer requests in create_pull_request, allowing users to opt out if they lack the necessary permissions or don't want automatic reviewer assignment.

Acceptance Criteria:

• New workflow configuration option: disable_reviewer_requests
• Documentation updated to explain option
• Default behavior unchanged (backwards compatible)
• Permission errors automatically suggest disabling reviewer requests

Technical Approach:

1. Add disable_reviewer_requests to workflow schema
2. Pass configuration to create_pull_request job
3. Skip reviewer request logic if disabled
4. Update documentation with examples
5. Add intelligent error message suggesting this option when permissions fail

Example Configuration:

safe-outputs:
  create-pull-request:
    disable_reviewer_requests: true  # Don't try to add reviewers

Files to Modify:

• Workflow schema definition
• create_pull_request safe output script
• Documentation (safe outputs guide)

Dependencies: Work Item 1 (or can be done independently)

Testing Plan:

• Test with option enabled → no reviewer requests attempted
• Test with option disabled → reviewer requests attempted (default)
• Verify documentation examples work correctly

---

Work Item 3: Improve create_pull_request Error Messaging

Type: Enhancement
Priority: 🟡 Medium
Estimated Effort: Small (1-2 hours)

Description:
Improve error messages in create_pull_request to clearly distinguish between different failure scenarios and provide actionable guidance.

Acceptance Criteria:

• Different error messages for different failure types
• Actionable suggestions in error messages
• Links to documentation for troubleshooting
• Clear indication of what succeeded vs failed

Technical Approach:

1. Identify all failure points in create_pull_request
2. Add specific error messages for each scenario
3. Include documentation links
4. Add suggestions for resolution
5. Update job summary to show partial success states

Example Messages:

• PR creation failed: Link to troubleshooting guide
• Reviewer request failed: Suggest disabling reviewer requests or checking permissions
• Workflow push failed: Explain workflow permission requirement and fallback behavior

Files to Modify:

• create_pull_request safe output script
• Error message templates

Dependencies: None (can be done independently)

---

Historical Context

Trends Over Time

Date	Total Jobs	Failed	Success Rate	Critical Issues
Nov 16	9	3	66.7%	1 (new)
Nov 15	11	0	100%	0
Nov 14	164	3	98.2%	0
Nov 13	48	2	95.8%	0
Nov 12	52	8	84.6%	1

Analysis

Recent Degradation:

• Nov 15: Perfect health (100% success rate)
• Nov 16: Significant degradation (66.7% success rate)
• Root cause: New reviewer request permission errors in Tidy workflow

Pattern Observation:

• create_issue and create_discussion remain stable and reliable (100% success)
• create_pull_request has recurring permission-related failures
• Permission issues are the dominant failure mode across all audits

Most Common Recurring Issue:
Permission errors account for 100% of failures in this audit period:

• PR reviewer requests (2 failures)
• Workflow file modifications (1 failure with working fallback)

Improvement Since Last Major Issue (Nov 12):
Success rate improved from 84.6% (Nov 12) to 95.8% (Nov 13) to 100% (Nov 15), but has now regressed to 66.7% (Nov 16) due to the new reviewer request issue.

---

Metrics and KPIs

Overall Safe Output Health

• Overall Success Rate: 66.7% (degraded from 100%)
• Most Reliable Job Type: create_issue (100% success rate, 5/5)
• Most Problematic Job Type: create_pull_request (0% success rate, 0/3)
• Average Job Execution: Safe output jobs in ~3 of 33 workflow runs (9% of runs)

Error Distribution

Error Type	Count	Percentage	Severity
PR Reviewer Permission	2	66.7%	Critical
Workflow Push Permission	1	33.3%	Medium
Total Errors	3	100%	-

Failure Impact

• Workflows Failed Due to Safe Output Errors: 2 (Tidy runs)
• Workflows Succeeded Despite Safe Output Errors: 1 (Q run - fallback worked)
• False Negative Rate: 66.7% (2 of 3 failures were PRs that actually succeeded)

---

Next Steps

Immediate Actions (This Week)

• Implement error handling for reviewer requests (Work Item 1)
• Add configuration option to disable reviewer requests (Work Item 2)
• Update documentation with permission requirements
• Monitor Tidy workflow for continued reviewer request failures

Follow-up Investigations

• Audit all safe output jobs for similar brittle error handling
• Review other GitHub API calls that may have optional permission requirements
• Consider adding permission detection/validation before attempting operations

Long-term Improvements

• Implement comprehensive error handling framework for safe outputs
• Add permission requirement validation at workflow compile time
• Create safe output job health dashboard
• Establish SLOs for safe output job reliability (target: 95%+)

---

References:

• §19366739905
• §19375050197
• §19390353878

AI generated by Safe Output Health Monitor

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 1 week ago.