Skip to content

fix(ci): stop AI PR Review from failing a successful review - #17

Merged
shawnmuggle merged 1 commit into
masterfrom
fix/ai-pr-review-red-on-success
Sep 1, 2026
Merged

fix(ci): stop AI PR Review from failing a successful review#17
shawnmuggle merged 1 commit into
masterfrom
fix/ai-pr-review-red-on-success

Conversation

@shawnmuggle

Copy link
Copy Markdown
Member

AI PR Review has failed 26 times across 4 repos in the last 72h while the review itself succeeded every time. Two independent causes, both in the wrap-up after the review is already posted:

  • v1 (ai_pr_review.py): the auto-approve POST returns 401 Unauthorized (AI_REVIEW_APPROVER_TOKEN expired or under-scoped) and raises, after the review comment and pass label have already landed. Now caught: the run degrades to "reviewed, not approved" and prints the HTTP code and likely cause in the step summary.
  • v2 (ai-pr-review-v2.yml): --max-turns 15 is below what a real review costs — gh pr diff + per-file reads + one inline comment per finding. Observed Claude reported a successful result after 18 turns and error_max_turns (no structured_output) on a 3-file PR. Raised to 40; spend is bounded by the daily USD cap, not by this number.

Applied identically across all 16 repos carrying these workflows (v1 script sha was d7a86c89 everywhere, v2 workflow sha 4f88ddb9 everywhere).

Note: the 401 itself is a separate follow-up — the approver PAT still needs rotating for auto-approve to resume.

The review comment and pass label are posted before the approval call, so
a dead AI_REVIEW_APPROVER_TOKEN was turning a completed review into a red
check. Degrade to "reviewed, not approved" and surface the HTTP code in
the step summary instead of raising.
@vercel

vercel Bot commented Sep 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
rozo-deeplink-demo Ready Ready Preview Sep 1, 2026 12:24pm UTC

Request Review

@shawnmuggle
shawnmuggle merged commit 5099dc8 into master Sep 1, 2026
2 of 3 checks passed
@shawnmuggle
shawnmuggle deleted the fix/ai-pr-review-red-on-success branch September 1, 2026 12:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 64afd41cb8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/ai_pr_review.py
f"Review posted, but auto-approve failed "
f"(HTTP {e.code}). {hint}"
)
except urllib.error.URLError as e:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Treat approval timeouts as non-fatal

When GitHub's approval request reaches the 30-second timeout, urllib.request.urlopen can raise TimeoutError, which is not a subclass of urllib.error.URLError. In that transient network scenario, the exception still escapes after the comment and label were posted, leaving the successful review as a failed workflow—the behavior this change is intended to prevent. Catch TimeoutError alongside URLError, as the Anthropic request path already does.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T12:27:13.088216Z 64afd41 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant