Skip to content

fix(review): avoid publishing empty review output - #3030

Merged
IsmaelMartinez merged 4 commits into
The-PR-Agent:mainfrom
oleksii-tumanov:fix/review-empty-output
Sep 6, 2026
Merged

fix(review): avoid publishing empty review output#3030
IsmaelMartinez merged 4 commits into
The-PR-Agent:mainfrom
oleksii-tumanov:fix/review-empty-output

Conversation

@oleksii-tumanov

@oleksii-tumanov oleksii-tumanov commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #2730, which fixed #2729.

Related to #2522, fixed by #2542, which handles empty model content earlier in the response pipeline.

Problem

When /review receives a non-empty model response that cannot produce a non-empty review mapping, _prepare_pr_review() returns an empty string. The publication path currently treats that result as publishable, so a persistent rerun can replace a previous review with empty content.

Changes

  • Validate that review is a non-empty mapping before structured, push, or comment publication, while preserving the existing GitHub Action output behavior.
  • Treat an empty per-chunk review as unusable so the existing single-call fallback runs.
  • Route an unusable prepared review through the existing review-failure lifecycle.
  • Add regression coverage for invalid YAML, an empty review, and an invalid review shape across persistent and non-persistent runs.

Testing

  • PYTHONPATH=. uv run pytest -q tests/unittest — 3629 passed, 1 skipped, 1 xfailed
  • uv run ruff check pr_agent/tools/pr_reviewer.py tests/unittest/test_pr_reviewer_core.py tests/unittest/test_review_large_diff_chunking.py — passed

@github-actions github-actions Bot added the bug label Sep 3, 2026

@IsmaelMartinez IsmaelMartinez left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, and for the regression coverage that came with it.

Correct fix, and it covers more than your description claims: against main, a chunked review where every chunk returns review: {} publishes a comment that is nothing but the "reviewed in N chunks" footer.

One follow-up, and it is not a fault in what you wrote. pr_reviewer.py:328 still accepts an empty review, so it now disagrees with your new guard at :391. Main skips the single-call fallback for that input too, so this predates you, but your guard turns the case into "Failed to review PR". Adding or not data["review"] at :328 fires that fallback for the first time and keeps the suite green. It sits between your two hunks so I could not leave it as a suggestion inline.

@oleksii-tumanov

Copy link
Copy Markdown
Contributor Author

Good point. I updated the chunk validation so review: {} now triggers the existing single-call fallback, and added regression coverage for that path.

@IsmaelMartinez IsmaelMartinez left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the :328 guard is exactly right and the new fallback test covers it: 90 passed on the merged tree, 13 red with the source restored.

One question inline. Separately, this now conflicts in pr_agent/tools/pr_reviewer.py, which is my doing: #3036 and #2722 both landed there in the last hour. Sorry for the rebase.

Comment thread pr_agent/tools/pr_reviewer.py Outdated
get_logger().exception("Failed to parse review data", artifact={"data": data})
return ""

github_action_output(data, 'review')

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this below the guard is a behaviour change that is not in the description. On main github_action_output always ran, so an empty or malformed review still wrote review={} to $GITHUB_OUTPUT. Here it writes nothing at all, and a workflow reading that output goes from {} to unset.

Deliberate? I can see the argument either way, but it is visible outside pr-agent, so it is worth a line in the description rather than riding along with the fix.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. I kept the existing GitHub Action behavior, so a malformed or empty review still writes the same output as before. The guard now only stops it from reaching the review publication paths, and I added coverage for that.

@qodo-code-review

qodo-code-review Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Code Review by Qodo

Grey Divider

Sorry, something went wrong

We weren't able to complete the code review on our side. Please try again manually by commenting /agentic_review on this PR.

Grey Divider

Qodo Logo

@IsmaelMartinez IsmaelMartinez left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that is exactly it. Approving.

github_action_output is back above the guard, so $GITHUB_OUTPUT keeps the contract it had on main, and your new test pins it. I merged onto today's main and ran the suite: 91 passed, and 14 go red with pr_reviewer.py restored, so the coverage holds.

@IsmaelMartinez
IsmaelMartinez merged commit 9426bd6 into The-PR-Agent:main Sep 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

load_yaml returns None and nine call sites crash instead of degrading

2 participants