Skip to content

Change final review logic for PRs#3072

Draft
Phlip79 wants to merge 3 commits intoNVIDIA:mainfrom
Phlip79:philip/remove-final-for-low
Draft

Change final review logic for PRs#3072
Phlip79 wants to merge 3 commits intoNVIDIA:mainfrom
Phlip79:philip/remove-final-for-low

Conversation

@Phlip79
Copy link
Member

@Phlip79 Phlip79 commented Jan 25, 2026

What does this PR do ?

This PR changes a few things for review logic:

  • removes core-nemo and core-adlr as code owners for all code that has an expert group.
  • adds core-nemo as a required reviewer in a separate step to core and training code
  • adds core-adlr as a required reviewer in a separate step to core and training code if the PR is not labeled with complexity: low

Note: if core-adlr is still a code owner and the PR has label complexity: low, core-adlr will still be required to review.

Contribution process

flowchart LR
    A[Pre-checks] --> B[PR Tests]
    subgraph Code Review/Approval
        C1[Expert Review] --> C2[Final Review]
    end
    B --> C1
    C2 --> D[Merge]
Loading

Pre-checks

  • I want this PR in a versioned release and have added the appropriate Milestone (e.g., Core 0.8)
  • I have added relevant unit tests
  • I have added relevant functional tests
  • I have added proper typing to my code Typing guidelines
  • I have added relevant documentation
  • I have run the autoformatter.sh on my PR

Code review

The following process is enforced via the CODEOWNERS file for changes into megatron/core. For changes outside of megatron/core, it is up to the PR author whether or not to tag the Final Reviewer team.

For MRs into `main` branch

Feel free to message or comment the @mcore-oncall to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!

(Step 1): Add PR label Expert Review

(Step 2): Collect the expert reviewers reviews

  1. Attach the Expert Review label when your PR is ready for review.
  2. GitHub auto-assigns expert reviewers based on your changes. They will get notified and pick up your PR soon.

⚠️ Only proceed to the next step once all reviewers have approved, merge-conflict are resolved and the CI is passing.
Final Review might get declined if these requirements are not fulfilled.

(Step 3): Final Review

  1. Add Final Review label
  2. GitHub auto-assigns final reviewers based on your changes. They will get notified and pick up your PR soon.

(Optional Step 4): Cherry-pick into release branch

If this PR also needs to be merged into core_r* release branches, after this PR has been merged, select Cherry-pick to open a new PR into the release branch.

For MRs into `dev` branch The proposed review process for `dev` branch is under active discussion.

MRs are mergable after one approval by either eharper@nvidia.com or zijiey@nvidia.com.

Merging your PR

Any member of core-adlr and core-nemo will be able to merge your PR.

Summary by CodeRabbit

  • Chores
    • Reorganized code ownership structure to better align responsibility across different code modules.
    • Enhanced code review approval workflow to validate required team approvals and automatically request reviews based on change complexity levels.

@Phlip79 Phlip79 requested a review from a team as a code owner January 25, 2026 19:30
@copy-pr-bot
Copy link

copy-pr-bot bot commented Jan 25, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ko3n1g ko3n1g requested a review from a team January 25, 2026 19:30
@Phlip79 Phlip79 requested a review from ko3n1g January 25, 2026 19:30
@Phlip79
Copy link
Member Author

Phlip79 commented Jan 25, 2026

/ok to test eb6ddb9

@Phlip79 Phlip79 requested a review from ericharper January 25, 2026 19:34
@ko3n1g ko3n1g added this to the Core 0.16 milestone Jan 25, 2026
@Phlip79 Phlip79 changed the title Change final review logic for low complexity PRs Change final review logic for PRs Jan 25, 2026
@Phlip79
Copy link
Member Author

Phlip79 commented Jan 29, 2026

@greptile Please review

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@Phlip79 Phlip79 removed request for a team, ericharper and ko3n1g February 2, 2026 16:07
@Phlip79
Copy link
Member Author

Phlip79 commented Feb 2, 2026

@coderabbitai review

@coderabbitai
Copy link

coderabbitai bot commented Feb 2, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai
Copy link

coderabbitai bot commented Feb 2, 2026

Walkthrough

This pull request reassigns code ownership in CODEOWNERS to more specialized teams by removing @NVIDIA/core-adlr and @NVIDIA/core-nemo from multiple paths, and adds approval decision logic to the multi-approval-bot workflow for validating core changes.

Changes

Cohort / File(s) Summary
Code Ownership Reassignment
.github/CODEOWNERS
Consolidates ownership of multiple megatron/core paths to specialized teams: gpt, multi-modal, hybrid-mamba, datasets, megatron-fsdp, dist-checkpointing, dist-optimizer, quantization-and-inference, pipeline-parallelism, mixture-of-experts, inference, post-training, and cuda-graphs. Removes @NVIDIA/core-adlr and @NVIDIA/core-nemo from most affected paths.
Workflow Approval Logic
.github/workflows/multi-approval-bot.yml
Adds approval decision logic based on "complexity: low" label and detection of core changes. Introduces conditional steps to request reviews from required teams (core-nemo and conditionally core-adlr) and validate approvals from these teams for pull requests affecting megatron/core or megatron/training.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 The warren reorganizes its burrows today,
Each tunnel now claimed by its specialist way,
With approval gates standing for core changes bright,
The bot checks the teams—all done right!
Hopping forward with clarity, our code runs true! 🥕✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Change final review logic for PRs' accurately summarizes the main objective of the PR, which modifies the review and approval workflow for pull requests.
Description check ✅ Passed The PR description closely follows the template structure, includes the required 'What does this PR do?' section with clear explanations of changes, maintains the contribution process diagram, and includes all pre-checks and code review sections.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In @.github/workflows/multi-approval-bot.yml:
- Around line 83-93: The current APPROVERS extraction uses all historical
APPROVED reviews and can count stale approvals; change the gh api/jq pipeline
that sets APPROVERS so it groups reviews by .user.login and selects each
reviewer’s latest review (e.g., sort_by(.submitted_at) | last) and then filters
those latest reviews for state == "APPROVED"; update the variable referenced as
APPROVERS and keep the subsequent loop over MEMBERS and the grep check unchanged
so only reviewers whose latest review is APPROVED are considered.
- Around line 111-121: The APPROVERS extraction can include stale approvals
because it doesn't reduce multiple reviews per user to their latest state;
update the APPROVERS assignment (the gh api call that produces APPROVERS) to
sort reviews by submitted_at, group by .user.login, take the last review per
user, then select those with state == "APPROVED" and extract their login; this
ensures APPROVERS only contains users whose most recent review is APPROVED when
you later compare against MEMBERS in the for-loop.
🧹 Nitpick comments (1)
.github/workflows/multi-approval-bot.yml (1)

53-68: Silent failure handling may mask configuration issues.

The 2>/dev/null || true pattern ensures the workflow doesn't fail if review requests can't be made. While this is reasonable for a non-critical operation (requesting reviews), it could mask issues like:

  • Insufficient PAT permissions for team review requests
  • Team names that don't exist or have changed

Consider adding a warning log when the request fails to aid in debugging:

💡 Suggested improvement for better observability
          # Request core-nemo for core changes
-         gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/requested_reviewers \
-           --method POST -f team_reviewers[]="core-nemo" 2>/dev/null || true
+         gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/requested_reviewers \
+           --method POST -f team_reviewers[]="core-nemo" 2>/dev/null || echo "Warning: Could not request review from core-nemo"

          # Request core-adlr unless complexity:low
          if [ "$IS_LOW_COMPLEXITY" != "true" ]; then
            gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/requested_reviewers \
-             --method POST -f team_reviewers[]="core-adlr" 2>/dev/null || true
+             --method POST -f team_reviewers[]="core-adlr" 2>/dev/null || echo "Warning: Could not request review from core-adlr"
          fi

Comment on lines +83 to +93
# Get PR approvers
APPROVERS=$(gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews \
--jq '[.[] | select(.state == "APPROVED")] | [.[].user.login] | unique | .[]')

# Check if any team member approved
for member in $MEMBERS; do
if echo "$APPROVERS" | grep -q "^${member}$"; then
echo "✅ core-nemo approval found from: $member"
exit 0
fi
done
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Potential issue: Stale approvals may be counted.

The current logic fetches all reviews with state APPROVED and checks if any team member is in that list. However, GitHub's reviews API returns all historical reviews. If a reviewer first approves, then later submits a "Request Changes" review, the old approval will still be counted.

Consider filtering to only count the latest review state per reviewer:

🐛 Proposed fix to consider only the latest review per reviewer
          # Get PR approvers
-         APPROVERS=$(gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews \
-           --jq '[.[] | select(.state == "APPROVED")] | [.[].user.login] | unique | .[]')
+         # Get the latest review state per reviewer, then filter for APPROVED
+         APPROVERS=$(gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews \
+           --jq 'group_by(.user.login) | map(max_by(.submitted_at)) | [.[] | select(.state == "APPROVED")] | .[].user.login')
🤖 Prompt for AI Agents
In @.github/workflows/multi-approval-bot.yml around lines 83 - 93, The current
APPROVERS extraction uses all historical APPROVED reviews and can count stale
approvals; change the gh api/jq pipeline that sets APPROVERS so it groups
reviews by .user.login and selects each reviewer’s latest review (e.g.,
sort_by(.submitted_at) | last) and then filters those latest reviews for state
== "APPROVED"; update the variable referenced as APPROVERS and keep the
subsequent loop over MEMBERS and the grep check unchanged so only reviewers
whose latest review is APPROVED are considered.

Comment on lines +111 to +121
# Get PR approvers
APPROVERS=$(gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews \
--jq '[.[] | select(.state == "APPROVED")] | [.[].user.login] | unique | .[]')

# Check if any team member approved
for member in $MEMBERS; do
if echo "$APPROVERS" | grep -q "^${member}$"; then
echo "✅ core-adlr approval found from: $member"
exit 0
fi
done
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Same stale approval issue applies here.

This step has the same potential issue as the core-nemo check where historical approvals may be counted even if the reviewer later changed their review state. The same fix should be applied here.

🐛 Proposed fix for core-adlr approval check
          # Get PR approvers
-         APPROVERS=$(gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews \
-           --jq '[.[] | select(.state == "APPROVED")] | [.[].user.login] | unique | .[]')
+         # Get the latest review state per reviewer, then filter for APPROVED
+         APPROVERS=$(gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/reviews \
+           --jq 'group_by(.user.login) | map(max_by(.submitted_at)) | [.[] | select(.state == "APPROVED")] | .[].user.login')
🤖 Prompt for AI Agents
In @.github/workflows/multi-approval-bot.yml around lines 111 - 121, The
APPROVERS extraction can include stale approvals because it doesn't reduce
multiple reviews per user to their latest state; update the APPROVERS assignment
(the gh api call that produces APPROVERS) to sort reviews by submitted_at, group
by .user.login, take the last review per user, then select those with state ==
"APPROVED" and extract their login; this ensures APPROVERS only contains users
whose most recent review is APPROVED when you later compare against MEMBERS in
the for-loop.

@Phlip79 Phlip79 marked this pull request as draft February 10, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants