Skip to content

[Execute Infrastructure] ci: fix approval reminder wording & drop pull_request_review trigger - #1463

Merged
swgu98 merged 3 commits into
PaddlePaddle:developfrom
liuhao2638:ci/approval-reminder-or-fix
Jul 23, 2026
Merged

[Execute Infrastructure] ci: fix approval reminder wording & drop pull_request_review trigger#1463
swgu98 merged 3 commits into
PaddlePaddle:developfrom
liuhao2638:ci/approval-reminder-or-fix

Conversation

@liuhao2638

@liuhao2638 liuhao2638 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

PR Category

Execute Infrastructure

PR Types

Devs

Description

承接 #1460,对已合入的 bot-approval-required.yml 按 Paddle 的同样形式继续调整,并与 Paddle / PaddleFormers 的门禁保持一致:

  1. 检查流程:新增 check-bypass job 复用 ./.github/workflows/check-bypass.yml,支持 skip-ci label / 评论豁免;check-bot-approval 依赖 check-bypass,当 can-skip != 'true' 时才执行审批检查。
  2. 审批逻辑:仅统计当前 head commit 上的评审。AI review 机器人 risemeup1111 approve,或 sneaxiy / From00 名单中任一账号 approve 即可通过。
  3. 触发器:仅保留 pull_request: [opened, synchronize, reopened],不再使用 pull_request_review

是否引起精度变化

否。本 PR 仅修改 GitHub Actions CI workflow,不涉及框架代码,不引起任何精度变化。

PaddlePaddle-bot

This comment was marked as outdated.

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查,未发现需要阻塞合入的问题;这次提醒文案与当前门禁逻辑一致。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

@liuhao2638 liuhao2638 changed the title [Execute Infrastructure] ci: fix approval reminder wording (sneaxiy/From00 或) [Execute Infrastructure] ci: fix approval reminder wording & drop pull_request_review trigger Jul 14, 2026

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Paddle-CI-Agent | pr_review | 2026-07-14 20:51:46

📋 Review 摘要

PR 概述:调整 bot approval 门禁的提示文案,并移除 pull_request_review 触发器。
变更范围.github/workflows/bot-approval-required.yml
影响面 TagCI

问题

级别 文件 概述
🔴 Bug .github/workflows/bot-approval-required.yml:23 审批提交后不再触发门禁复检,required check 会停留在失败状态

📝 PR 规范检查

符合规范。

总体评价

提示文案从“和”改为“或”与脚本里的 || 逻辑一致;但移除 pull_request_review 会破坏审批后自动放行路径,需要恢复或提供等价复检触发。

# 评审后推送新 commit(synchronize)或重开 PR 时自动复检;旧 commit 上的审批在 push 新提交后自动失效。
on:
pull_request:
types: [opened, synchronize, reopened]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Bug 删除 pull_request_review 后,审批提交不会触发门禁复检。

这个 job 只在 pull_requestopened/synchronize/reopened 事件上读取 reviews。若 PR 打开或推送时还没有 risemeup1111sneaxiyFrom00 对当前 head 的 APPROVED,本检查会失败;随后任一账号在同一 head 上 approve 时,不会再有事件重跑 latest_state(),required check 仍保持失败,提示中的“联系 ... 进行 approve”也无法自动放行。

建议恢复 review 事件(至少 submitted,并保留 dismissed 以便撤销审批后复检):

on:
  pull_request:
    types: [opened, synchronize, reopened]
  pull_request_review:
    types: [submitted, edited, dismissed]

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查新提交。此前 Nyanpasu 没有未解决的 inline 线程;本次新提交移除了 review 事件触发器,我在变更行留了一个 P1 评论,需要修复后再复查。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment on lines 20 to 23
# 评审后推送新 commit(synchronize)或重开 PR 时自动复检;旧 commit 上的审批在 push 新提交后自动失效。
on:
pull_request:
types: [opened, synchronize, reopened]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

移除 pull_request_review 触发后,这个门禁在当前 head 先失败时,后续 risemeup1111 / sneaxiy / From00 提交或撤销 review 都不会自动触发复检。结果是:approve 后失败的 Require review-bot approval 可能一直停留到作者再推新提交或人工重跑;反过来,如果已有成功 run,审批被撤销后也不会自动回到失败状态,门禁语义会失真。请保留 review 事件触发器,或提供等价的自动复检机制;当前改动可以直接恢复为:

Suggested change
# 评审后推送新 commit(synchronize)或重开 PR 时自动复检;旧 commit 上的审批在 push 新提交后自动失效。
on:
pull_request:
types: [opened, synchronize, reopened]
# 评审提交/修改/撤销或推送新 commit 时自动复检;旧 commit 上的审批在 push 新提交后自动失效。
on:
pull_request:
types: [opened, synchronize, reopened]
pull_request_review:
types: [submitted, edited, dismissed]

处理要求:请针对该评论修复并提交新的 commit。

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

复查当前 head 后,这个问题仍未解决。新提交新增的 check-bypass 只是一个 workflow_call 里的跳过判定,只有在本 workflow 已经被某个事件触发后才会执行;但当前 on: 仍只包含 pull_request: [opened, synchronize, reopened]。因此审批提交、审批撤销,或 PR 已失败后再加 skip-ci label / 评论,都不会自动产生本 workflow 的新 run,Require review-bot approval 仍可能停留在旧状态。

请恢复 review 状态变化触发,或提供等价的自动复检事件源,例如:

on:
  pull_request:
    types: [opened, synchronize, reopened]
  pull_request_review:
    types: [submitted, edited, dismissed]

处理要求:请针对该评论修复并提交新的 commit。

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查新提交。上一轮 P1 仍未解决:新增的 check-bypass 不能替代 review 提交/撤销时的自动复检触发;我已在原线程补充当前证据,未新增重复 inline 评论。请修复后再复查。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

@risemeup1 risemeup1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Paddle-CI-Bot

Copy link
Copy Markdown

PaddleFleet Log Analysis

Run #29912281756 · Attempt 1

日志分析报告

流水线名称 问题标签 修复建议 日志片段
Integration test (H20, multi-card) — Qwen vl moe CUDA error(719) / kernel launch failure in get_rope_index 排查 modeling_fleet.py:1163get_rope_indexmrope_position_deltas 中存在无效 GPU tensor 被 .numpy() 触发 kernel,加 .cpu() 转换后再传入 paddle.to_tensor 报错代码

失败的测试case:

Integration test (H20, multi-card) > Qwen vl moe
  - qwen3vl_sft_moe (config: tests/config/ci/qwen3vl_sft_moe.yaml, 8卡 MoE SFT)
    Container rank 2 exited with code -6 (SIGABRT)
    OSError: (External) CUDA error(719), unspecified launch failure
    at: paddleformers/datasets/collate.py:771 mm_collate_fn
        -> qwen3_vl/modeling_fleet.py:1163 get_rope_index
        -> paddle_patch.py:98 enhance_to_tensor -> paddle.to_tensor -> np.array(data)
        -> paddle_patch.py:137 _numpy -> origin_numpy -> CUDA error(719)

根本原因分析:

PR #1463 仅修改了 .github/workflows/bot-approval-required.yml(CI 基础设施),与此次失败无直接关联。失败根因在于 get_rope_indexmodeling_fleet.py:1163)在 MoE SFT 数据 collate 阶段,将一个仍在 GPU 上的 tensor mrope_position_deltas 直接传给 paddle.to_tensor(),内部走 np.array(data) 路径触发 .numpy() 调用,在 MoE 多卡场景下某 rank 的 CUDA kernel 先行失败(unspecified launch failure, code 719),导致后续 .numpy() 时触发 cudaErrorLaunchFailure,rank 2 以 SIGABRT 退出,整个 pod 失败。

修复建议:

  1. 定位 PaddleFormers/paddleformers/transformers/qwen3_vl/modeling_fleet.py:1163

    # 当前
    mrope_position_deltas = paddle.to_tensor(mrope_position_deltas).unsqueeze(1)
    
    # 修复:若 mrope_position_deltas 已是 Tensor,先移到 CPU 再转换,避免在 GPU tensor 上触发隐式 numpy
    if isinstance(mrope_position_deltas, paddle.Tensor):
        mrope_position_deltas = mrope_position_deltas.cpu().tolist()
    mrope_position_deltas = paddle.to_tensor(mrope_position_deltas, place=paddle.CPUPlace()).unsqueeze(1)
  2. 同步检查 collate.py:771 mm_collate_fnget_rope_func 的调用路径,确保传入前所有中间 tensor 不残留在 GPU 上。

  3. 重点在 MoE 多卡(8卡以上)场景复现:单卡不一定触发,因为 MoE expert routing 在多卡下会有跨 rank 的额外 CUDA kernel 调度,更易暴露 cudaErrorLaunchFailure


🔍 准确性记录:请点击评论底部 😊 图标,选择 👍(准确)或 👎(有误),将自动记录到 CI 监控系统

🔄 每次 Re-run 后自动更新

@swgu98
swgu98 merged commit ad6efc9 into PaddlePaddle:develop Jul 23, 2026
36 of 39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants