ci(workflows): make apt fetches fail fast and retry instead of hanging - #507
Merged
Milhouszhang merged 1 commit intoJul 9, 2026
Merged
Conversation
The Ubuntu mirror intermittently accepts a connection and then stops sending data. apt has no read timeout by default, so it blocks forever: run 29012965487 stalled 8.5 min on the 25 MB libwebkit2gtk-4.1-0 fetch and the Desktop backend job was killed by its 20 min timeout-minutes without compiling a single line of Rust. Set a 30s read timeout so a stalled transfer aborts, and 3 retries so apt re-fetches (picking another mirror) instead of failing the job. Applied to both apt-using jobs via a shared env var. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR #506 的 CI 挂在
Desktop backend (test + clippy)上,一行 Rust 都没编译就被杀了。根因
Ubuntu 镜像偶发「接受连接但不再发数据」。apt 默认既没有读超时,也不重试,于是整个
apt-get install无限期挂起,直到 job 的timeout-minutes把它杀掉。run 29012965487 的实测时间线:
Install system dependencies开始,apt-get update正常(10.2 MB / 2s)libwebkit2gtk-4.1-0(25.1 MB)##[error]The operation was canceled.← job 启动后正好 20 分钟后续
rust-toolchain/nextest/rustfmt/clippy全部 skipped。同一 run 里Rust (build + test)的 apt 步骤 6 秒跑完,说明不是网络整体故障,是那个镜像在大包上抽风。修法
调大
timeout-minutes不是解 —— 那只会让它卡得更久。真正要做的是让停滞的连接自己失败,然后重试:Acquire::http::Timeout=30—— 30 秒收不到数据就断开,把「无限挂起」变成「一次失败」Acquire::Retries=3—— 失败后自动重试(会换镜像)两个用到 apt 的 job(
rust、desktop-backend)共用一个APT_RESILIENCEenv var。不引入任何第三方 action。备选方案(本 PR 未采用)
用
awalsh128/cache-apt-pkgs-action缓存 desktop-backend 那 79 个包,提速明显,但引入第三方依赖 + 缓存一致性风险。跟「修 flake」不是一个关注点,留作后续。验证
APT_RESILIENCE在 GHA 默认的bash -e下正确词分割成-o Acquire::http::Timeout=30 -o Acquire::Retries=3四个参数(本 PR 改的两个 job 都没有shell:覆盖,只有 desktop/playwright job 有defaults.run.working-directory)。🤖 Generated with Claude Code