test: reuse published default sysroots in E2E - #200
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Review: prepare glibc sysroot with llar make
Clean, self-contained CI change. It replaces the pinned Bootlin toolchain download with llar make bminor/glibc@glibc-2.27, which — as the PR notes — reuses the published artifact through the normal Kodo cache path (verified: cmd/llar/internal/make.go wraps the build in a readThroughCache over the public Kodo store before any source build). Running llar make before the local url.insteadOf redirects are configured is therefore intentional: this step fetches the real published sysroot, while the redirects only apply to the later local-fetch E2E step. The tar packaging (-C .../glibc-sysroot .) also pairs correctly with the consumer's install-sysroot ... 0 (--strip-components=0), and install-bootlin-sysroot remains in use by crosscompile-e2e.yml, so it is correctly left in place.
Findings below are minor/advisory — nothing blocking.
| "https://toolchains.bootlin.com/downloads/releases/toolchains/aarch64/tarballs/aarch64--glibc--stable-2018.11-1.tar.bz2" \ | ||
| --output "$archive" | ||
| echo "abae0522480b9f37ff6cee4249e147e7cb78e1997cc6f76dba7e0fb8ec04221d $archive" | sha256sum --check | ||
| "$RUNNER_TEMP/llar" make bminor/glibc@glibc-2.27 \ |
There was a problem hiding this comment.
[P3] New step runs PR-controlled llar on the secret-bearing tailnet
This step replaces a hermetic, hash-pinned data download (curl + sha256sum --check) with executing the PR-built llar binary and its formula-driven logic. Because the workflow is pull_request_target with allow-unsafe-pr-checkout: true, and this step runs after the Tailscale OAuth secrets have already joined the runner to the private tailnet (and before the QINIU_* secrets are used later), a PR author's code now executes inside a secret-bearing job — a net increase in exposure versus the data-only step it replaces.
Note this is bounded by the existing model: for PR checks pull_request_target runs the base branch's workflow (as the PR description acknowledges), so a fork PR cannot alter this step until it lands on the base branch. Still, if hardening is desired, consider moving the build + llar make steps into a separate job with no secrets/tailnet access and passing only the produced sysroot artifact to the secret-bearing job.
| --arch arm64 \ | ||
| --output "$RUNNER_TEMP/glibc-sysroot" \ | ||
| --verbose | ||
| tar -cJf "$RUNNER_TEMP/glibc-sysroot.tar.xz" -C "$RUNNER_TEMP/glibc-sysroot" . |
There was a problem hiding this comment.
[P3] Optional: sanity-check the sysroot before packaging
llar make failing is caught by the shell's default -e, but a run that silently produces an empty or partial tree would still tar successfully and only surface as a confusing failure much later in the llard build. A one-line guard after generation (e.g. test -d "$RUNNER_TEMP/glibc-sysroot/usr") would fail fast with a clear signal. Advisory only.
Use
llar maketo resolve published default sysroots in both the CLI cross-compilation E2E and the LLARD cluster E2E. This removes manual preparation of the default glibc and macOS SDK and avoids packing Bootlin runtime links in the cluster test.The implementation includes:
llar maketo resolve default sysroots from the real Formula store and public cache.llar make --jsonfor LLARD and uses the existing Kodo cache API to seed the run's isolated prefix once before the workers become ready. Removes the cluster's tar and fake glibc source setup.libcmatrix dimension, currently return 404, so those cases still use the existing Bootlin fixture installer.Local validation passed: workflow YAML/shell checks, crosscompile and CLI tests, race-enabled cache/artifact tests, seed-command vetting, and E2E harness compilation. In a Linux amd64 container, real
llar makepopulated the glibc workspace cache; repeated output matched and the seed command successfully packed the result. Real Kodo upload was not run locally because CI credentials are unavailable.The ordinary cross-compilation E2E run passed on commit
f77f4ba: both Linux-host builds, all default/custom-libc and pkg-config checks, and native Linux arm64 and macOS arm64 consumers succeeded. The cluster workflow usespull_request_target, so its updated preparation steps must be exercised after the workflow is on the base branch.Default sysroot handling now follows the CLI's existing cache mechanism while custom-libc coverage and the cluster's isolated cache assertions remain intact.