CI takes nine minutes on a pull request; binary and smoke are most of it

[lens0021]

A pull request takes about nine minutes to go green. Measured on #458, which touched Dockerfile, includes/, extension.json and smoke.yml:

	duration
wall clock, first job start to last job finish	535s
binary (x86_64)	467s
smoke	321s
preview	180s
coverage	172s
phan (master) / phan (REL1_46)	126s / 121s
docker-image	52s
everything else	≤ 36s each

Two jobs set the pace. Nothing below is a decision yet — the point is that the levers are now measured rather than guessed at.

binary is the long pole, at 467s

It runs on a pull request touching Dockerfile, .dockerignore, binary.Dockerfile, bin/build.php, caddy/** or its own workflow — so any image change pays eight minutes, single-arch. Worth asking what that check is buying on a pull request that a release build would not catch, and whether it belongs on the critical path or on a nightly. It is also the job that has already failed once for reasons outside the repository (dl.static-php.dev answering 500), which is its own argument for not gating on it.

smoke bakes twice, sequentially: 209s of its 321s

Step timings:

 36s  Build and load the wikven image
105s  Bake the docs site
  3s  Assert the bake produced a complete, self-contained site
104s  Bake the docs site again and assert the two bakes are identical
 33s  Install the browser test tooling
 24s  Assert the rendered site in a browser

The second bake is #411's reproducibility check and the two are independent — separate containers, separate output directories — so they could run at once and the diff follow. That would cut roughly 100s if the runner has the cores for it: #437 made the skin passes concurrent, and BuildConcurrency sizes itself from /proc/cpuinfo and the cgroup rather than from current load, so two bakes would each claim the whole machine and contend. It might come out slower. It needs measuring on a real runner, not reasoning.

npx playwright install --with-deps chromium is 33s of the same job and is cacheable (~/.cache/ms-playwright, keyed on the lockfile).

Six workflows build the image

docker-image.yml, pr-preview.yml, deploy-docs.yml, smoke.yml, binary.yml and translations.yml all build it. That is already thought about — smoke.yml says only docker-image exports the layer cache and the others read it, since concurrent mode=max writers overwrite each other — and smoke's own build taking 36s suggests the cache is being hit. Worth confirming the other readers hit it too rather than assuming.

Done when

A pull request that changes only PHP under includes/ goes green in noticeably less than nine minutes, without giving up a check that has ever caught something.

🤖 Generated with Claude Code

[lens0021]

Measurements that change what this issue is asking for, plus what is now open against it.

The nine minutes is not one problem, it is two

#458, which this issue measured, touched Dockerfile, so binary ran. binary is path-filtered and does not run on a pull request that only touches PHP. #466 was exactly that, and every one of its 21 checks started within two seconds of the others:

317s  smoke
186s  coverage
145s  phan (master)
143s  phan (REL1_46)
 52s  phpunit (REL1_46)
 43s  docker-image
 43s  phpunit (master)
 31s  phpcs

So a PHP-only pull request already goes green in about five and a half minutes, and smoke is the whole of its critical path. The nine minutes belongs to pull requests that touch the image, where binary at 467s is the long pole. Two problems, two levers, and the "done when" above is about the first one.

(#466's raw wall clock reads 692s, which is not a measurement of anything: the GitHub incident of 2026-08-17 killed its phan job on a codeload 429 and the rerun landed three minutes after the rest.)

The two bakes: measured, and the barrier the issue guessed at is not there

The issue was right to say this needed measuring on a real runner rather than reasoning, and right that BuildConcurrency sizes itself from the machine. But contention turns out not to eat the win, because most of a bake is a single process: only the skin passes run beside each other. Two containers pinned to four cores, the runner's count:

	wall clock
one bake alone	118s
two, one after the other	257s
two at once, four cores between them	150s
two at once, two cores each	166s

Peak combined resident memory 1 GiB against the runner's 16, no OOM, and byte-identical output in every configuration. Capping each bake at half the runner is slower than letting both size themselves, because the cap also halves the concurrency of the part that does parallelise.

#470 does this.

Playwright: cacheable, but the cache is not allowed

~/.cache/ms-playwright is indeed cacheable, and zizmor refuses it. actions/cache in a job that also runs docker/build-push-action is its cache-poisoning audit at high severity, which is the same objection smoke.yml already records against setup-node's caching.

Fetching less needs no exception: playwright install chromium pulls Chrome for Testing (379 MiB) alongside the headless shell (262 MiB), and these specs only ever start the shell. Asking for chromium-headless-shell drops the 379 MiB. Verified by running the suite with nothing else on disk: 57 passed.

#471 does this.

Together those two take smoke from 321s to roughly 190s, which should put a PHP-only pull request near three and a half minutes.

Something the issue did not have on its list

Every bake clones its third-party skins and extensions at run time, and every one of those fetches gave up on the first refusal. That is not a time lever, it is a flakiness one, and it is the same class as the dl.static-php.dev 500 this issue already notes: the GitHub incident today killed a local bake mid-clone and a phan job on a 429. #469 gives those fetches three tries with a doubling wait.

Still open, and needing a decision rather than a measurement

binary at 467s on image-touching pull requests. The question the issue asks -- what that check buys on a pull request that a release build would not catch -- is a judgement call, not something to measure, so it is left here rather than guessed at in a PR.

Its fetching cannot be retried the way #469 retries ours: it happens inside build-static.sh, which the FrankenPHP image owns, and a retry there means re-running an eight-minute build.

[lens0021]

Re-measured on #473, which changed PHP under includes/, maintenance/ and docs/ — the shape of pull request the "Done when" is written about.

	#458 (this issue)	#473
wall clock, first job start to last job finish	535s	231s
smoke	321s	229s
preview	180s	189s
coverage	172s	148s
phan (master) / phan (REL1_46)	126s / 121s	138s / 121s
docker-image	52s	55s
phpunit (REL1_46) / phpunit (master)	≤ 36s	48s / 42s
binary (x86_64)	467s	did not run

The two rows that moved are smoke and binary, and only one of them moved because of work.

smoke: 321s → 229s

 36s  Build and load the wikven image
133s  Bake the docs site twice        <- was 209s, two bakes one after the other
  4s  Assert the bake produced a complete, self-contained site
  0s  Assert the two bakes are identical
 10s  Install the browser test tooling  <- was 33s
 25s  Assert the rendered site in a browser

The concurrent bake was worth measuring (#470). This issue said it might come out slower, since BuildConcurrency sizes itself from the machine rather than from current load and two bakes would each claim it. On the runner's four cores: 257s sequential against 150s together, output byte-identical either way, about 1 GiB at peak against the runner's 16. Most of a bake is a single process — only the skin passes run beside each other — so the second bake fills cores the first leaves idle.

Playwright caching turned out to be the wrong lever (#471). Rather than cache ~/.cache/ms-playwright, the install stopped needing a cache: --with-deps was running apt-get for browser libraries the runner image already ships, and chromium was pulling a full browser where chromium-headless-shell is what the tests drive. 33s → 10s with nothing to key or restore. That step is also what cost a six-hour job the day the runner's Ubuntu mirror went quiet mid-apt-get, which is a second reason it is better gone than cached.

binary: unchanged, and off this pull request's path

467s then, 504s on its last run (32203489763). Nothing was done to it. It did not run on #473 only because #473 touched none of its paths.

The other question this issue asked

Do the other image builders hit the layer cache? They do, confirmed rather than assumed — on #473: preview built in 34s, smoke in 36s, and check-translations finished the whole job in 59s. Only docker-image exports, and the readers get the layers.

Done when

A pull request that changes only PHP under includes/ goes green in noticeably less than nine minutes, without giving up a check that has ever caught something.

Met: 3m51s, and no check was dropped or weakened.

For that shape of pull request there is no meaningful headroom left, either. Wall clock is smoke exactly, and smoke is now 133s of baking plus ~96s of everything else; preview at 189s sits under it, so making preview faster would not move the number at all. What remains is the bake itself, which is wikven doing its job rather than CI overhead.

Closing this. What is left is the question this issue raised but could not answer with a measurement — whether binary belongs on a pull request at all — and that is a decision, not a timing. Filed separately.

---

Generated by Claude Code