fix: give a fetch that reaches the network more than one go

[lens0021]

A bake fetches third-party extensions and skins over the network, and the image build downloads three tarballs, and every one of those gave up on its first refusal. That is a build failed by somebody else having a bad minute rather than by anything in the source.

It happened twice while this was being written. GitHub answered 500 for git and archive traffic for hours on 2026-08-17, which killed a local bake mid-clone:

Wikven: cloning extension 'MobileFrontend' from https://github.com/wikimedia/mediawiki-extensions-MobileFrontend.git
error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
fatal: expected 'packfile'

and codeload answered 429, which killed the phan job on #466 and stopped an unrelated composer install here six times running. Neither says anything about the commit being built.

The bake

Attempts holds the policy: three tries, waiting 2s then 4s. Doubling rather than a flat wait because the two failures worth retrying want opposite things. A 500 wants a moment; a 429 wants to be asked less often, and asking again at the same rate is what keeps it a 429.

Only the fetching retries. git init, the checkout and the extraction work on what is already local, so one failure from those is real and still ends the build at once.

Measured against a repository that does not exist, both shapes of git fetch:

Wikven: failed to clone extension 'Nowhere' (exit 128); trying again in 2s (attempt 2 of 3)
Wikven: failed to clone extension 'Nowhere' (exit 128); trying again in 4s (attempt 3 of 3)
Wikven: failed to clone extension 'Nowhere' (exit 128).

9s where it was 1s, still non-zero at the end. A failure that is going to fail is only slower; a blip now passes.

Two things worth calling out

git clone will not run twice over its own leftovers. It refuses a destination with anything in it, and a clone that died partway leaves one, so the clone path clears the debris between attempts. Verified rather than assumed: all three attempts fail with git's own could not read Username, never with destination path already exists, which is the error the second attempt would give if the first one's leftovers were still there.

There is no autoloader this early. fetchExtensions.php runs before MediaWiki loads the extensions, that being what it is for, which is why loadConfig() already require_onces SiteConfig directly. The first version of this crashed with Class "MediaWiki\Extension\Wikven\Attempts" not found on every bake that declares a repository. Found by running it, not by reading it.

One loop, not two

RetryingForeignRepo has had a retry loop of its own since it was written, and the two disagreed about the thing they share. It handed its pause callback a number of seconds and waited 1s then 2s; Attempts hands its callback the number of the attempt to come and waits 2s then 4s. The same "three attempts" meant three seconds of waiting in one place and six in the other, and the schedule was written down twice.

The difference that seemed to justify two loops — one repeats a request answering with a body or with false, the other repeats work that either worked or did not — is one contract, not two, once until() answers with what the attempt answered instead of a bool. A body comes back to the caller that wants a body; true comes back to the caller that only wants to know.

False is the failure and only false. A request can succeed with an empty body, which a truthiness test would have retried and then reported as a failure. RetryingForeignRepoTest covered exactly that case, and it moves to AttemptsTest along with the loop it tests; that file held nothing else, so it goes.

Commons lookups now back off 2s and 4s where they backed off 1s and 2s. That is the schedule the fetching already used, and a repository answering 429 is the case for asking less often rather than more.

Nothing off the shelf does this

Recorded in the class comment so the next reader does not have to check again:

• MediaWiki core does not retry. HttpRequestFactory and MWHttpRequest contain no retry code at all.
• wikimedia/wait-condition-loop, which core bundles, spends a time budget waiting for a condition to come true. One fetch here is a composer update that can take minutes, which is not that shape.
• Guzzle's retry middleware, which core also bundles, would reach the HTTP caller — and HttpRequestFactory does not expose the handler stack, so using it means building a client by hand and re-doing core's proxy, CA and user-agent setup. It would not reach the other caller at all, which repeats a subprocess.

The image

curl --retry 5 --retry-delay 2 --retry-all-errors, in an ARG so the three fetches share one policy. --retry-all-errors as well as --retry, because curl files a connection reset under non-transient and would not repeat it otherwise.

Confirmed in the image: the same doomed request takes 2s bare and 13s with the flags, so the attempts are real. Its curl is 8.21.0, well past the 7.71 that added the flag.

The SifterSearch tarball is downloaded before it is extracted now, instead of piped into tar. That is not tidying. A retried transfer starts over, and tar on the far end of the pipe has already been fed the first attempt's bytes. The block below it already avoids piping and says why.

Left alone

binary.Dockerfile. Its fetching happens inside build-static.sh, which the FrankenPHP image owns, and wrapping an eight-minute build in a retry loop is a different question. #433 died that way, on dl.static-php.dev answering 500, and deserves its own look.

The tests

Attempts::until never sleeps: the pause is the caller's hook, so the tests record which attempts were waited for rather than spending the seconds. That is the seam RetryingForeignRepo kept for the same reason, and it is why the suite covering a backoff policy costs nothing to run.

AttemptsTest covers the loop from both sides — a bool caller and a value caller — including the empty answer that must not count as a failure, the wait that arrives before each retry and not before the first, and the caller who asks for no attempts and still gets one. phpunit (REL1_46), phpunit (master) and coverage all run it green on this branch.

Refs #461.