feat: a git change provider, and a git rung that reads its own repository

[behinddwalls]

Summary

Why?

Every change provider so far asks a service what a change contains. GitHub and Phabricator each have an API that knows; fake invents an answer; routing picks between the first two. A plain git remote has no such service, so there was no way to run the queue against one and have it know what a change touched.

That gap is what the demo's git rung was built around. The merge there is real — a real fetch, cherry-pick and push — and everything upstream of it was not. Because the fake provider cannot read a repository, make demo-requests wrote the paths it had just committed onto the change URI (sq-files=…) and the fake read them back, so the conflict analyzer had something to key on.

That worked for changes the demo created and for nothing else. A branch pushed by hand carried no marker, so as far as the analyzer could tell it touched nothing and conflicted with nothing — on the rung whose whole point is that the repository is real. It was also what pushed change URIs into the 255-byte storage limit and forced the paths to be budgeted down to one per directory.

What?

Four commits, each independently revertible:

refactor(test) — five test files each carried their own transcription of "resolve the Bazel-pinned git from runfiles", and they had drifted: two handled both spellings of an external path, three handled one; two verified what they resolved, one did not. Extracted to platform/gitexec/gitexectest before adding a sixth.

feat(changeprovider) — the provider. It keeps its own copy of a remote and computes each change from the commits: --numstat for files and line counts, the commit for the author.

The baseline chains through a stack, and that is the part worth reviewing. A git:// URI names a commit and a ref and nothing else — unlike a pull request it carries no base. It cannot be the target branch: a stack's changes are cut one from the next, so measuring each against the target reports the second change as containing the first, and anything summing line counts across a batch counts them twice. The first change is measured from where it diverged from the target, each one after it from its predecessor. This is wrong by default and fails silently, so it is the first thing the tests pin — and the assertion is mutation-tested.

Authentication is injected, never derived. The provider takes an Auth and calls it before each fetch; it never reads an environment variable or decides what a credential is. A nil Auth covers a local path and SSH served by the host's own config and agent.

feat(orchestrator) — changeProvider: {type: git, git: {…}}, and the default Auth implementation, which lives in the wiring layer where a deployment can replace it. Two queues may share a copy — that is how they share its lock — but sharing a path while disagreeing about the remote is rejected at startup.

Provisioning fetches. It runs at wiring time to fail a misconfigured service rather than bury the failure in a queue's retry loop, and writing the test showed that claim was not yet true: initializing a directory and recording a remote succeeds whether or not the remote exists.

feat(demo) — both git-rung queues switch to it, gitSource stops stating anything about what it touched, and the orchestrator image gains git and a writable directory, the same two things Runway's image needed for the same reasons.

Test Plan

• ✅ FOLDERS=1, six changes, no marker on any URI → a full dependency chain. With sq-files= gone, the only way pathoverlap could see a shared directory is from paths read out of the repository
• ✅ FOLDERS=50, five changes → no dependencies, landing in 3s rather than 10-14s, so it is keying on paths rather than serializing everything
• ✅ the case that was impossible before: two branches pushed by hand were described correctly — {"author": {"name": "Hand", …}, "changed_files": [{"path": "shared/a.txt", "lines_added": 1, …}]} straight out of the change table
• ✅ the container first, with the orchestrator still on fake: git present, /srv/git mounted, repos directory writable
• ✅ provider unit tests against real repositories: three-step stack (mutation-tested), multi-commit change, binary file, rename, unfetched commit, unrelated history, concurrent reads
• ✅ the wiring seam: configuration in, a provider out reporting what the repository says; and an unreachable remote failing startup
• ✅ make e2e-git-test with both queues switched; make test (106); integration suites; make lint, make gazelle

A risk flagged when planning did not materialise. TestLand_ResubmittedAfterLanding_IsRejectedAsStale asserts only that a resubmission errors, and the worry was it would now error inside the provider — the head branch has moved, so the original SHA is unreachable from the ref — and stay green while no longer testing staleness. Reproduced by hand: it still fails on the staleness check, because the copy keeps the object it fetched the first time.

Worth knowing what the E2E does not buy: e2e-git-queue uses analyzer: {type: none}, so nothing there consumes the provider's output. Switching it proves the provider runs end to end without breaking a land; the metadata checks are the FOLDERS runs above, hand-observed rather than asserted in CI.

Local note: make e2e-git-test and the integration suites need --sandbox_writable_path=$HOME/.docker on macOS, or they fail on open ~/.docker/buildx/activity/…: operation not permitted before any test runs.

Note for reviewers

The config block is nested under git: rather than flat as the plan sketched. Flat would have matched merge.yaml more literally, but this file's grammar is a named block per provider, and consistency inside the file being edited seemed worth more than symmetry with another service's file.

GitHub mode deliberately stays on the GitHub API provider: it reads pull request identity a git remote cannot give. So github and git answer "what changed" by different routes.