feat(demo): one quickstart, three provider modes

## Summary

### Why?

The repository had two divergent local paths, and the quickstart documented the wrong one.

`make local-submitqueue-start` brought up an all-fake stack — fake change provider, fake CI, **noop merger** — and `doc/howto/QUICKSTART.md` (#591) described that. It reached `landed` in seconds with no credentials, but nothing was ever pushed, so it proved the pipeline's choreography and nothing about merging. `make local-provider-start` ran the real thing, but was GitHub-only: it hardcoded the token-requiring overlay, so the credential-free provider configuration that already existed could not be reached by hand at all. `PROVIDER=local` failed at `docker compose up` on a missing `GITHUB_TOKEN`, and would have merged nothing past that, since the overlay it needs bind-mounts a repository the other one has never heard of.

Those two are not variants of one thing. Faking every edge and merging into a real repository answer different questions, and calling both "local" is what made them hard to keep apart.

Three further confusions came out of reading the result back:

- **Two stack families did the same thing.** Once `fake` was a first-class provider, `local-submitqueue-start` and `local-provider-start` were the same run under *different compose project names* — which had already produced a wrong instruction, since `make local-submitqueue-ps` inspects project `submitqueue` and reports "not running" for a stack started as `submitqueue-provider`.
- **One document mixed two audiences.** `PROVIDER-E2E.md` opened with a tier table pairing bazel test targets with hand-run demo commands, as though `make e2e-test` and "open a pull request and watch it land" were rungs of one ladder. Automated suites belong to testing, which `TESTING.md` already documents.
- **`demo-pr` was named for something two of the three modes do not have.** A fake change is a URI and a git change is a branch; neither is a pull request.

### What?

`PROVIDER` now names three modes, and is the only thing that changes between them:

| `PROVIDER` | A change is | Landing it | Needs |
|---|---|---|---|
| `fake` (default) | a URI, and nothing else | reports success without touching a repository | nothing |
| `git` | a branch in a bare repository on disk | a real fetch, cherry-pick and push | nothing |
| `github` | a real pull request | a real push to a real repository | a repository and a token |

**Provider directories.** `demo/provider/local/` becomes `demo/provider/git/` — both old names were "local" — and gains a `demo-queue` entry so the default `QUEUE` works there. It needs its own `checkoutPath`: Runway refuses two queues that share a checkout with differing merger configuration, and `demo-queue` squash-rebases where `e2e-git-queue` rebases. A new `demo/provider/fake/` spells out the all-fake mode rather than leaving it implied by absent configuration. `e2e-git-queue` is untouched, and `make e2e-git-test` is the gate that proves it.

**One stack target.** `local-submitqueue-start` takes `PROVIDER` and layers that mode's overlay; `local-provider-start`, `-stop` and `-clean` are gone. One compose project again, so `ps`, `logs`, `stop` and `clean` all describe whatever is running. Which overlay a mode needs is a Makefile map, since it is not something the two config files can express — `github` requires a credential, `git` requires the sandbox mounted, `fake` neither — and a new `docker-compose.fake.yml` covers the third.

**One demo document.** `PROVIDER-E2E.md` is merged into `QUICKSTART.md`, which walks all three rungs in order and carries what only that document had: the token permission table, the GitHub configuration, why a landed pull request shows as *Merged*, wiring real GitHub Actions, and the GitHub-specific failure modes. Its provider-neutral operational material — `land-list` / `land-watch`, `SQ_TOKEN`, service logs, `QUEUE_LOG_LEVEL` — moves with it, into the document people read first. No test target is mentioned in any of it.

**`make demo-requests`** (was `demo-pr`, in `service/submitqueue/demo/requests`) works without GitHub. `createOne` was the only function that touched the provider; it now describes a change and hands it to a `changeSource`. Three implement it: `fakeSource` (no I/O at all — mints a reproducible `git://` URI), `gitSource` (pushes real branches with the pinned `@git//:git`, serializing its commands because one working tree cannot take concurrent checkouts), and `githubSource` (the REST client, extracted unchanged). `GITHUB_TOKEN` is read only when it is needed.

**Folder overlap, made real.** A change writes all its files into one of three folders (`demo/alpha`, `demo/beta`, `demo/gamma`) chosen from the run tag, and states the paths it touched on its change URI (`sq-files=`, beside the existing `sq-fake=`) for the fake change provider to report back. All three profiles then use `pathoverlap` by directory, so a run shows both halves of the queue's behaviour: changes sharing a folder are batched in order and speculate on each other, changes in different folders go out beside each other. The previous layout could not do this — it hashed each file's own name into two levels of 256 buckets, so a twelve-file run had roughly a one-in-a-thousand chance of any two files sharing a directory. The change is the bucket count and the granularity, not the naming.

**`tool/gitsandbox`** provisions the bare repository `PROVIDER=git` merges into, idempotently, so a restart keeps whatever landed. **`platform/gitexec`** locates git and strips the ambient environment, so a developer's hooks or signing key cannot fail a demo.

Three fixes fell out of getting this to work at all:

- Runway's checkout directory moves to a named volume unless a path is given. It is where git clones, cherry-picks and commits, and on macOS a freshly written loose object read back over a bind mount can report `loose object … is corrupt` — which failed the first land against every new stack. The E2E still passes a path and still gets a bind mount.
- The Runway image now creates `/var/runway/checkouts`, so a named volume mounted there starts with a mode the service can write. Without it the service failed at boot with `mkdir: permission denied` whenever it ran as a non-root user.
- Change URIs are storage keys capped at 255 bytes, so the file marker names one path per directory rather than every file — for a directory-keyed analyzer that is the same set of keys — and drops paths that would not fit rather than truncating one into a different directory.

## Test Plan

Every command in the rewritten quickstart was run by hand against a live stack, and the output quoted in it is what it printed.

- ✅ `make local-submitqueue-start` (defaults to `fake`) then `make demo-requests COUNT=6` — all six landed
- ✅ inspected `batch_dependent` for that run: the four `gamma` changes form one dependency chain, the two `beta` changes form their own, and **no dependency crosses folders**. Batches from before this change show the old `all` behaviour, every batch depending on every later one
- ✅ `PROVIDER=git make local-submitqueue-start` from a clean slate, then `PROVIDER=git make demo-requests` — real branches and multi-file commits, verified with `git -C /tmp/sq-sandbox/sandbox.git log --oneline main`
- ✅ `PROVIDER=git make demo-requests STACKED=true` — three changes, exactly one more entry in the target's reflog, so the stack landed atomically
- ✅ first land against a brand-new stack, repeatedly, which is the case the named volume fixes
- ✅ the 255-byte limit is a real one this hit first: an eight-file change was rejected with `change URI exceeds 255 bytes` before the marker was budgeted. Covered now by a unit test that builds a URI from `changeFilePath` output and asserts both the length and that it still parses
- ✅ `make test` (105 targets), `make lint`, `make check-gazelle`, `make check-tidy`
- ✅ `make e2e-test` and `make e2e-git-test`, after the rename and again after the compose change

`PROVIDER=github` is unchanged in behaviour but needs a token, so it has not been re-run; the GitHub source is the previous code path moved behind the interface.

## Note for reviewers

`make local-provider-start` is gone — use `make local-submitqueue-start`, which with no arguments now means `fake` where the old provider target meant GitHub.

Author: behinddwalls

Makefile

@@ -6,7 +6,6 @@ COMPOSE = docker-compose
 
 # SubmitQueue compose files
 COMPOSE_FILE = service/submitqueue/docker-compose.yml
-PROVIDER_COMPOSE_FILE = service/submitqueue/docker-compose.provider.yml
 GATEWAY_COMPOSE_FILE = service/submitqueue/gateway/server/docker-compose.yml
 ORCHESTRATOR_COMPOSE_FILE = service/submitqueue/orchestrator/server/docker-compose.yml
 
@@ -46,13 +45,34 @@ PROTO_PACKAGES = api/base/change api/base/mergestrategy api/base/messagequeue ap
 # Set REPO_ROOT for docker-compose
 export REPO_ROOT := $(shell pwd)
 
-# Which provider the demo stack targets. Selects a configuration directory rather
-# than a code path, so adding a provider means adding a directory — see
+# Which provider the demo stack targets, and the only difference between a free
+# local run and a live one. Selects a configuration directory rather than a code
+# path, so adding a provider is mostly adding a directory — see
 # service/submitqueue/demo/provider/README.md.
-PROVIDER ?= github
+#
+#   fake    a change is a URI; nothing merges anywhere. Needs nothing.
+#   git     branches in a bare repository on disk; real fetch, cherry-pick, push.
+#   github  real pull requests. Needs a repository and GITHUB_TOKEN.
+PROVIDER ?= fake
 export SQ_PROVIDER_CONFIG_DIR ?= $(REPO_ROOT)/service/submitqueue/demo/provider/$(PROVIDER)
 
-# Defaults for `make land` / `make demo-pr` against the provider demo stack.
+# Which compose overlay each mode needs. This cannot live in the provider
+# directory: the two config files say how the services are configured, not what
+# has to be mounted or which credential has to be present for them to start.
+PROVIDER_COMPOSE_FILE_fake = service/submitqueue/docker-compose.fake.yml
+PROVIDER_COMPOSE_FILE_git = service/submitqueue/docker-compose.git.yml
+PROVIDER_COMPOSE_FILE_github = service/submitqueue/docker-compose.provider.yml
+PROVIDER_COMPOSE_FILE = $(PROVIDER_COMPOSE_FILE_$(PROVIDER))
+
+# Where PROVIDER=git keeps the bare repository it merges into. Outside the
+# repository, so a demo leaves nothing in a checkout, and bind-mounted rather
+# than kept in a volume so `git log` on the host can show what landed.
+#
+# SQ_RUNWAY_CHECKOUT_DIR is deliberately not set: unset, Runway's working trees
+# live in a named volume instead of on the host. See docker-compose.git.yml.
+export SQ_GIT_SANDBOX_DIR ?= /tmp/sq-sandbox
+
+# Defaults for `make land` / `make demo-requests` against the provider demo stack.
 DEMO_REPO ?= behinddwalls/sq-demo
 COUNT ?= 3
 FILES ?= 3
@@ -77,7 +97,7 @@ define assert_clean
 	fi
 endef
 
-.PHONY: build build-all-linux build-runway-linux build-submitqueue-gateway-client build-submitqueue-gateway-linux build-submitqueue-gateway-server build-submitqueue-orchestrator-linux build-stovepipe-linux build-stovepipe-linux-debug check-gazelle check-mocks check-tidy clean clean-proto deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-binary lint-fmt lint-license local-init-runway-queue-schema local-init-stovepipe-schemas local-runway-start local-runway-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-debug-start local-stovepipe-logs local-stovepipe-start local-stovepipe-stop mocks proto query-deps query-targets run-client-runway run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe run-queue-admin test test-no-cache tidy tidy-bazel tidy-go help
+.PHONY: build build-all-linux build-runway-linux build-submitqueue-gateway-client build-submitqueue-gateway-linux build-submitqueue-gateway-server build-submitqueue-orchestrator-linux build-stovepipe-linux build-stovepipe-linux-debug check-gazelle check-mocks check-tidy clean clean-proto demo-requests deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-binary lint-fmt lint-license local-init-runway-queue-schema local-init-stovepipe-schemas local-provider-clean local-provider-start local-provider-stop local-runway-start local-runway-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-debug-start local-stovepipe-logs local-stovepipe-start local-stovepipe-stop mocks proto query-deps query-targets run-client-runway run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe run-queue-admin test test-no-cache tidy tidy-bazel tidy-go help
 
 
 build: ## Build all services and examples
@@ -172,9 +192,11 @@ clean-proto: ## Clean generated proto files
 	@rm -f $(foreach p,$(PROTO_PACKAGES),$(p)/protopb/*.pb.go $(p)/protopb/*.pb.yarpc.go)
 	@echo "Proto clean complete!"
 
-demo-pr: ## Create N PRs in the demo repo, enqueue each as it is created, and watch (COUNT=3 FILES=3 CONCURRENCY=5; needs GITHUB_TOKEN)
-	@$(BAZEL) run //service/submitqueue/demo/pr -- \
+demo-requests: ## Create N changes, enqueue each as it is created, and watch (PROVIDER=fake|git|github COUNT=3 FILES=3 CONCURRENCY=5)
+	@$(BAZEL) run //service/submitqueue/demo/requests -- \
+		-provider $(PROVIDER) \
 		-repo $(DEMO_REPO) \
+		-sandbox-dir $(SQ_GIT_SANDBOX_DIR) \
 		-count $(COUNT) \
 		-files $(FILES) \
 		-concurrency $(CONCURRENCY) \
@@ -301,26 +323,6 @@ local-submitqueue-gateway-stop: ## Stop Gateway service
 	@$(COMPOSE) -f $(GATEWAY_COMPOSE_FILE) -p $(SUBMITQUEUE_LOCAL_PROJECT) down
 	@echo "Gateway services stopped."
 
-local-provider-start: build-all-linux ## Start the full stack against a real provider (PROVIDER=github; needs GITHUB_TOKEN)
-	@echo "Starting full stack against provider '$(PROVIDER)' ($(SQ_PROVIDER_CONFIG_DIR))..."
-	@test -f "$(SQ_PROVIDER_CONFIG_DIR)/merge.yaml" \
-		|| { echo "No such provider '$(PROVIDER)': $(SQ_PROVIDER_CONFIG_DIR)/merge.yaml not found"; exit 2; }
-	@$(COMPOSE) -f $(COMPOSE_FILE) -f $(PROVIDER_COMPOSE_FILE) -p $(PROVIDER_LOCAL_PROJECT) up -d --build --wait
-	@echo "Applying database schemas..."
-	@$(MAKE) -s local-init-submitqueue-schemas SUBMITQUEUE_LOCAL_PROJECT=$(PROVIDER_LOCAL_PROJECT)
-	@echo ""
-	@echo "✅ Stack is running against provider '$(PROVIDER)'."
-	@echo ""
-	@echo "Gateway gRPC port: $$(docker port $(PROVIDER_LOCAL_PROJECT)-gateway-service-1 8080 2>/dev/null | cut -d: -f2 || echo 'unknown')"
-	@echo ""
-	@echo "Land a change with:"
-	@echo "  make land PR=https://github.com/owner/repo/pull/7 GATEWAY_ADDR=localhost:<gateway port>"
-
-local-provider-stop: ## Stop the provider demo stack
-	@echo "Stopping provider stack..."
-	@$(COMPOSE) -f $(COMPOSE_FILE) -f $(PROVIDER_COMPOSE_FILE) -p $(PROVIDER_LOCAL_PROJECT) down
-	@echo "Provider stack stopped."
-
 local-init-submitqueue-schemas: ## Manually apply all database schemas
 	@echo "Applying storage schema to mysql-app..."
 	@for file in submitqueue/extension/storage/mysql/schema/*.sql; do \
@@ -429,15 +431,38 @@ local-submitqueue-restart: build-all-linux ## Restart all services (rebuild and
 	@echo "Services restarted!"
 	@make local-submitqueue-ps
 
-local-submitqueue-start: build-all-linux ## Start full stack (Gateway + Orchestrator + MySQL)
-	@echo "Starting full stack with docker-compose..."
-	@$(COMPOSE) -f $(COMPOSE_FILE) -p $(SUBMITQUEUE_LOCAL_PROJECT) up -d --build --wait
+local-submitqueue-start: build-all-linux ## Start full stack (PROVIDER=fake|git|github; github needs GITHUB_TOKEN)
+	@echo "Starting full stack against provider '$(PROVIDER)' ($(SQ_PROVIDER_CONFIG_DIR))..."
+	@test -f "$(SQ_PROVIDER_CONFIG_DIR)/merge.yaml" \
+		|| { echo "No such provider '$(PROVIDER)': $(SQ_PROVIDER_CONFIG_DIR)/merge.yaml not found"; exit 2; }
+	@test -n "$(PROVIDER_COMPOSE_FILE)" \
+		|| { echo "Provider '$(PROVIDER)' has no compose overlay; add PROVIDER_COMPOSE_FILE_$(PROVIDER) to the Makefile"; exit 2; }
+	@if [ "$(PROVIDER)" = "git" ]; then \
+		$(BAZEL) run //tool/gitsandbox -- -sandbox-dir "$(SQ_GIT_SANDBOX_DIR)" || exit 1; \
+	fi
+	@# Rootless Docker maps container root to the host user; rootful Docker needs
+	@# the host UID:GID explicitly, or the services write files into the sandbox
+	@# bind mount that the host user cannot then read or remove. Resolved here
+	@# rather than at parse time, so `make help` does not shell out to Docker.
+	@set -e; \
+	if docker info --format '{{json .SecurityOptions}}' 2>/dev/null | grep -q 'name=rootless'; then \
+		export SQ_CONTAINER_USER=0:0; \
+	else \
+		export SQ_CONTAINER_USER=$$(id -u):$$(id -g); \
+	fi; \
+	$(COMPOSE) -f $(COMPOSE_FILE) -f $(PROVIDER_COMPOSE_FILE) -p $(SUBMITQUEUE_LOCAL_PROJECT) up -d --build --wait
 	@echo "Applying database schemas..."
 	@$(MAKE) -s local-init-submitqueue-schemas
 	@echo ""
-	@echo "✅ Full stack is running!"
+	@echo "✅ Stack is running against provider '$(PROVIDER)'."
 	@echo ""
-	@make local-submitqueue-ps
+	@echo "Gateway gRPC port: $$(docker port $(SUBMITQUEUE_LOCAL_PROJECT)-gateway-service-1 8080 2>/dev/null | cut -d: -f2 || echo 'unknown')"
+	@if [ "$(PROVIDER)" = "git" ]; then \
+		echo "Merge target:      $(SQ_GIT_SANDBOX_DIR)/sandbox.git"; \
+	fi
+	@echo ""
+	@echo "Generate traffic with:"
+	@echo "  make demo-requests GATEWAY_ADDR=localhost:<gateway port>"
 
 local-stop: ## Stop all services (keep data)
 	@echo "Stopping all services..."

README.md

@@ -15,7 +15,7 @@ Cross-domain Go code (errors, metrics, consumer framework, HTTP helpers, shared
 
 ## Quick Start
 
-Land a change and watch it reach `landed`. Requires Docker and Docker Compose, and nothing else — no repository, no account, no token. See [Development Setup](doc/howto/DEVELOPMENT.md) for full prerequisites.
+Put traffic through the queue and watch it land. Requires Docker and Docker Compose, and nothing else — no repository, no account, no token. See [Development Setup](doc/howto/DEVELOPMENT.md) for full prerequisites.
 
 ```bash
 # Start the full stack (Gateway + Orchestrator + Runway + MySQL)
@@ -25,28 +25,33 @@ make local-submitqueue-start
 make local-submitqueue-ps
 export GATEWAY_ADDR=localhost:<gateway port>
 
-# Submit a change, and follow the receipt it returns
-make land QUEUE=test-queue \
-  URI='git://git.example.com/demo/refs%2Fheads%2Ffeature-a/1111111111111111111111111111111111111111'
-make land-status QUEUE=test-queue SQID=test-queue/1
+# Create changes, enqueue each as it is created, and watch them settle
+make demo-requests
 
 # Stop services
 make local-stop
 ```
 
-Every integration at the edges is faked — the change provider, CI, and the merge itself — so the run is free and finishes in seconds. The queue's own logic is real: validation, batching, conflict analysis, and speculation all run, and the request log records the full trail from `accepted` to `landed`. Nothing is pushed to any repository.
+`PROVIDER` decides where changes come from and what landing them does, and it is the only thing that changes between them:
 
-[Quickstart](doc/howto/QUICKSTART.md) explains the change URI, how to make a change fail on demand, and what this does and does not prove. From there, `make e2e-git-test` adds a real git merge (still no credentials), and [PROVIDER-E2E.md](doc/howto/PROVIDER-E2E.md) adds a live provider. See [service/README.md](service/README.md) for running individual services and clients.
+| `PROVIDER` | A change is | Landing it | Needs |
+|---|---|---|---|
+| **`fake`** (default) | a URI, and nothing else | reports success without touching a repository | nothing |
+| **`git`** | a branch in a bare repository on disk | a real fetch, cherry-pick and push | nothing |
+| **`github`** | a real pull request | a real push to a real repository | a repository and a token |
+
+The queue's own logic is real in all three: validation, batching, conflict analysis, speculation, and a request log recording the full trail from `accepted` to `landed`. `PROVIDER=git make local-submitqueue-start` is the first rung where a commit actually reaches a branch, and it still needs no credential.
+
+[Quickstart](doc/howto/QUICKSTART.md) walks all three rungs — proving a change landed with `git log`, making one fail on demand, and what a live provider needs. See [service/README.md](service/README.md) for running individual services and clients.
 
 ## Documentation
 
 | Document | Description |
 |----------|-------------|
-| [Quickstart](doc/howto/QUICKSTART.md) | Land a change locally with no credentials |
+| [Quickstart](doc/howto/QUICKSTART.md) | Run the stack and land changes — fake, local git, or GitHub |
 | [Development Setup](doc/howto/DEVELOPMENT.md) | Prerequisites, build, environment, IDE setup |
 | [Contributing](CONTRIBUTING.md) | How to contribute, workflow, guidelines |
 | [Testing Guide](doc/howto/TESTING.md) | Unit, integration, and E2E testing patterns |
-| [Landing real changes](doc/howto/PROVIDER-E2E.md) | Running the pipeline against a live provider |
 | [Architecture Guide](CLAUDE.md) | Project layout, patterns, conventions |
 | [Examples](service/README.md) | Running services, clients, API reference |
 | [RFCs](doc/rfc/index.md) | Design documents and proposals |

doc/howto/DEVELOPMENT.md

@@ -64,16 +64,14 @@ make local-submitqueue-start
 make local-submitqueue-ps
 export GATEWAY_ADDR=localhost:<gateway port>
 
-# 4. Land a change and follow it to a terminal status
-make land QUEUE=test-queue \
-  URI='git://git.example.com/demo/refs%2Fheads%2Ffeature-a/1111111111111111111111111111111111111111'
-make land-status QUEUE=test-queue SQID=test-queue/1
+# 4. Create changes, enqueue them, and watch them land
+make demo-requests
 
 # 5. Stop services
 make local-stop
 ```
 
-[QUICKSTART.md](QUICKSTART.md) walks through the same run in detail — what the change URI has to look like, how to make a change fail on demand, and which parts of the pipeline are faked.
+[QUICKSTART.md](QUICKSTART.md) walks through the same run in detail, and on to `PROVIDER=git`, which lands real commits into a repository on disk — still with no credential.
 
 If any step fails, see [Troubleshooting](#troubleshooting) below.