Skip to content

Commit c122fc0

Browse files
committed
docs(quickstart): correct the commands that no longer work, and lead GitHub with demo-requests
## Summary ### Why? The quickstart was audited against a live stack on all three rungs, including a real GitHub run. Most of it holds. Four things did not. **Three commands named a compose project that no longer exists.** Folding `local-provider-*` into `local-submitqueue-*` left one project, `submitqueue`, but the troubleshooting and log-tailing commands still said `submitqueue-provider`. Every one of them fails: `docker logs submitqueue-provider-mysql-queue-1` finds no such container, and a `down -v` against that project silently does nothing — which is worse, because it is offered as the fix for a stack that will not start, and appears to work. **The GitHub rung led with the one command that is not the quickstart.** Its first instruction was to open a pull request by hand and land it with `make land PR=…`, while `make demo-requests` — the command the other two rungs use, which opens the pull requests for you — was a sentence above it. The rung that needs the most setup was the one asking for the most manual work. **`make local-submitqueue-stop` claimed to preserve data it does not.** Both MySQL services mount *anonymous* volumes, so `down` detaches them and the next `up` gets empty databases. The quickstart says so; the Makefile printed "Data volumes preserved", which reads as "your requests are still there". They are not. **A failing change was documented without saying which rung it belongs to.** The `?sq-fake=build-fail` URI is a `git://` one pointing at nothing, which only resolves where the change provider is fake. It sits after the GitHub section, so it reads as applying there, where it fails for an unrelated reason. ### What? The stale project name is corrected in all three places, and the `down -v` recipe becomes `make local-submitqueue-clean`, which is the same operation with the right project and overlay. `PROVIDER_LOCAL_PROJECT` is deleted from the Makefile — its last use went with `local-provider-*`, and a variable that still defines the wrong answer is how the wrong answer gets copied again. **The GitHub rung now leads with `make demo-requests`**, with real output from a scratch-repo run, and keeps `make land PR=…` below it under a heading that says what it is for — landing a pull request you opened yourself. Nothing was cut; the stack case moves to `make demo-requests STACKED=true`, which is what the other rungs already use for it. It also now says plainly that **your CI does not run these builds**: the build runner is fake on the GitHub rung too, so `landed` there does not mean anything was tested. That was in the "Using real CI" section further down, which is too late for a reader who has just watched three pull requests merge. The stop message says what actually happens, and the failing-change section says it belongs to the `fake` rung. ## Test Plan Run against a live stack, in this order: - ✅ `PROVIDER=github` end to end for the first time, against a scratch repository: `make demo-requests COUNT=3` opened pull requests 522–524, landed all three, and the API confirms `"merged": true` on each with their commits on `main` — so the rung the documentation described but nobody had run, works - ✅ that run is what the new output block quotes, rather than an invented one - ✅ the fake build runner claim is now checked two ways: `demo-queue` inherits `buildRunner: {type: fake}` in the GitHub profile, and the repository's newest `workflow_dispatch` run predates the run by six days. Zero check-runs on the landed head confirms nothing else fired either - ✅ the failing change reaches `error`, as documented: submitted the exact URI from the doc and read `status: error` back - ✅ `make local-submitqueue-stop` prints the corrected message - ✅ `make lint`, `make gazelle` Checked and found accurate, so left alone: every `make demo-requests` option (`COUNT`, `FOLDERS`, `FILES`, `CONCURRENCY`, `STACKED`, `LAND`) against the recipe's flags; `land`, `land-status`, `land-list`, `land-watch` and their variables; all six `sq-fake` tokens against the fakes that read them; `SQ_TOKEN` and `-token-env`; `QUEUE_LOG_LEVEL`; the sandbox path; and the anonymous-volume behaviour the clean-up section describes. `make land-list` against the failed request also confirms the documented difference between `list` and `watch` — the row showed `error` alone, with no trail, because `list` does not fetch a history per request.
1 parent e9422ab commit c122fc0

2 files changed

Lines changed: 40 additions & 17 deletions

File tree

Makefile

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,6 @@ ORCHESTRATOR_COMPOSE_FILE = service/submitqueue/orchestrator/server/docker-compo
1212
# Fixed project name for local manual testing (tests use unique random names)
1313
SUBMITQUEUE_LOCAL_PROJECT = submitqueue
1414

15-
# Separate project for the provider demo stack, so it can run alongside the plain
16-
# local stack without the two sharing containers or volumes.
17-
PROVIDER_LOCAL_PROJECT = submitqueue-provider
18-
1915
# Stovepipe compose file (single Ping-only service)
2016
STOVEPIPE_COMPOSE_FILE = service/stovepipe/docker-compose.yml
2117
STOVEPIPE_DEBUG_COMPOSE_FILE = service/stovepipe/docker-compose.debug.yml
@@ -525,10 +521,14 @@ local-submitqueue-start: build-all-linux ## Start full stack (PROVIDER=fake|git|
525521
@echo "Generate traffic with:"
526522
@echo " make demo-requests"
527523

528-
local-submitqueue-stop: ## Stop the SubmitQueue stack (keeps data and PROVIDER=git's sandbox)
524+
local-submitqueue-stop: ## Stop the SubmitQueue stack (keeps PROVIDER=git's sandbox; the databases do not survive)
529525
@echo "Stopping SubmitQueue services..."
530526
@$(COMPOSE) -f $(COMPOSE_FILE) -f $(PROVIDER_COMPOSE_FILE) -p $(SUBMITQUEUE_LOCAL_PROJECT) down
531-
@echo "SubmitQueue services stopped. Data volumes preserved."
527+
@# Both MySQL services mount anonymous volumes, so `down` detaches them and
528+
@# the next `up` creates fresh ones. Saying "data preserved" here would be
529+
@# read as "your requests are still there", which they are not.
530+
@echo "SubmitQueue services stopped. The databases were on anonymous volumes and start empty next time;"
531+
@echo "'make local-submitqueue-clean' deletes the orphaned ones."
532532
@if [ -d "$(SQ_GIT_SANDBOX_DIR)" ]; then \
533533
echo "Sandbox repository left at $(SQ_GIT_SANDBOX_DIR); remove it with 'make local-submitqueue-clean'."; \
534534
fi

doc/howto/QUICKSTART.md

Lines changed: 34 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,8 @@ SQ_TOKEN=$(cat ~/.sq-token) bazel run //service/submitqueue/gateway/client:gatew
163163
### Service logs
164164

165165
```bash
166-
docker compose -p submitqueue-provider logs -f runway-service
166+
make local-submitqueue-logs # every service
167+
docker compose -p submitqueue logs -f runway-service # one of them
167168
```
168169

169170
The message queue logs a line per message published, fetched, leased and acked, which at debug level buries everything else a service says. It is levelled separately from the rest of the service, at info by default. To follow the queue itself — chasing a message that never arrived, or a partition that never got leased — turn it back up:
@@ -276,19 +277,39 @@ PROVIDER=github make local-submitqueue-start
276277

277278
The token is required rather than defaulted: a stack that silently falls back to the fake integrations reports changes as landed without having gone near the provider, which is a much worse way to find out.
278279

279-
From here everything is as before — `make demo-requests` opens real pull requests, enqueues them and watches them land, having picked up from the running stack that this one is GitHub.
280+
Then the same command as the other two rungs:
280281

281-
### Land a pull request
282+
```bash
283+
make demo-requests
284+
```
285+
286+
It opens real pull requests, enqueues each as it is created, and watches them land — having picked up from the running stack that this one is GitHub. A three-change run against a scratch repo:
287+
288+
```
289+
REQUEST CHANGES ELAPSED STAGE
290+
──────────── ──────────────────────────────────────────────── ─────── ──────────────────────────────
291+
demo-queue/1 https://github.com/behinddwalls/sq-demo/pull/522 21s accepted → … → landed
292+
demo-queue/2 https://github.com/behinddwalls/sq-demo/pull/523 22s accepted → … → landed
293+
demo-queue/3 https://github.com/behinddwalls/sq-demo/pull/524 25s accepted → … → landed
294+
```
282295

283-
Open a pull request against `main` in the scratch repo, then:
296+
All three show **Merged** on GitHub and their commits are on `main`.
297+
298+
Worth understanding *why* they show merged, because nothing called an API to close them. A provider marks a change merged once its head commit is reachable from the target branch. `SQUASH_REBASE` rewrites the commits, so a pull request's original head is nowhere in `main` — and `updateHeadBranch` therefore moves its branch to the commit it landed as. GitHub draws its own conclusion from that.
299+
300+
`make demo-requests STACKED=true` submits a chain instead, each pull request targeting the previous one's branch. All of them land as one push to `main`, and all of them show as merged.
301+
302+
**Your CI does not run these builds.** Even here the build runner is fake, so a land takes seconds and costs no Actions minutes — GitHub supplies the change metadata and takes the push, and nothing else. That is worth knowing before reading `landed` as "CI passed on the combination", because it did not run. See [Using real CI](#using-real-ci) below.
303+
304+
### Land an existing pull request
305+
306+
For a pull request you opened yourself rather than one the demo created:
284307

285308
```bash
286309
make land PR=https://github.com/<you>/<repo>/pull/1
287310
```
288311

289-
`land` resolves the pull request's head commit and prints the change URI it built, so there is no 40-character SHA to copy. It returns an `sqid` to follow with `make land-status`. When the request reaches `landed`, the pull request shows **Merged** and its commit is on `main`.
290-
291-
Worth understanding *why* it shows merged, because nothing called an API to close it. A provider marks a change merged once its head commit is reachable from the target branch. `SQUASH_REBASE` rewrites the commits, so the pull request's original head is nowhere in `main` — and `updateHeadBranch` therefore moves the pull request's branch to the commit it landed as. GitHub draws its own conclusion from that.
312+
`land` resolves the pull request's head commit and prints the change URI it built, so there is no 40-character SHA to copy. It returns an `sqid` to follow with `make land-status`.
292313

293314
A stack is a chain of pull requests where each targets the previous one's branch, submitted in order:
294315

@@ -298,7 +319,7 @@ make land PRS="https://github.com/<you>/<repo>/pull/1 \
298319
https://github.com/<you>/<repo>/pull/3"
299320
```
300321

301-
The order of `PRS` is the stack order. All three land as one push to `main`, and all three show as merged.
322+
The order of `PRS` is the stack order.
302323

303324
### Using real CI
304325

@@ -343,7 +364,7 @@ One caveat worth understanding before you rely on the result. A workflow that on
343364

344365
## Make a change fail
345366

346-
The fakes take instructions through the change URI itself, so a failure needs no configuration change and no restart. Append `?sq-fake=build-fail`:
367+
Back on the `fake` rung, where a change is a URI and nothing has to exist for one to name it. The fakes take instructions through the change URI itself, so a failure needs no configuration change and no restart. Append `?sq-fake=build-fail`:
347368

348369
```bash
349370
make land QUEUE=demo-queue \
@@ -352,6 +373,8 @@ make land QUEUE=demo-queue \
352373

353374
That request walks the same path as far as `speculating`, records `building`, and then goes terminal at `error` instead of landing. Other tokens follow the same `sq-fake=<token>` convention and are documented on the fake they drive — `provider-error` on the change provider, `unmergeable` and `mergecheck-error` on the merge checker, `trigger-error` and `build-error` on the build runner.
354375

376+
A hand-written URI like the one above belongs to the `fake` rung alone. On `git` it names a commit the merger cannot fetch, and on `github` the change provider tries to resolve it as a pull request — both fail, but for reasons that have nothing to do with the marker.
377+
355378
Submit a good change into the **same folder** as a failing one and you can watch what makes a queue worth having: the two are batched in order, and the second speculates on the first landing. When the first fails, that guess is contradicted, the second re-plans, and it lands anyway.
356379

357380
## Clean up
@@ -367,10 +390,10 @@ Both MySQL services mount **anonymous** volumes, so a stop/start cycle orphans a
367390

368391
## Troubleshooting
369392

370-
**MySQL exits immediately, and the stack fails with `dependency failed to start`.** Check `docker logs submitqueue-provider-mysql-queue-1`. Two causes look similar:
393+
**MySQL exits immediately, and the stack fails with `dependency failed to start`.** Check `docker logs submitqueue-mysql-queue-1`. Two causes look similar:
371394

372395
- `No space left on device` — Docker is full, usually of the orphaned volumes above; `docker system df` shows the total. `docker volume prune` reclaims every detached volume on the machine, so check `docker volume ls -f dangling=true` first if anything else of yours might be in there.
373-
- `--initialize specified but the data directory has files in it` — a previous run died partway through initializing. Remove that stack's volumes with `docker compose -f service/submitqueue/docker-compose.yml -p submitqueue-provider down -v` and start again.
396+
- `--initialize specified but the data directory has files in it` — a previous run died partway through initializing. Remove that stack's volumes with `make local-submitqueue-clean` and start again.
374397

375398
**A land is rejected before it returns an sqid.** The URI failed validation: 40 hex characters of SHA, and a percent-encoded `refs/…` ref.
376399

0 commit comments

Comments
 (0)