smelt: .env.published was missing the indexer, and nothing could notice

smelt/.env.published

@@ -1,4 +1,4 @@
-# Published image references for the seven services this repository builds.
+# Published image references for the eight services this repository builds.
 #
 # Why this file exists: the compose interpolations for these images used to
 # carry a silent default (`${HILT_IMAGE:-ghcr.io/fil-forge/hilt:main}`), so a
@@ -19,8 +19,25 @@
 #
 # The same list lives in pkg/stack/options.go (publishedImages). That
 # duplication is deliberate: the Makefile shells out to `docker compose`
-# directly, so a Go table cannot reach it. Neither copy can drift silently —
-# a missing entry fails at `compose up` naming the variable.
+# directly, so a Go table cannot reach it.
+#
+# This file used to claim the two copies could not drift silently, because a
+# missing entry fails at `compose up` naming the variable. They drifted
+# anyway: indexing-service was added to the Go table and not here, so
+# `make up` was broken for as long as the indexer had been in the stack.
+# Nothing in CI runs `make up`, so "not silent" and "not noticed" were the
+# same thing. `TestPublishedImageSetAgrees` in pkg/stack is what makes the
+# claim true now: it checks this file against publishedImages and
+# envImageOptions, name by name AND reference by reference. The pairwise part
+# matters — the first attempt compared the lists as SETS and so reported
+# agreement on a tree where PIRI_IMAGE and HILT_IMAGE had been swapped here,
+# each service booting the other's image.
+#
+# It does NOT read the compose files, so it does not check that compose
+# requires exactly this set. A revision that did cost six review rounds
+# reimplementing compose's grammar, and was measured not to be load-bearing
+# for the drift above. The test's own comment says what that gives up and how
+# to get it back.
 
 PIRI_IMAGE=ghcr.io/fil-forge/piri:main
 HILT_IMAGE=ghcr.io/fil-forge/hilt:main
@@ -29,3 +46,4 @@ UPLOAD_IMAGE=ghcr.io/fil-forge/sprue:main
 DELEGATOR_IMAGE=ghcr.io/fil-forge/delegator:main
 SIGNER_IMAGE=ghcr.io/fil-forge/piri-signing-service:main
 SWARF_IMAGE=ghcr.io/fil-forge/swarf:main
+INDEXER_IMAGE=ghcr.io/fil-forge/indexing-service:main

smelt/CLAUDE.md

@@ -462,8 +462,17 @@ Available variables: `PIRI_IMAGE`, `GUPPY_IMAGE`, `DELEGATOR_IMAGE`, `INDEXER_IM
 
 Where the defaults come from depends on whether this repository builds the image:
 
-- **Built here** — `PIRI_IMAGE`, `HILT_IMAGE`, `INGOT_IMAGE`, `UPLOAD_IMAGE`, `DELEGATOR_IMAGE`, `SIGNER_IMAGE`. These have **no compose default**: the interpolation is `${PIRI_IMAGE:?...}`, so compose fails with the variable's name rather than quietly booting `ghcr.io/fil-forge/piri:main`. That silent fallback used to let a test that forgot its overrides pass while exercising published code instead of the commit under test. Published references for these live in `.env.published`, which `make up` loads via `--env-file`; a Go test asks for them with `stack.WithPublishedImages()`.
-- **Not built here** — everything else (guppy, indexer, ipni, swarf, plc, blockchain, minio). These keep an inline `:-` default in their `systems/*/compose.yml`, because nothing in this repository can produce an alternative. `SWARF_IMAGE` joins the required set when the branch that builds swarf lands.
+- **Built here.** These have **no compose default**: the interpolation is `${PIRI_IMAGE:?...}`, so compose fails with the variable's name rather than quietly booting `ghcr.io/fil-forge/piri:main`. That silent fallback used to let a test that forgot its overrides pass while exercising published code instead of the commit under test. Published references for these live in `.env.published`, which `make up` loads via `--env-file`; a Go test asks for them with `stack.WithPublishedImages()`.
+
+  **The set is not listed here**, because this paragraph carried a hand-written copy of it and went stale twice over: it named six when there were eight, and still called swarf and the indexer "not built here" long after both were imported. Derive it:
+
+  ```sh
+  find systems -name compose.yml -exec grep -hoE '\$\{[A-Z_]+_IMAGE:\?' {} + \
+    | sed -E 's/\$\{([A-Z_]+):\?/\1/' | sort -u
+  ```
+
+  That misses `PIRI_IMAGE`, which appears in no compose file — `pkg/generate/compose.go` emits piri's service definition. `TestPublishedImageSetAgrees` in `pkg/stack` checks `.env.published`, `publishedImages` and `envImageOptions` against each other — name by name and reference by reference, so it catches two of them naming *different* images for the same service. It deliberately does not read the compose files; its own comment says what that gives up and how to get it back.
+- **Not built here** — guppy, ipni, plc, blockchain, minio. These keep an inline `:-` default in their `systems/*/compose.yml`, because nothing in this repository can produce an alternative, and they are pinned by digest instead (`.github/scripts/check-stack-images.sh`).
 
 `.env` is yours: every line in it is commented out, and `make up` loads it *after* `.env.published`, so anything you uncomment there wins.
 

smelt/pkg/stack/options.go

@@ -406,13 +406,20 @@ var envImageOptions = []struct {
 // builds, alongside the config field it lands in. It is the Go-side twin of
 // .env.published, which says the same thing for `make up` — the Makefile
 // shells out to `docker compose` directly, so it cannot read this table.
-// Neither copy can drift silently: the compose interpolations for these are
-// required (`${X:?...}`), so a missing entry fails at `compose up` naming
-// the variable rather than quietly booting :main.
 //
-// Only what this repository builds. The other images the stack runs (guppy,
-// ipni, plc, blockchain, minio) come from elsewhere and keep their `:-`
-// compose defaults, so there is nothing for this to fill in.
+// THIS COMMENT USED TO SAY the two copies could not drift silently, because
+// the interpolations are required (`${X:?...}`) so a missing entry fails at
+// `compose up` naming the variable. That is true and it is not enough: they
+// drifted anyway — indexing-service was added here and not to .env.published,
+// and `make up` had been failing ever since, because nothing in CI runs it.
+// TestPublishedImageSetAgrees is what checks it now: this table against
+// .env.published and envImageOptions above, name by name and reference by
+// reference.
+//
+// Only what this repository builds; the rest keep their `:-` compose defaults,
+// so there is nothing for this to fill in. That set is not listed here — it
+// was, and the copy in smelt/CLAUDE.md went stale twice over — it is whatever
+// the test derives.
 var publishedImages = []struct {
 	ref string
 	get func(*config) *string