From 5c084451f709b6e80c560a50acf75adb8ca96bfe Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 17 Aug 2026 11:20:28 +0000 Subject: [PATCH] feat: index each language separately, and assert it stays that way Picks up SifterSearch v0.7.2, whose fix indexes each page in the language it is written in rather than the wiki's content language (chaotic-ground/SifterSearch#69). Pagefind builds one index per language it finds and its client selects by the language of the page the reader is on, so until now this site shipped one English index holding all 59 pages: Korean and Khmer were tokenised and stemmed by English rules, and a reader searching from a translated page was answered out of the English index because there was no other to fall back to. A bake now produces three -- en 39, ko 19, km 1. The smoke test says so out loud and fails if English and Korean are not indexed apart. It is the assertion this site was missing: nothing here would have caught a search answering a Korean reader out of an English index, because the search still worked. The existing Pagefind checks assert a bundle exists per skin copy (#399); none of them looked at what was in it. Khmer shows in the printed counts without being required, one page being a thin basis to gate on. Reproducibility is already handled: three languages made Pagefind's language map come out in a different order each bake, which #451 settled ahead of this so the bump would not fail the two-bake check for a reason unrelated to it. Verified before this pin moved, against the nightly carrying the same fix. Closes #400. --- _Generated by [Claude Code](https://claude.ai/code/session_01NiyBN88bnGhcXVUXkpD7Mc)_ Co-authored-by: Claude --- .github/workflows/smoke.yml | 16 ++++++++++++++++ Dockerfile | 2 +- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/.github/workflows/smoke.yml b/.github/workflows/smoke.yml index c54359ef3..fbc5ed91d 100644 --- a/.github/workflows/smoke.yml +++ b/.github/workflows/smoke.yml @@ -98,6 +98,22 @@ jobs: fi done + # Each page is indexed in the language it is written in, so Pagefind builds one index per + # language and a reader searching from a translated page is answered out of that + # language's index (#400). A single index means every translation was stamped with the + # wiki's content language and stemmed by English rules, which is what this site looked + # like until SifterSearch indexed per page rather than per wiki. + echo "Search indexes built:" + ls dist/pagefind/*.pf_meta | xargs -n1 basename + echo "Indexed pages per language:" + ls dist/pagefind/fragment | sed 's/_.*//' | sort | uniq -c + for lang in en ko; do + if ! ls dist/pagefind/pagefind."${lang}"_*.pf_meta >/dev/null 2>&1; then + echo "::error::dist/pagefind has no $lang index; pages are not indexed in their own language" + exit 1 + fi + done + # Every printfooter "Retrieved from" link must resolve from the page that carries it # (#394). Core builds it from the page's own URL and expands away the "./" wikven writes, # so a page exported into a subdirectory needs the "../" per level rename.php adds; the diff --git a/Dockerfile b/Dockerfile index 6629d8692..d6be662c5 100644 --- a/Dockerfile +++ b/Dockerfile @@ -23,7 +23,7 @@ RUN apk add --no-cache rsvg-convert imagemagick-jpeg imagemagick-webp # SifterSearch (client-side Pagefind search) ships built in. Its release tarball carries the # per-arch Pagefind binary a git clone omits, so fetch the one matching this build's architecture. ARG TARGETARCH -ARG SIFTERSEARCH_VERSION=v0.7.1 +ARG SIFTERSEARCH_VERSION=v0.7.2 RUN arch="$TARGETARCH" \ && if [ "$arch" = amd64 ]; then arch=x64; fi \ && curl -fsSL "https://github.com/chaotic-ground/SifterSearch/releases/download/${SIFTERSEARCH_VERSION}/SifterSearch-linux-${arch}.tar.gz" \