The gap
The wiki-as-memory loop this tool implements is canonically ingest → query → lint → write back — but openwiki currently ships without the lint step. The synthesis agent updates pages locally without re-verifying the wiki's cross-reference graph, so regenerations reliably ship a class of internal-consistency drift that review bots then catch one item per round:
- Stale counts/enumerations: a page opening still said "the five existing dimensions" two paragraphs above freshly regenerated content documenting the sixth; a "planned but not yet authored" list contradicted by the same run's own additions.
- Dead internal anchors: a heading edited during regeneration left links in sibling pages pointing at the old anchor.
- Repo-state claims that drift from the committed tree: generated pages describing files that exist only in the working tree (e.g., the generator's own uncommitted CI scaffold), which reads as broken guidance to anyone at the committed tree.
Observed across several 0.1.2 code-mode wikis (both --update and --init paths) and equally applicable to personal-brain wikis.
Proposal
Two tiers, cheapest first:
-
Deterministic validator (no LLM) — after generation completes, before .last-update.json is written and before the generated CI workflow opens its PR: resolve every relative [text](path.md#anchor) link against files that exist and anchors GitHub actually generates (lowercase → strip non [\w\- ] → each space becomes one hyphen, so "a + b" → a--b; duplicate headings get -1, -2 suffixes; root-relative /path.md resolves from the repo root). ~60 lines of stdlib Python/TS; fail the run or surface the list to the agent for a self-repair turn.
-
LLM lint pass (the missing loop step) — a cheap final turn over the diff'd pages asking specifically for cross-page contradictions the local edits introduced: counts/enumerations that no longer match, claims about repo state, "planned" lists contradicted by shipped content. This is much cheaper as a dedicated pass than as N rounds of external review-bot findings.
Happy to contribute the tier-1 validator as a PR if there's interest — we're running exactly this as a post-generation gate in our repos' openwiki CI workflows and it took the review-round churn to zero.
The gap
The wiki-as-memory loop this tool implements is canonically ingest → query → lint → write back — but openwiki currently ships without the lint step. The synthesis agent updates pages locally without re-verifying the wiki's cross-reference graph, so regenerations reliably ship a class of internal-consistency drift that review bots then catch one item per round:
Observed across several 0.1.2 code-mode wikis (both
--updateand--initpaths) and equally applicable to personal-brain wikis.Proposal
Two tiers, cheapest first:
Deterministic validator (no LLM) — after generation completes, before
.last-update.jsonis written and before the generated CI workflow opens its PR: resolve every relative[text](path.md#anchor)link against files that exist and anchors GitHub actually generates (lowercase → strip non[\w\- ]→ each space becomes one hyphen, so"a + b"→a--b; duplicate headings get-1,-2suffixes; root-relative/path.mdresolves from the repo root). ~60 lines of stdlib Python/TS; fail the run or surface the list to the agent for a self-repair turn.LLM lint pass (the missing loop step) — a cheap final turn over the diff'd pages asking specifically for cross-page contradictions the local edits introduced: counts/enumerations that no longer match, claims about repo state, "planned" lists contradicted by shipped content. This is much cheaper as a dedicated pass than as N rounds of external review-bot findings.
Happy to contribute the tier-1 validator as a PR if there's interest — we're running exactly this as a post-generation gate in our repos' openwiki CI workflows and it took the review-round churn to zero.