At commit 151a8d1c922ffadad08399508efe46b207a5894e:
// scripts/wiki-synthesis/synthesize-wiki.mjs (around L194)
LLM_MODEL: fileEnv.LLM_MODEL || process.env.LLM_MODEL || "anthropic/claude-haiku-4-5",
synthesize-wiki.mjs serializes all thoughts within a year bucket into one LLM call (per --topic). The default model is Haiku 4.5, which has a 200k context window.
For an 8-employee deployment with ~366 thoughts in the corpus, --topic autobiography produces a ~203k-token prompt and aborts:
LLM 400: ContextWindowExceededError: prompt is too long: 203107 tokens > 200000 maximum
model=anthropic/claude-haiku-4-5
This will hit any deployment whose year-bucket exceeds ~200k tokens, which is a pretty common threshold once a few hundred thoughts accumulate.
Suggested directions (pick one)
- Bump default to a 1M-context model.
anthropic/claude-sonnet-4-6 with the anthropic-beta: context-1m-2025-08-07 header gives 1M tokens for ~5x the cost. Topic-wiki runs are typically weekly per topic, so the marginal cost is small in absolute dollars.
- Add
--bucket-size N flag to chunk year-buckets when the input exceeds a target token count. Keeps Haiku as default; trades single-call narrative coherence for multi-call cost.
- At minimum: README note + early validation. Sum input tokens before the call, abort with a friendly error pointing at the model-choice trade-off, rather than the current behavior (one wasted LLM call, then a crash).
What I did locally
Patched the default to anthropic/claude-sonnet-4-6-1m (a LiteLLM model entry that adds the 1M-context beta header). Topic synthesis then succeeded against the full 366-thought corpus and produced a coherent autobiography wiki.
Context
Davies Farms self-hosted deployment. Same setup as #313 / #314 / #315. The fix is small and is probably worth landing upstream because the 200k threshold is reachable surprisingly fast.
Happy to PR.
At commit
151a8d1c922ffadad08399508efe46b207a5894e:synthesize-wiki.mjsserializes all thoughts within a year bucket into one LLM call (per--topic). The default model is Haiku 4.5, which has a 200k context window.For an 8-employee deployment with ~366 thoughts in the corpus,
--topic autobiographyproduces a ~203k-token prompt and aborts:This will hit any deployment whose year-bucket exceeds ~200k tokens, which is a pretty common threshold once a few hundred thoughts accumulate.
Suggested directions (pick one)
anthropic/claude-sonnet-4-6with theanthropic-beta: context-1m-2025-08-07header gives 1M tokens for ~5x the cost. Topic-wiki runs are typically weekly per topic, so the marginal cost is small in absolute dollars.--bucket-size Nflag to chunk year-buckets when the input exceeds a target token count. Keeps Haiku as default; trades single-call narrative coherence for multi-call cost.What I did locally
Patched the default to
anthropic/claude-sonnet-4-6-1m(a LiteLLM model entry that adds the 1M-context beta header). Topic synthesis then succeeded against the full 366-thought corpus and produced a coherent autobiography wiki.Context
Davies Farms self-hosted deployment. Same setup as #313 / #314 / #315. The fix is small and is probably worth landing upstream because the 200k threshold is reachable surprisingly fast.
Happy to PR.