Repository navigation
offline profile 6: narratives and collective statements can run on a local Ollama model, with provenance shown on the report - #2945
Open
colinmegill wants to merge 3 commits into
Open
colinmegill wants to merge 3 commits into
colinmegill wants to merge 3 commits into
Conversation
With LLM_PROVIDER=ollama, 801 sends the same prompts it would batch, in the same section order, to OLLAMA_HOST/OLLAMA_MODEL and stores each section as it completes, under the section names 803 would use. No batch is submitted and no checker job is created: the job completes when 801 exits, and a failed, timed-out or malformed section fails the job (exit 1, error_message on the row) with an error placeholder in that section. Provenance: each NarrativeReports item carries provider "ollama" and the model, both at the top level and inside its existing metadata map (which the reports route already returns); the job row records provider and model. No new table or column. Bounded: one request at a time by default (OLLAMA_NARRATIVE_CONCURRENCY), a per-request timeout (OLLAMA_REQUEST_TIMEOUT_SECONDS, 600), a per-job budget (OLLAMA_NARRATIVE_JOB_TIMEOUT_SECONDS, 3000) and a context window (OLLAMA_NUM_CTX, 16384). Compose forwards all four. The poller passes OLLAMA_MODEL to 801 for ollama instead of requiring ANTHROPIC_MODEL. The Ollama client is now bound to its endpoint with ollama.Client(host=...): the old module-attribute assignment raised AttributeError for any non-default endpoint whenever the ollama package was installed, which it is in the image. LLM_PROVIDER=anthropic is unchanged: the batch custom_id code moved into a helper with the same output, and the batch and 803 paths are untouched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s
…a model, and the report says when a local model wrote a section collectiveStatement.ts gains the provider switch Delphi uses: with LLM_PROVIDER=ollama it posts the same system and user prompt to OLLAMA_HOST/OLLAMA_MODEL (non-streaming, JSON format, bounded by OLLAMA_REQUEST_TIMEOUT_SECONDS, context OLLAMA_NUM_CTX) and parses the reply through the existing JSON path and fallback. A failed call answers 500 and stores nothing. Any other value keeps claude-opus-4-8 with the same parameters. Each stored statement records provider and model as attributes of its existing Delphi_CollectiveStatement item (no new table or column), and the POST response also returns created_at, model and provider, which the report already reads. When the statement fires is unchanged. client-report: a LocalModelNote shows "Generated by a local model (<model>); quality differs from the hosted model." under a statement (topic page, statement modal, statements report) and under a narrative section (comments report, topic report, topic page) whose provider is "ollama". Hosted output renders as before. docs/configuration.md: LLM_PROVIDER=ollama now covers topic names, narratives and statements; models, the four knobs and the quality note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s
colinmegill
added a commit
that referenced
this pull request
Oct 3, 2026
…ocal narratives example.offline.env sets OLLAMA_HOST=http://ollama:11434: the server reads it for the narrative and collective statement and would otherwise use localhost. docs/offline.md takes the lines the data export PR (#2940) and the Ollama narrative PR (#2945) left for it: checklist item 6 adds the conversation data export zip, item 7 has the narrative and collective statement from the local model, and the known gaps drop the Anthropic-only line and describe the streamed export. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s
… nothing, statement timeout under nginx's limit - client-report: LocalModelNote takes as="span"; the topic page renders it that way inside the statement's metadata paragraph (a <p> cannot contain a <p>). - delphi: error placeholders are written only after at least one section was stored. A local run in which every section fails writes no rows and fails the job, so the report (which shows the newest job that has rows) keeps showing the previous good run. - server: the local statement request timeout is capped at 240 s, below nginx's 300 s proxy_read_timeout, so a slow local model returns a clean 500 and stores nothing instead of a 504 followed by a stored statement. Documented in docs/configuration.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s
Contributor
Delphi Coverage Report
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Item 13 of the offline profile. With
LLM_PROVIDER=ollama, the narrative report and the collective statement now run on the local Ollama model, the same switch and settings that topic naming already uses. The report says when a local model wrote a section or a statement.The provider switch
Narrative report (delphi,
801_narrative_report_batch.py)LLM_PROVIDER=ollama, 801 builds the same requests it would batch (same prompts, same section order,max_tokens8000). It sends them toOLLAMA_HOST/OLLAMA_MODELas non-streaming chat calls in JSON format.custom_id(e.g.batch_re_global_groups, which the report looks up). The row shape is the one 803 writes.AWAITING_NARRATIVE_BATCHchecker row is created. The job completes when 801 exits.OLLAMA_NARRATIVE_CONCURRENCYsets how many requests run at once (default 1).OLLAMA_REQUEST_TIMEOUT_SECONDSlimits one request (default 600).OLLAMA_NARRATIVE_JOB_TIMEOUT_SECONDSis the job's budget (default 3000, under the poller's 3600). Once it is spent, the remaining sections are not sent.OLLAMA_NUM_CTXsets the context window (default 16384). Ollama's own default window would silently cut a prompt that carries 100 comments.paragraphs) or fails to write, 801 exits 1. The poller then marks the job FAILED, and 801 writeserror_messageon the row ("k of n sections failed …").OLLAMA_MODELto 801 for ollama, instead of requiringANTHROPIC_MODEL.OllamaProvidernow binds anollama.Client(host=…)to its endpoint. The oldollama.client.api_base = …assignment raisedAttributeErrorfor every non-default endpoint whenever theollamapackage was installed, and it is installed in the image. As a result,LLM_PROVIDER=ollamatopic naming could not reachhttp://ollama:11434and fell back to keyword labels.Collective statement (server,
collectiveStatement.ts)LLM_PROVIDER=ollama, the route sends the same system and user prompt toOLLAMA_HOST/OLLAMA_MODEL. It uses the same JSON parse and the same fallback shape downstream.proxy_read_timeout. A slow local model therefore returns that clean 500, rather than a 504 followed by a stored statement. The cap is documented indocs/configuration.md.llmProvider,ollamaHost,ollamaModel,ollamaNumCtxandollamaRequestTimeoutSeconds. The server reads them from its env file.Provenance
Delphi_NarrativeReportsitem written by the local path carriesprovider: "ollama"and the model. They are top-level attributes, and they are also inside the item's existingmetadatamap, whichGET /delphi/reportsalready returns. The job'sDelphi_JobQueueitem getsproviderandmodelattributes.Delphi_CollectiveStatementitem recordsproviderandmodel. The POST response also returnscreated_at,modelandprovider; the report already read those fields.LocalModelNoterenders "Generated by a local model (); quality differs from the hosted model." whenproviderisollama. It appears in six places:Unchanged with
LLM_PROVIDER=anthropicNo behaviour change with LLM_PROVIDER=anthropic.
custom_idcode moved into a helper whose output is identical, and a test pins it.claude-opus-4-8with the same parameters.The only differences on the hosted path are additive fields: the statement item and the POST response now also name the provider, and the POST response also returns
created_atandmodel.No schema change (provenance stored in the existing JSON payloads — confirm no new column; if a column would be needed, store it inside the existing item/JSON and say so). Confirmed: no Postgres table or column is touched. Provenance lives as attributes of the existing DynamoDB items (schemaless) and inside the narrative item's existing
metadatamap. No new table or index. No Lambda.Tests
No live model calls; all fixtures are generated.
delphi (
tests/test_narrative_ollama.py, 25 tests). A fake Ollama HTTP server runs in-process. The cases cover:main, and the poller's model choice;custom_id;tests/test_compose_math_env.py(LLM keys) andtests/topic_namingpass.server (
__tests__/unit/collectiveStatementProvider.test.ts, 9 tests). They cover:tsc --noEmitis clean.client-report.
LocalModelNote.test.jsxpasses, including a case that renders the note as a span inside an existing paragraph (the topic page does this, because a<p>cannot hold a<p>). The existingCommentsReportsuite also passes.Follow-ups
server/src/routes/reportNarrative.ts, behind/api/v3/reportNarrative, uses Anthropic, Gemini or OpenAI.docs/configuration-env-reference.mddoes not list the new keys yet: the server'sLLM_PROVIDERandOLLAMA_*, and Delphi'sOLLAMA_NARRATIVE_*,OLLAMA_NUM_CTXandOLLAMA_REQUEST_TIMEOUT_SECONDS.LLM_PROVIDER=ollamaswitches topic names, narratives and statements together. A per-use override (e.g. keep statements hosted) is left for a later PR.Follow-ups for the offline overlay (#2938)
docs/offline.mdis not onedgeyet, so its checklist is not edited here. Once offline profile 4: the offline compose overlay, the image bundle script and the install guide (stacked on #2929, #2937, #2936) #2938 lands, item 7 should read: "Topic tier: the admin starts a Delphi run. The local model names the topics, or keyword labels do if the model is absent. The narrative report and the collective statement come from the local model, and the report marks them 'Generated by a local model ()'." Drop the "Narrative report and collective statement" line from its known gaps.OLLAMA_HOSTfrom its env file.example.offline.envsetsLLM_PROVIDERandOLLAMA_MODELbut notOLLAMA_HOST, so the server would default tolocalhost:11434. It needsOLLAMA_HOST=http://ollama:11434.🤖 Generated with Claude Code
https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s