Skip to content

Promote edge to stable: Polis 2.44.0 - #3426

Open
github-actions[bot] wants to merge 7 commits into
stablefrom
edge
Open

github-actions[bot] wants to merge 7 commits into
stablefrom
edge

Conversation

@github-actions

@github-actions github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Release plan

Version: 2.44.0

Reason: new selected migrations

Previous stable: bce88d88bba0f25cdfc1e0b4d38f0091a042af95

Candidate: 1559ab192dce597928372e2bf111f593f72409b7

release.txt additions:

  • 000046_add_granite_embedding_model.sql

Merge the reviewed version preparation into edge, then refresh this plan for the resulting edge SHA before merging the promotion with a merge commit.

colinmegill and others added 2 commits November 15, 2026 14:09
…listed conversations, switched off (#3422)

* delphi: a run can leave out its visualization pages and remove its local files (switched off)

Two switches for run_delphi.py, both off unless set to exactly "1"; off, a
run starts the same stages and writes the same files as before.

DELPHI_SKIP_DATAMAPPLOT=1 does not start the per-layer datamapplot step
(700_datamapplot_for_layer.py uploads each page, which holds comment text,
to the S3 bucket), and the UMAP stage writes none of its local layer pages.
Every result family is still written; a failed UMAP stage still fails the
run.

DELPHI_LOCAL_OUTPUT_CLEANUP=1 removes polis_data/<zid> and
visualizations/<zid> below the app directory and the working directory when
the run ends, however it ends, so repeated runs leave nothing on the
container's disk. The paths are built from the conversation id as an
integer; another conversation's files are never touched.

Tests: tests/test_run_delphi_shadow_switches.py, on the stub-stage fixture
of test_run_delphi_exit_codes.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: the image carries all-MiniLM-L6-v2 at a pinned revision, read only when asked (switched off)

The Delphi image's build fetches the embedding model's eleven loading files
from the Hugging Face hub once, at revision 1110a243fdf4706b3f48f1d95db1a4f5529b4d41
(the hub's main for sentence-transformers/all-MiniLM-L6-v2 today), and
refuses the build when any file's SHA-256 differs from
delphi/embedding-model.lock.json. The files (about 92 MB, weights as
model.safetensors only: no ONNX, OpenVINO, TensorFlow or PyTorch-bin copies)
and the lock are copied into the final stage under /opt/polis/embedding.

At run time the UMAP stage reads that directory, with local_files_only and
so no network, only when DELPHI_EMBED_MODEL_PATH names it; with
DELPHI_EMBED_MODEL_LOCK also set, the directory is checked file by file
against the lock first and a difference refuses the run. Unset (every
service today, and the image sets neither), the stage loads the model by
name from the hub exactly as before.

polismath/embedding_model.py has three commands: fetch (the build's),
verify (a directory against the lock) and compare, which encodes the same
texts with the model loaded by name (today's run) and loaded from the
image's directory and prints both SHA-256 digests of the float32 vectors;
it exits non-zero unless they are byte for byte equal.

.dockerignore lets the lock into the build context; the deploy-hook test
inputs carry the lock and .dockerignore so the Dockerfile checks run in the
test container. Tests: tests/test_embedding_model.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: every step of a run can report its seconds and peak memory (switched off)

With DELPHI_STEP_LOG=1 a full-pipeline run prints one line per step on
standard output:

  delphi_step/1 {"step":"evoc","status":"ok","seconds":4.218,"peak_rss_mb":1208.4,...}

Under the job daemon a child's standard output is stored as its attempt's
log lines (polis_queue_logs, keyed by the attempt id its run record names)
and goes to the container's log stream, so each run reports what each step
cost beside its run record. Steps: census and manifest (the daemon's input
and output records), pipeline (the UMAP stage as a child process, with its
exact peak from wait4 and its CPU seconds), inputs, embed_load (and whether
the model came from the image or the hub), embed_encode, umap, evoc (layers,
clusters per layer, whether it fell back to KMeans, and Numba's threading
layer), store_base, keywords, layer_pages, topic_names (one per layer),
datamapplot (one per layer, or one line saying it was skipped), total and
run. A step's peak memory is sampled from /proc every
DELPHI_STEP_SAMPLE_MS (default 20) milliseconds while it runs; each line
also carries the process's exact peak so far. Lines hold counts, seconds,
megabytes and closed labels: no comment, text or conversation id.

Off (unset, or anything but "1"), nothing is printed and every stage runs
as before: children run through subprocess.run as they did. The UMAP
stage's conversation processing is split into three functions (inputs,
the first writes, the rest) with no change to what it reads or writes.

Tests: tests/test_step_log.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs: polis-delphi-shadow admits free-only Delphi runs one at a time, paced (switched off)

A new binary beside polis-jobs, built into the Delphi image, off unless
DELPHI_SHADOW_ENABLED=1. It admits full-pipeline runs, through the queue's
own pd_writer_admit, of live conversations (a comment created or moderated
in the last DELPHI_SHADOW_LIVE_DAYS, default 30) whose comment set changed
since their last shadow run, under a result env no reader serves:
shadow-delphi-1 by default, and any env that does not start with "shadow-"
is refused at start, as is a QUEUE_ENV that differs from it. A polis-jobs
daemon claiming that env runs them; nothing here starts one.

What it admits. The comment set's digest is computed with the input
bundle's own function from the same columns, so it equals the
comment_set_sha256 the run's census records; an unchanged set is never run
again (and is counted: a rerun there would recompute what is stored). Sets
outside 20 to 5,000 comments are skipped, smallest first, at most 200 runs
a day. The idempotency key is the comment set itself, so a restart admits
nothing twice; the configuration names the set, its size class and the
scheduler, and is in the job's run record (config_effective.shadow).

How it is paced, as the math backfill is. One run at a time: nothing is
admitted until the last admission is terminal (or past
DELPHI_SHADOW_JOB_DEADLINE_SECONDS). After each run it rests for the duty
cycle (DELPHI_SHADOW_DUTY_CYCLE, default 0.5: as long as the run took).
Before each admission it pauses, and logs why, for the operator's pause
file, a disk below DELPHI_SHADOW_DISK_FREE_MIN_MB (10 GiB) or above
DELPHI_SHADOW_DISK_USED_MAX_PCT (85%), a database whose median SELECT 1
round trip is over DELPHI_SHADOW_PROBE_MAX_MS (250, resuming below half),
a math engine whose newest write trails the newest vote by more than
DELPHI_SHADOW_MATH_LAG_MAX_SECONDS (180), and the daily cap.

The participation path. The scheduler holds no pool and shares no
connection with the server or the Rust API: it opens one connection per
login for one call and closes it. Its reads (comments, math_main, the
newest vote's time; never a vote's value) are each one READ ONLY
transaction with its own statement and lock timeouts; on the queue login it
calls only the queue's functions, whose admission takes FOR KEY SHARE on
the conversation's row. tests/delphi_shadow.rs (jobs-integration, run by
ci/local/queue-rs.sh) holds the read transaction and an admission open
while a comment, the conversation update votes and comments cause, a vote
(a pass), the next comment's candidate read and the notification update
run under a 50 ms lock timeout on the same conversation, and samples
pg_stat_activity through whole cycles: at most one scheduler connection at
a time, none left open.

It writes one delphi_shadow/1 line per event (admitted, finished with the
run's attempt and whether the census digest matched, unchanged, paused,
resumed, idle) with job ids, counts, digests and closed words, and keeps a
small state file (written whole and renamed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* compose, deploy: the Delphi shadow's two services on the Delphi box, started only by their switch (off)

docker-compose.yml gains delphi-shadow (a polis-jobs daemon) and
delphi-shadow-scheduler (polis-delphi-shadow), both in the profile
delphi-shadow and both off: each process exits 0 at once unless
DELPHI_SHADOW_ENABLED=1, and the deploy hook starts them on the Delphi box
only when the environment document carries the line
DELPHI_SHADOW_ENABLED=1 exactly, after delphi and math-python, from the
Delphi image pulled by digest. A failure to start them is logged and never
fails the deploy. Without the line the hook's commands and its image
override are the same as before (the existing tests' UNCHANGED lines).

The daemon claims only DELPHI_SHADOW_ENV (default shadow-delphi-1), the
full pipeline only, one job at a time, with the input bundle and the
embeddings table on. It is free by construction: no LLM provider key, no
Anthropic model, no AWS key and no S3 setting is listed, so none reaches
the container whatever the environment document holds; topic naming falls
back to keyword labels under the model "none". It writes no visualization
page, reads the embedding model from the image offline, and prints its
step lines. Its scratch, Hugging Face and Numba caches and the pipeline's
local outputs are tmpfs (2 GiB, 512 MiB and 64 MiB, inside its memory
limit), cleaned per run; only the daemon's journal and the scheduler's
state file are on the volume delphi-shadow-state. Limits: 2 CPUs and 6 GiB
for the daemon, a quarter CPU and 256 MiB for the scheduler. Both use the
queue login and transport settings the delphi service already has; every
scheduler bound is forwarded with its default.

Tests: tests/test_compose_delphi_shadow.py (profiles, defaults off, no key
reaches either container from a production-like environment document, the
daemon's settings and limits, both name the same shadow env, the delphi
service unchanged) and three cases in tests/test_after_install_hook.py
(not started without the exact line, started after the engine with the
Delphi image when it is there, never on another role).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs: twin runs repeat a shadow run once on the same comment set and measure the difference (off)

With DELPHI_SHADOW_TWIN_EVERY=N (default 0, off), every Nth first shadow run
that succeeds is repeated once, on the same comment set, before any new set
is admitted. Before the twin is admitted the scheduler keeps the first
run's cluster assignments (from the shadow env's current rows) and reads
the comment set again; a set that changed in between drops the twin.

The twin's admission names the run it repeats, so its run record carries
it (config_effective.shadow.twin_of, beside the comment set's digest), and
both records carry every family's content_sha256: whether each family
repeated byte for byte is read from the records themselves. When the twin
succeeds the scheduler adds what digests cannot say, for the cluster
assignments, layer by layer: the adjusted Rand index between the two runs
(1.0 for the same partition, whatever the cluster numbers), the cluster
counts of each, and how many comments moved under the best one-to-one
matching of the two labelings. It logs one delphi_shadow/1 "twin" line with
both job ids, the families' equality and those measures, keeps the newest
500 in its state file, and counts compared and identical twins.

This is the measurement the clustering's run-to-run difference on today's
image needs before any change to the image or the seeding.

Tests: twin.rs (the textbook adjusted Rand value, relabelled partitions,
moved counts, family comparison) and a jobs-integration case (the twin goes
first, carries twin_of and the same digest in its run record, and is
dropped when the set changed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs: a twin is identical when its clustering is, not when every family is

Families whose rows carry the run's own job id or time (the topic names'
keys, the UMAP configuration's dates) differ between any two runs, so the
first rehearsal of twin runs reported every twin as different, including
twins whose assignments were byte for byte the same. A twin is now
identical when it has the same comments and layers, no comment moved in
any layer, and the assignments family's digest is not different; the
per-family equality is still reported as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: a rehearsal of the shadow on public fixtures, with the network sealed

delphi/tests/shadow/rehearsal.sh runs the repository's own delphi-shadow
and delphi-shadow-scheduler services (docker-compose.yml unchanged, with an
override for the network and the image) against a throwaway Postgres with
the release migration chain and public fixtures only: conversation 1424
(the biodiversity export with its votes, through the vote convention
module) and seed_public.py's comment-only conversations of 25 to 441
comments made from the two public exports. It also publishes a collective
statement under the env readers serve, before anything starts.

In order: the byte-for-byte check of the embedding model (the public
comments encoded by name from the hub and from the image's files); the
seal (Postgres moves to an internal network with no route out; proofs that
no internet host is reachable, that the model cannot be downloaded, and
that the image's model encodes offline, again from inside the running
daemon at its start and its end); both services, with twin runs on; then
one added comment and one moderation, and their reruns; then report.py:
every job's run record facts and step lines, which envs hold rows in each
queue and result table, whether the published pointer moved, and the
tables' sizes. The stack is removed on exit, volumes included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* deploy: the Delphi shadow starts after the role chain, so the Delphi role keeps its one start line

The deploy hook's Delphi role must have exactly one compose `up` line, the
one that starts delphi and math-python (tests/test_compose_math_env.py pins
it); the shadow's switch-gated start was a second one inside that branch.
It now follows the role chain as its own block. DELPHI_SHADOW_SELECTED is 1
only on the Delphi box and only for the exact switch line, so what runs
where is unchanged; the start is still non-fatal. The deploy-hook test's
constant for the shadow's start line is renamed: it had replaced the
existing SHADOW_UP of the web role's vote shadow tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: the shadow rehearsal checks its published port by the shared host-port rule

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: the image test looks for an ENV instruction setting the model switch, not the word in a comment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* deploy: switching the Delphi shadow off removes its containers by name

With its -f files named, Compose v2.40.0 (the version the boxes run) leaves
a profile-gated service's containers running through a `down` that does not
name the profile. So the Delphi box's `down` never removed delphi-shadow or
delphi-shadow-scheduler: with the switch line taken out of the environment
document, a shadow a previous deploy started kept running, on that deploy's
image, through every later deploy.

On the Delphi box with the switch off, the hook now names both services in
`rm -s -f` after the engine's start; with none there it does nothing, and a
failure is reported without failing the deploy. Switched on, the start's
--force-recreate replaces them as before; other roles never touch them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs: a twin across a deploy is dropped, not compared

A deploy recreates the scheduler and the daemon on the new image, and the
scheduler's state file keeps a pending twin through it. A first run and a
twin either side of a deploy (or a twin queued before one and run after it)
were compared as if they repeated one build, so their difference would have
been read as the clustering's nondeterminism.

A twin is now admitted only when its first run's record carries this
scheduler's code version, and compared only when both records do; otherwise
the scheduler writes a twin_dropped line (code_version_changed) and moves on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs, delphi: the shadow's tests name no real env and pass the vote-sign lint

The participation test's next-comment read wrote the rejected-comment value
as a literal beside a vote, which the vote-sign lint counts as a new sign
site, so the lint exits non-zero. It now names the moderation value as a
constant, as the Rust next-comment read does; the statement is unchanged.

The configuration test and the rehearsal's default named the result env a
production reader serves; they now use a neutral example name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs, deploy: the Delphi shadow's daemon and scheduler start only on a shadow env

The shadow's daemon is polis-jobs, which claims and publishes under any env
it is given, and its env came from one line of the environment document
(DELPHI_SHADOW_ENV) that nothing checked: that line naming the env a reader
serves would have made the shadow a second production daemon, claiming that
env's full-pipeline jobs and publishing under it.

The daemon's container now runs `polis-delphi-shadow daemon`, which refuses
to start (exit 2, one line, no value printed) unless QUEUE_ENV and
DELPHI_RESULT_ENV are the same env beginning shadow-, and only then
replaces itself with polis-jobs (exec, so it is the same process and the
daemon's signal handling and drain are unchanged). The scheduler already
refused such an env in its configuration; a test now runs both binaries as
the container starts them: a stand-in polis-jobs on PATH runs, as the same
process, only for a shadow env, and the scheduler exits 2 for a served one.
An argument the binary does not know is refused rather than taken as the
scheduler.

Before it starts the shadow, the deploy hook reads back the composed
configuration of both services (the image override included, with the
delphi-shadow profile): the daemon's queue env must be a shadow env, its
result env the same, its command the checked start, and the scheduler's
two envs that same env. jq prints the names of what is not so, never a
value. Refused, or unreadable, the shadow is not started and its
containers are removed by name; the Delphi box's own deploy goes on, as it
does when the shadow's start fails. Tests in the hook suite's style: the
read-back runs before the Delphi box replaces anything, each wrong setting
is refused, nothing is read back when the shadow is not selected, and the
real jq names exactly the wrong settings; the compose suite runs the
hook's program on docker-compose.yml as it is (it passes) and with the
environment document giving the shadow a served env (it is refused).

The production delphi service is unchanged: it never runs the new
subcommand, and the switch-off path is the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs, compose: the Delphi shadow stops admitting at a storage budget read from the database

Shadow rows cannot be deleted (result rows, families and embeddings are
immutable, and nothing prunes a shadow env), and nothing bounded how many
the shadow could add. The scheduler now holds them under a budget:
DELPHI_SHADOW_STORAGE_BUDGET_MB in all (1024, 1 GiB, by default) and
DELPHI_SHADOW_STORAGE_DAILY_MB per UTC day (100 by default; never more than
the budget). Before each admission it pauses, with a `paused` line naming
storage_budget or storage_daily and the bytes against each bound, once
either is reached, or when the next run's estimate (its comments times the
bytes a comment has measured per run, 8 KiB before any is measured, plus
16 KiB for the queue's own rows of a job) would pass either. A waiting twin
is held the same way and kept.

The bytes are read from the database, never by scanning a table on a
poll. Each run is read once, when it is terminal and before the next
admission, by one read-only statement on the read login whose every read
is an index range on that run's own keys: its attempts, and per attempt
(a result batch is an attempt) its result rows, families, log lines, and
the embeddings it first wrote. The bytes are modelled row by row from the
stored values (pg_column_size): the heap row's share of a page of rows its
size, or a small row and TOAST chunks when its values are moved out, and
each index entry at half a leaf, since every index of these tables is led
by the run's own random id. Against what the four tables really grew by
for a run of 12,000 result rows, 2,000 embeddings and 6,000 log lines, the
measure reads 4 to 7% over (per table 1.00 to 1.07); a test holds it within
10% under and 20% over, holds the indexes it models to the ones the
migrations create, and holds the plan to index ranges.

The total is kept in the state file; the file is a cache of the database,
not the record. A scheduler that starts without one lists every job of
every shadow- env from delphi_jobs once and measures each again, twenty per
poll, admitting nothing until it has (storage_unmeasured); a run whose
bytes cannot be read holds admission the same way. Tests: the ledger and
the pause order as pure functions; a fresh scheduler finding a 28 MB
shadow run already stored pauses at an 8 MiB budget, then at an 8 MiB
daily cap under a larger budget, keeps its count across a restart, and
with room admits, and reads the run's bytes before admitting another.

Compose passes both settings to the scheduler with these defaults. The
production delphi service and every switched-off path are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* deploy: tests that a Delphi shadow failing to start never fails the deploy, and never holds up the math engine's stop

The Delphi role names its services in DELPHI_SERVICES and ValidateService
requires exactly those; the shadow's two services are optional work and are
never named there. These tests pin that:

- a shadow whose start fails leaves the deploy exit 0, with the engine's
  start unchanged and the previous deploy's shadow containers removed
  before it;
- the real hook's DELPHI_SERVICES line names delphi and math-python only,
  and neither shadow service has a depends_on (starting them never starts
  anything else);
- switched on with math-python taken off the list (its durable stop): the
  hook removes math-python and the shadow's containers, starts delphi, then
  the shadow, and never starts math-python;
- ValidateService on the Delphi box with the shadow's containers absent
  passes, and never polls them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs, compose: the Delphi shadow's storage budget is 3 GiB in all and 300 MiB a day by default

DELPHI_SHADOW_STORAGE_BUDGET_MB defaults to 3072 (was 1024) and
DELPHI_SHADOW_STORAGE_DAILY_MB to 300 (was 100), in the scheduler and in
the Compose service, so the first pass can cover the live conversations and
a generous set of others. Either is still set by the environment document;
a day may still not hold more than the whole budget. The default tests (the
scheduler's config and the Compose environment) say the new numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs, compose: the Delphi shadow also runs a listed set of conversations and every large one, whatever their activity

The scheduler picks from three sources, in priority order, each still held
to the comment bounds, the rerun window, the pacing and the storage budget:

1. An important-conversations list, DELPHI_SHADOW_IMPORTANT: conversation
   codes (zinvites.zinvite, the code in a conversation's URL) separated by
   commas, highest priority first, empty by default. It is set only in the
   environment document; nothing names a conversation in the repository. An
   entry that is not a code (1-64 letters and digits) refuses the start by
   its position, never its text; a repeated code keeps its first place; at
   most 10,000.
2. Every conversation with at least DELPHI_SHADOW_LARGE_PARTICIPANTS
   participants (200 by default; 0 turns it off), most participants first,
   from conversations.participant_count (kept by the participant trigger).
3. The live conversations whose comment set changed, smallest first, as
   before.

The first two are read as one roster, in one read-only transaction, at the
start and once each UTC day (a failed read keeps the roster there is and is
retried after DELPHI_SHADOW_ROSTER_RETRY_SECONDS, an hour by default): the
codes through zinvites' unique index, the large conversations by one pass
over conversations (that column has no index), each with its comment count
by the comments' zid index. No poll reads either again: between reads each
roster entry takes its fresh count and newest change from the live read the
poll already makes, so a listed conversation that changes is seen at the
next poll. A conversation in two sources is tried once, at its first.

The first candidate whose run does not fit the budget holds admission, so a
listed or large conversation keeps its place rather than letting smaller
ones go first. One whose run alone would pass a whole day's cap is never
tried (too_costly), so it cannot hold back the others every day. The log
carries counts only (roster, idle) and each admission's source (selected:
important, large, live or twin); the state file counts the admissions from
the first two.

The participation path: the new reads are ACCESS SHARE in the scheduler's
read-only transaction, on a connection opened for it and closed after,
which neither waits on nor makes wait the conversations updates that votes
and comments cause, nor the zinvites read a participant's request makes.
The isolation test now holds both reads open across the vote, comment and
next-comment statements, and reads the roster again in each of its whole
cycles. Its connection count is taken in the test's own database: with two
test threads another test's scheduler was counted beside it.

Tests: the list's parsing and refusals; the three sources' order, once each,
with an unchanged set skipped and a changed one first again; the roster's
overlay; a run too costly for a day skipped; against Postgres, the list,
then large conversations, then live ones admitted in that order, a listed
conversation that changes after the roster read admitted at the next poll,
and a conversation that grows past the threshold waiting for the next day's
roster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* queue-rs: the important list's refusal test names what was accepted instead of unwrapping

The workspace's clippy denies unwrap_err() in tests; the two refusals now
read the error with let-else and say which entry was accepted if one ever
is. The assertions are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: the hook test's fake compose keeps the failed Rust start as its last line

The fake docker-compose exits with its last line's status, which is how a
failed polis-api start reaches the hook. The Delphi shadow's start-failure
line had been appended after it, so a failed Rust start exited 0 and three
of the Rust start tests failed. The shadow's line now comes before it, and a
comment says why the order matters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* deploy: the Delphi shadow's start block names the Delphi role, so the web upstreams check reads it as Delphi-only

The check of the web boxes' startup (file-server/nginx/production_upstreams.py)
reads every block of the deploy hook that a role condition does not rule out
as a possible web start. The shadow's start was guarded only by
DELPHI_SHADOW_SELECTED, which is 1 only on the Delphi box, so the check read
its `up ... || echo` line as a web start and refused the `||`. Rewriting the
line as `if ! ...; then ...; fi` is not enough: the check then counts the
shadow's two services as started on a web box and refuses the mismatch with
the web images.

The block's condition now names the role as well:
`[ "$SERVICE_FROM_FILE" == "delphi" ] && [ "$DELPHI_SHADOW_SELECTED" = 1 ]`.
DELPHI_SHADOW_SELECTED is already 0 on every other box, so the hook does
exactly what it did; the check sees a Delphi-only block, and the line keeps
its `|| echo`, so a failed shadow start still never fails the deploy. The
check itself is unchanged: it goes on refusing anything on a web path it
cannot read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi shadow: no daily storage limit by default, and 1 GiB of the budget kept for live conversations

The scheduler's daily storage limit (DELPHI_SHADOW_STORAGE_DAILY_MB) is
now off unless set: unset, empty or 0 means no daily limit, and the log
lines say "no daily limit" where they gave the bytes. Set above 0 it holds
as before, for every source. Compose passes 0.

Of the total (DELPHI_SHADOW_STORAGE_BUDGET_MB, 3072 by default),
DELPHI_SHADOW_LIVE_RESERVE_MB (1024 by default, 0 keeps none, never more
than the total) is kept for the live conversations. The important list,
the large conversations and twin runs stop at the total less the reserve
(2048 MB by default), pausing as storage_reserve; live runs go on and stop
only at the whole total. A listed or large conversation held by the
reserve alone holds the rest of those two sources, and the live ones are
still tried. A run that alone could never fit its share is never tried, as
before with the daily limit. The log lines carry the reserve in bytes.

The total, the run count a day, the pacing (database latency, math lag,
disk floor, pause file) and the selection are unchanged.

Tests: no daily limit by default and a set one still holding (config,
plan and the scheduler against the database's own bytes); the first two
sources stopping at the total less the reserve while live runs continue,
and live runs stopping at the total (plan and the scheduler); Compose's
defaults and pass-through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi shadow: three named sets, benchmark, featured (open data) and current, each run naming its set

The scheduler now selects from three named sets, in priority order:

- benchmark: the configured list of conversation codes, renamed from
  DELPHI_SHADOW_IMPORTANT to DELPHI_SHADOW_BENCHMARK (same rules: codes
  only, in priority order, empty by default, set only in the environment
  document);
- featured: every open-data conversation (conversations.is_data_open, the
  owner's "anyone can export a data dump" setting), most participants
  first;
- current: the live conversations whose comment set changed, as before.

The participant-count source is removed with its setting. The scheduler
refuses to start, naming the setting and never its value, if either
DELPHI_SHADOW_LARGE_PARTICIPANTS or the list's old name is set, so a stale
line is never silently ignored; Compose passes neither.

The open-data read replaces the participant-count read in the roster: one
pass over conversations (no index on the column) in the same read-only
transaction as the benchmark codes, at the start and once each UTC day,
never per poll, ACCESS SHARE only.

The live reserve now protects the current set: benchmark and featured
together (and twins) stop at the total less the reserve, current runs at
the whole total. A conversation in two sets runs once, in the first.

Every log line about a run (admitted, refused, unchanged, finished,
storage, storage_unread and the twin lines) carries "set": benchmark,
featured or current (a twin, its first run's), and so does the run
record (config_effective.shadow.set). The roster and idle lines count
each set. The set is not in the admission key: one comment set runs once
whichever set selects it. A state file an earlier build wrote still loads.

Tests: the order, each set once, a benchmark that is also open data
counted as a benchmark, the reserve by set, the set words and the
admission naming its set (plan); the renamed list and the retired
settings (config); an earlier state file (state); against Postgres 17 with
the release chain, the order and each run record's set, a conversation of
900 participants that is not open data never running until its data is
opened and then only at the next day's roster read, the reserve holding
benchmark and featured while current runs go on, a twin carrying its
first run's set, a job listed again from the database keeping its set, and
the participation-path test with the open-data read held open; Compose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@colinmegill
colinmegill marked this pull request as ready for review November 15, 2026 22:32
colinmegill and others added 5 commits November 16, 2026 00:37
* cdk: the Delphi box's root volume is 200 GiB

The Delphi small launch template's gp3 root volume goes from 50 to 200 GiB.
That disk holds every image the box pulls (the deploy hook pulls the new
release's images before it removes the previous release's, so two sets are
on disk while a deploy runs), and the Delphi shadow's scheduler stops
admitting work under 10 GiB free. Between deploys the disk is about 70 %
used; it has filled once.

The synthesized template differs from edge in that one VolumeSize. The
group has no rolling or replacing update policy and takes the template's
latest version number, so a cdk deploy writes a new launch template version
and changes no running instance: the box running now keeps its 50 GiB
volume until it is replaced or its volume is grown in place. cdk/README.md
says how to grow it in place. No schema change.

The stack template snapshot also takes the log group's 365-day retention,
which the stack has declared since the change that made the server log
group keep a year; the snapshot was not updated then and failed on edge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* cdk: the README's grow-in-place note names no volume and checks the root first

The note on growing the Delphi box's root volume in place now gives the
volume as a placeholder to fill with the verified root volume's id, says
the host commands run only once the modification is optimizing or
completed and the root is verified to be XFS directly on partition 1, the
last partition on the disk, and says precisely which update policy the
group lacks: one that replaces instances. It claims no restart or planned
outage, nothing more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* cdk: the stack template snapshot keeps edge's log retention line

The volume-size commit also set the stack template snapshot's log group
retention from 30 to 365 days, the value the stack has declared since the
server log group was made to keep a year. That stale line is a separate
defect that predates this branch, and is handled on its own. This restores
it to edge's value, so the branch changes only the Delphi small launch
template's volume size in the snapshot. As on edge, the stack template
snapshot test fails on that one retention line until it is repaired.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ce (used by nothing yet) (#3430)

* topic map: the generator crate joins the queue-rs workspace, with its datamapplot parity tests and two public fixtures

queue-rs/polis-topic-map is the generation-time half of a first-party topic
map: from a Delphi run's stored UMAP positions, cluster layers and topic
names it computes everything about the map that depends only on the run
(the standard size, the first view, the point radius, each topic's label
anchor and weight, its parent one level up, and both datamapplot palettes)
and writes it as one JSON document, polis-topic-map/1. A library (build)
and a command (polis-topic-map data | page). Nothing calls it yet: no
route, no pipeline stage, no stored data.

The source is the prototype's, formatted by rustfmt, with lint allowances
on six lines (three index loops over parallel arrays, two unwraps on values
that cannot be empty, one expect on the crate's constant colour table) so
the crate can deny unwrap and expect as the other workspace members do.
serde_json is asked for float_roundtrip, so numbers are read the same way
whichever crate it is built with (polis-api turns it on for a whole
workspace build).

Tests: tests/datamapplot_parity.rs (the port against the values
datamapplot 0.6.4 and colorspacious computed for the same runs) and
tests/document_recording.rs (the whole document's length and SHA-256 for
each fixture, from the library and from the command; the recorded values
are what the prototype wrote for the same inputs).

The public fixtures are a Delphi run over two compdemocracy/openData
conversations (CC BY 4.0), bg2050-volunteers and
scoop-hivemind.biodiversity: comment ids, positions, clusters, topic names
and datamapplot's own values, 1,759,374 bytes in six files, no comment text
and no vote. NOTICE carries datamapplot's MIT licence, the CET-C2s colour
map's CC BY 4.0 attribution and the fixtures'.

No new external package: Cargo.lock gains only the crate's own entry
(serde, serde_json and, for the tests, sha2 are already locked).

Co-Authored-By: Claude <noreply@anthropic.com>

* ci: make check-polis-topic-map runs the topic map generator's formatting, clippy and tests, and hosted CI calls it

ci/local/polis-topic-map.sh runs, in queue-rs, cargo fmt --check, cargo
clippy --all-targets -D warnings and cargo test for polis-topic-map, the
same three steps the other single-crate entry points run for theirs. It
needs the pinned toolchain and nothing else: no database, no Docker, no
network beyond fetching the locked crates.

make check-polis-topic-map calls it, docs/local-checks.md lists it, and
the queue-rs workflow has a new job, polis-topic-map, whose only step is
that make target. The workflow's path filters already cover every file the
check reads (queue-rs/**, ci/local/**, Makefile).

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
…ent package (no page uses it yet) (#3431)

* client-topic-map: the topic map, the topic list and their neutral theme become one package the clients can share; no client uses it yet

A folder beside the clients, client-topic-map/, holds the first-party
topic map as plain browser scripts with no dependencies and no build
step: topic-map.js (PolisTopicMap, one canvas, no library), topic-list.js
(PolisTopicList, the same topics as a tree) and theme-neutral.js (every
colour, type size, stroke, spacing and duration they use). The three
files are unchanged from the prototype they come from. Each works as a
classic script and as a CommonJS module.

The map measures and draws topic names on its canvas in the theme's
face, and which names show is decided from the shapes of their letters,
so the package carries the face: Roboto, variable weight, latin subset
(37,520 bytes, SIL Open Font License 1.1, with its licence), and
fonts/roboto.css, the @font-face rule that declares it.

Nothing is observable: no page, client or build imports the package.

Tests, with Node's own runner:
- geometry.test.js: the prototype's ten tests of the pure rules,
  unchanged except that the wrapping test reads its public fixture from
  test/fixtures/ (the topic names and datamapplot's label strings for
  two public conversations, 20,388 bytes);
- api.test.js: the methods and the three events (topicselect,
  topichover, levelchange) with their details and origins, on a public
  fixture document (29,111 bytes);
- self-contained.test.js: no script or style sheet of the package names
  a URL with a scheme or opens a request, and the style sheet's one URL
  is the font file beside it.

NOTICE names what the code is derived from or carries: datamapplot,
deck.gl and mjolnir.js (MIT), Python's textwrap (PSF License), Roboto
(OFL 1.1) and the public fixtures (CC BY 4.0, compdemocracy/openData).
README.md says how each client's build would take the package in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: make check-client-topic-map runs the topic map package's tests, locally and in hosted CI

ci/local/client-topic-map.sh checks for Node 24.22.1, as the other
client checks do, and runs node --test over client-topic-map/test. The
Makefile target calls it, a new workflow calls the Makefile target on
pull requests that touch the package, ci/local or the Makefile and on
pushes to edge, and docs/local-checks.md lists it. The package has no
dependencies, so nothing is installed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client-topic-map: the notice for the port of Python's textwrap names its source, Python 3.12, and keeps that source's copyright notices

The port of textwrap in topic-map.js (WORDSEP and wrapLabel) was made
from Lib/textwrap.py of Python 3.12, the Python that runs datamapplot in
the Delphi image; that file is the same in every 3.12 release from
v3.12.0. NOTICE now names that revision, keeps the module's own notices
(Gregory P. Ward 1999-2001, the Python Software Foundation 2002 and
2003), and carries the Python Software Foundation's notice and licence
exactly as Python 3.12's LICENSE gives them; before, it gave a shortened
notice taken from a later LICENSE. The summary of changes also names one
more: in the second look-behind for a hyphenated word, the character
after the first hyphen is matched by the class of word characters and
punctuation, where Python matches a letter.

Only NOTICE changes; the scripts are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… dimensions); one row in delphi_embedding_models (migration 000046) (#3428)

* migrations: granite's row in delphi_embedding_models, so Delphi can store its embeddings beside MiniLM's (migration 000046)

One data row in an existing table: model_id 2,
'ibm-granite/granite-embedding-311m-multilingual-r2@44399559930365213510b1ee2eb15ded83374f0e:384',
384 dimensions. The name is the one a Delphi run records for that model at
that revision, kept to its first 384 dimensions. No table, column, index,
constraint, function, trigger or grant changes; model_id 1 and every stored
embedding and set stay as they are.

Without the row, a run with the per-row embeddings table on (the Delphi
shadow) cannot store granite's vectors: pd_embedding_put and
pd_embedding_close look the model up by name and raise "unknown embedding
model".

Locks: the INSERT takes ROW EXCLUSIVE on delphi_embedding_models until
commit, milliseconds. That conflicts only with SHARE and stronger locks,
which no reader or writer of the table takes. Nothing on the vote, comment
or next-comment path reads the table.

Declared additive in compat.txt and docs/migrations.md. Reversal:
down/000046_remove_granite_embedding_model.sql deletes the row only while
nothing is stored under model_id 2, with the row's immutability trigger
disabled inside that one transaction and enabled again before commit.

The first-deploy report, the forward proof's additive ledger and the
coordinator's schema census include it; the census, measured on PostgreSQL
17.11, finds no table or column change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: comments are embedded with granite-embedding-311m-multilingual-r2, kept to 384 dimensions

all-MiniLM-L6-v2 is English-only. On other languages its nearest
neighbours mostly do not share a comment's topic, and a conversation that
mixes languages clusters by language. granite-embedding-311m-multilingual-r2
(Apache-2.0, revision 44399559930365213510b1ee2eb15ded83374f0e) clusters
such conversations by topic. It is 768 wide and Matryoshka-trained; Delphi
keeps the first 384 dimensions, the width it has always stored, and
normalizes them.

- The image fetches granite by delphi/embedding-model.lock.json (ten files,
  each SHA-256 checked) into
  /opt/polis/embedding/granite-embedding-311m-multilingual-r2: about 627 MB,
  where MiniLM was 92 MB.
- polismath/embedding_model.py names a run's model once:
  repo@revision:384 from the lock for the image's directory, and the same
  for the default model by name, which is pinned to the lock's revision. A
  directory without its lock, or a SENTENCE_TRANSFORMER_MODEL that names
  another model than the directory holds, refuses, so vectors are never
  stored under another model's name. Every load truncates to 384 and every
  encode normalizes.
- The pipeline stores under that name: in the per-row embeddings table
  (granite is model_id 2) and in the DynamoDB rows' embedding and
  conversation labels. The visualisation's served-set read asks for the
  deployment's model instead of MiniLM. The importer maps granite's name to
  its row, and still refuses a conversation whose rows come from two models.
- The Delphi shadow reads granite from the image, offline; the delphi
  service's default name is granite.
- The job graph's embedding stage is unchanged: it declares, checks and
  stores MiniLM, and refuses vectors that are not 384 wide, which granite's
  untruncated output is not.

Vectors from the two models are not comparable. Nothing stored moves; a
conversation run again under granite gets a new map, new clusters and new
topic names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: the run record names the embedding model by repository, revision and width

The full pipeline's run record wrote models.embed from
SENTENCE_TRANSFORMER_MODEL: no revision, no truncation, and nothing checked
it against the directory the pipeline loaded. It now writes the name
polismath/embedding_model.py resolves from the same environment and lock the
pipeline loads with, as the stores record it:

  ibm-granite/granite-embedding-311m-multilingual-r2@44399559930365213510b1ee2eb15ded83374f0e:384

so two runs can be compared by the exact model they used. A model that
cannot be named fails the manifest instead of recording a guess.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: the default embedding model loaded by name is checked against the shipped lock first

With DELPHI_EMBED_MODEL_PATH unset, the default model was handed to
SentenceTransformer by its hub name and revision, and nothing compared the
files it read with the lock's SHA-256s. The run record still named
repo@revision:384, a requested revision rather than checked bytes.

Now, by name, the default model's files are fetched once, at the lock's
revision, into a directory under the Hugging Face cache. A first fetch goes to
a private directory and takes the shared name only after it verifies. Every
load checks that directory against the lock and builds the model from it with
no network, as the image's directory is loaded. A changed byte, a missing or
extra file, or another revision's files refuse the run before any model is
built or any embedding written.

The shipped lock was looked up beside the package (/app in the image), where
the image does not carry it; in the image the module may also be imported
from site-packages. It is now found beside the package in a checkout, else
where the image installs it (/opt/polis/embedding/...lock.json).

A model other than the default, named on purpose, is still loaded from the
hub as given, unchecked, and recorded under the name as given.

The tests drive load() with the directory variables absent, replacing only
the hub download and the model constructor: a valid snapshot loads; a
one-byte change as served, or in the stored copy, refuses before
construction; another revision's files refuse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* compose: the delphi service reads the embedding model baked into its image

The delphi service named the model only, so every new container downloaded
it from the model hub on its first run (about 658 MB for granite). It now has
the directory and lock the Delphi shadow reads, the copy baked into the image:
each load checks the files against the lock and reads them with no network,
and nothing is downloaded.

Both are written as the service's other model settings are, with a default
an environment document may override:
DELPHI_EMBED_MODEL_PATH=${DELPHI_EMBED_MODEL_PATH:-/opt/polis/embedding/granite-embedding-311m-multilingual-r2}
and its .lock.json beside it. A SENTENCE_TRANSFORMER_MODEL that names another
model now refuses the run unless both are overridden to that model's
directory and lock.

The test that pinned the directory's absence from this service keeps the
shadow's own run settings out of it; a new test pins the baked directory and
lock (equal to the shadow's) and their override. Configuration docs and
example.env say how the service gets its model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: the stored-copy test seeds the cached model itself before changing a byte

The test that a stored copy of the default model, changed by one byte, is
refused took its directory from a first load(). Before the check existed that
load never made a directory, so the test failed in its own setup with
FileNotFoundError rather than on the refusal it is there to pin.

It now writes the locked files where a by-name fetch keeps them
(<HF_HOME>/polis-embedding/<name>@<revision>) without going through the code
under test, changes one byte, and expects load() to refuse with nothing
fetched and no model built. Without the check it fails with DID NOT RAISE
ModelMismatch; with it, it passes. No product code changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* delphi: tests that run run_delphi.py name the default model whatever the image sets

The writers check runs its Python tests in the job graph's test image
(delphi/tests/dynamo_removal/Dockerfile), which points DELPHI_EMBED_MODEL_PATH
at a MiniLM directory, with no lock, for the graph's embedding stage. Three tests
there run run_delphi.py, whose run record now names the model from those
settings and refuses a directory without its lock, so each exited 5:
test_actual_pipeline_orchestration_uses_private_reset_and_v2_manifest,
test_daemon_mode_leaves_stdout_exactly_as_the_legacy_path_prints_it and
test_the_retired_stages_never_run_on_either_path. The Delphi suite's own image
sets none of these, so it passed there.

These runs mean the default model by name. The fixtures behind them
(app_dir in test_job_child_protocol.py, writer in test_delphi_writer.py) now
clear SENTENCE_TRANSFORMER_MODEL, DELPHI_EMBED_MODEL_PATH and
DELPHI_EMBED_MODEL_LOCK, as one test already did. The refusal and every
assertion are unchanged, and the graph stage keeps its MiniLM directory; the
test image's Dockerfile says why it sets it.

Comments and docs that still described MiniLM as the model a run downloads:
the two run-record conformance harnesses, the polis-contracts README and the
job queue schema's example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: the model setting's census rows and example.env say what each Delphi service does

docs/configuration-env-reference.md: the SENTENCE_TRANSFORMER_MODEL rows
pointed at lines this change had moved (converter.py :149 and :179 are now
:152 and :185; embedding_model.py reads it at :130 and :168, not :158), and
one row named 500_generate_embedding_umap_cluster.py, removed in September.

example.env: naming another model alone is refused by the delphi service,
whose directory holds the default model, but a queue worker box's daemon,
which has no directory set, loads it by name, unchecked. The comment said
only the first.

Documentation only; no test or check reads either file.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@colinmegill colinmegill changed the title Promote edge to stable: Polis 2.43.11 Promote edge to stable: Polis 2.44.0 Nov 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor Author

Delphi Coverage Report

File Stmts Miss Cover
polismath/init.py 2 0 100%
polismath/benchmarks/bench_pca.py 128 107 16%
polismath/benchmarks/bench_repness.py 81 65 20%
polismath/benchmarks/bench_update_votes.py 38 28 26%
polismath/benchmarks/benchmark_utils.py 34 0 100%
polismath/components/init.py 1 0 100%
polismath/components/config.py 165 70 58%
polismath/conversation/init.py 2 0 100%
polismath/conversation/conversation.py 1293 264 80%
polismath/conversation/group_vote_counts.py 38 2 95%
polismath/conversation/manager.py 131 42 68%
polismath/database/init.py 1 0 100%
polismath/database/dynamodb.py 392 176 55%
polismath/database/postgres.py 414 95 77%
polismath/delphi_storage/init.py 1 0 100%
polismath/delphi_storage/codec.py 295 22 93%
polismath/delphi_storage/embedding_spool.py 58 6 90%
polismath/delphi_storage/golden_corpus.py 60 9 85%
polismath/delphi_storage/import_report.py 51 3 94%
polismath/delphi_storage/legacy_import.py 171 16 91%
polismath/delphi_storage/postgres.py 49 22 55%
polismath/delphi_storage/production_import.py 613 131 79%
polismath/delphi_storage/resource.py 179 32 82%
polismath/delphi_storage/scan_pacing.py 77 0 100%
polismath/delphi_storage/writer.py 196 13 93%
polismath/embedding_model.py 182 47 74%
polismath/empty_output.py 29 2 93%
polismath/engine_adapter.py 317 265 16%
polismath/job_child/init.py 364 51 86%
polismath/job_child/census.py 74 9 88%
polismath/job_child/input_bundle.py 101 7 93%
polismath/pca_kmeans_rep/init.py 5 0 100%
polismath/pca_kmeans_rep/clusters.py 257 38 85%
polismath/pca_kmeans_rep/corr.py 98 17 83%
polismath/pca_kmeans_rep/group_k_smoother.py 28 0 100%
polismath/pca_kmeans_rep/legacy_kmeans.py 197 7 96%
polismath/pca_kmeans_rep/pca.py 162 19 88%
polismath/pca_kmeans_rep/repness.py 226 5 98%
polismath/poller/init.py 4 0 100%
polismath/poller/admission.py 318 15 95%
polismath/poller/backfill.py 936 32 97%
polismath/poller/capacity.py 485 27 94%
polismath/poller/capacity_queue.py 248 13 95%
polismath/poller/coordinator_bridge.py 317 317 0%
polismath/poller/math_writer.py 113 3 97%
polismath/poller/promotion.py 112 4 96%
polismath/poller/readiness.py 368 13 96%
polismath/poller/rebuild_child.py 133 5 96%
polismath/poller/service.py 756 54 93%
polismath/poller/worker_pool.py 140 0 100%
polismath/queue/executor.py 221 111 50%
polismath/regression/init.py 4 0 100%
polismath/regression/clojure_comparer.py 188 20 89%
polismath/regression/comparer.py 891 640 28%
polismath/regression/datasets.py 126 27 79%
polismath/regression/recorder.py 36 27 25%
polismath/regression/utils.py 130 85 35%
polismath/replay/init.py 2 0 100%
polismath/replay/attribution_capture.py 49 0 100%
polismath/replay/axis_continuity.py 347 24 93%
polismath/replay/certify.py 999 59 94%
polismath/replay/crosslang.py 128 2 98%
polismath/replay/decision_ties.py 51 8 84%
polismath/replay/diagnostics.py 50 7 86%
polismath/replay/driver.py 73 1 99%
polismath/replay/dsn_admission.py 36 1 97%
polismath/replay/equiv_query.py 10 0 100%
polismath/replay/event_ingress.py 95 1 99%
polismath/replay/fixture_bundle.py 815 62 92%
polismath/replay/fixture_config.py 266 11 96%
polismath/replay/fixture_extract.py 285 12 96%
polismath/replay/fixture_generate.py 103 1 99%
polismath/replay/fixture_samples.py 157 14 91%
polismath/replay/fixture_selection.py 119 0 100%
polismath/replay/fixture_survey.py 147 11 93%
polismath/replay/legacy_pca.py 29 17 41%
polismath/replay/polarity.py 319 25 92%
polismath/replay/poller_equiv.py 776 99 87%
polismath/replay/prodclone.py 193 2 99%
polismath/replay/real_data.py 146 10 93%
polismath/replay/schedule.py 205 10 95%
polismath/replay/served_math.py 159 4 97%
polismath/replay/shard_bench.py 189 65 66%
polismath/replay/stagecompare.py 890 161 82%
polismath/replay/stages.py 256 40 84%
polismath/replay/stepcompare.py 86 1 99%
polismath/replay/store.py 120 11 91%
polismath/replay/tie_capture.py 107 19 82%
polismath/replay/types.py 58 6 90%
polismath/run_math_pipeline.py 284 93 67%
polismath/step_log.py 95 7 93%
polismath/tools/engine_rebuild.py 230 11 95%
polismath/types.py 6 0 100%
polismath/utils/cli_flags.py 15 0 100%
polismath/utils/clj_hash.py 44 0 100%
polismath/utils/env_flags.py 13 0 100%
polismath/utils/general.py 63 39 38%
polismath/utils/output_profile.py 66 2 97%
polismath/utils/serialization.py 10 5 50%
polismath/utils/vote_convention.py 159 5 97%
umap_narrative/700_datamapplot_for_layer.py 514 470 9%
umap_narrative/701_static_datamapplot_for_layer.py 310 310 0%
umap_narrative/702_consensus_divisive_datamapplot.py 435 275 37%
umap_narrative/801_narrative_report_batch.py 657 209 68%
umap_narrative/803_check_batch_status.py 268 148 45%
umap_narrative/llm_factory_constructor/init.py 2 0 100%
umap_narrative/llm_factory_constructor/model_provider.py 232 106 54%
umap_narrative/narrative_data.py 178 55 69%
umap_narrative/numerical_stages.py 97 17 82%
umap_narrative/polismath_commentgraph/init.py 1 0 100%
umap_narrative/polismath_commentgraph/cli.py 268 268 0%
umap_narrative/polismath_commentgraph/core/init.py 3 3 0%
umap_narrative/polismath_commentgraph/core/clustering.py 108 108 0%
umap_narrative/polismath_commentgraph/core/embedding.py 104 104 0%
umap_narrative/polismath_commentgraph/lambda_handler.py 219 219 0%
umap_narrative/polismath_commentgraph/schemas/init.py 2 0 100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py 80 6 92%
umap_narrative/polismath_commentgraph/tests/conftest.py 17 17 0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py 74 74 0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py 55 55 0%
umap_narrative/polismath_commentgraph/tests/test_storage.py 87 87 0%
umap_narrative/polismath_commentgraph/utils/init.py 9 7 22%
umap_narrative/polismath_commentgraph/utils/converter.py 283 50 82%
umap_narrative/polismath_commentgraph/utils/group_data.py 332 279 16%
umap_narrative/polismath_commentgraph/utils/storage.py 615 486 21%
umap_narrative/reset_conversation.py 162 53 67%
umap_narrative/run_pipeline.py 391 228 42%
umap_narrative/topic_naming.py 148 11 93%
Total 25937 7454 71%

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant