feat(engine): replace the ablation scoring mode with a checked, digest-bound one

[monotophic]

Authored by Claude Opus in Claude Code, analysis in partnership with @monotophic

ABLATE_SCORE=<manifest> replaces the decode with a teacher-forced sweep: for
each item it configures the ablation cells, runs one prefill, and reads the
final logits at every target position. That mode already exists on dev. What
it cannot do is produce evidence anyone can check. It accepts whatever a
permissive whitespace scan can make of a line, skips lines it cannot parse and
carries on, silently truncates an over-long cell list, and writes records naming
neither the config nor the manifest they came from. A partial run looks exactly
like a complete one — it exits 0 after writing nothing but a header.

This PR replaces that mode's implementation, and the change is visible to
anyone parsing its output. The artifact's schema goes from coli-ablate/1 to
coli-ablate/2, the mode's exit status is now its own result instead of a
constant 0, and inputs the old mode would have partly executed are now refused
as a whole. docs/ENVIRONMENT.md describes the mode as it now behaves. We chose
to replace the previously-implemented ablation mode rather than retaining it and
adding an alternative. I think that replacement is the cleaner/more-correct
choice, but understand if you object to that approach. Let me know if you want
the old ablation mode back in.

What the new mode gives you: the manifest is a defined format with an exact
field count and single-space separators, so the same content always produces the
same digest; every field is bounded against the loaded config before anything
runs; anything outside that is refused as a whole, naming the record at fault,
including a duplicate item id. One item may declare at most a million tokens, so
a corrupt or hostile manifest cannot ask the loader for an arbitrary allocation.
A CRLF terminator and a missing final newline are both accepted and normalised —
a host editor produces them without meaning to — and all three framings of the
same content bind the same digest. The artifact gains the digests of the config
and the manifest, the item and target counts it expects, and a terminal record
carrying the counts actually completed, so a truncated file is detectable.
Positions whose logits are not finite are refused by name instead of being
written out as unusable numbers, the top-k list holds real vocabulary entries
instead of padding a fixed width with sentinels, and the output path is created
exclusively, never truncated. Every refusal says why on stderr before the mode
returns non-zero.

The first commit adds the two small foundations that make the above checkable: a
classified reduction of a logit row (logprob_row_checked), which reports why
a row was unusable rather than only that it was, so a mode can refuse a position
by name instead of writing a NaN a reader cannot tell apart from a measurement;
and a short self-contained SHA-256 (evidence_digest.h) so a mode can name the
exact bytes it consumed without the engine taking on a dependency it otherwise
never needs. The plain sampling path is untouched, and the new path is pinned to
agree with it on every well-behaved row.

The reported numbers are unchanged. The negative log-likelihood and the
partition function are computed exactly as before, in double, and are only
printed at full precision now — a reader comparing old and new files sees more
digits of the same value.

Context. This PR is one of four independent contributions derived from a
single locally-verified working tree (offline evidence tooling, this engine
mode, two engine-transcript check tools, and a raw wire-format adapter). This
one carries the engine block: the ablation scoring mode, the row-status and
digest headers, and their tests. It uses one file from the evidence-tooling PR
(#1355, #1355): the round-trip test here runs the real producer against
that PR's tools/check_ablate_evidence.py, so until that PR lands, this
branch's engine build, its two C test binaries and the mode's dispatch checks
all stand, and that one round-trip case fails rather than skips — deliberately,
because a silent skip there would read as a pass on the only check that the
producer and the checker still agree. The others are proposed separately, each
with its own evidence.

Behavioral contract

• No default decode path changes what it computes: an unchanged binary and this
  one produce the same output, token for token, on the same container.
• The mode runs only when ABLATE_SCORE is set, and no other path reaches it.
• Every rejected input is rejected by name, on stderr, before a non-zero return;
  no input is partly executed.
• The reported nll and partition function are bit-for-bit what dev computes.
• Producer output passes an offline checker written independently of the engine,
  for each manifest framing the engine accepts, and all three bind one digest.
• Everything that worked before still works: the branch builds and the existing
  suites pass with the other three PRs of the set absent.

Structural changes on the default load path. The mode's digest is computed
only when ABLATE_SCORE is set, so the load path does no new work on an
ordinary run. Eight things about it are nonetheless different from dev, and
you should not have to find them yourself:

1. two new unconditional includes, <stdarg.h> and <inttypes.h>;
2. a new unconditional #include "evidence_digest.h" — a new file whose
   functions are plain static;
3. Cfg gained char config_sha256[65], so every Cfg in the process grows;
4. cfg_root gained a third parameter;
5. the digest call inside cfg_root runs only when a caller passes a buffer;
6. load_cfg reads one environment variable on every call to decide whether to
   pass that buffer — one getenv per model load;
7. run_score's prefix probe passes NULL for the new parameter;
8. sample.h gained three static inline functions and two types; that header
   is compiled into olmoe.c as well, where they are unused.

Capstone matrix — one decisive artifact per claim.

claim	decisive evidence
decode is unchanged on a real model, CPU	two servers on one Linux host, one built from dev (e1efc68, binary SHA-256 6d90bfa4…) and one from this branch (58b1da45…), the same int3-spine/int4-expert container, the same request, the container's on-disk key/value cache cleared before each arm: both returned HTTP 200 and the same 24-token continuation, token for token, and each server's own stderr shows it computing its own prefill ([API] KV slot 0 prefix 0/4 token, prefill 4 on both arms) — with a positive-coverage assertion, so an empty-vs-empty "match" cannot pass
decode is unchanged on a real model, CUDA	the same comparison on an NVIDIA GB10 (sm_121) host against an fp8 container: both arms [CUDA] device 0 with COLI_CUDA=1, both HTTP 200, the same 24 tokens, cache cleared before each arm
the mode is unreachable without its variable	tests/test_ablate_mode_gate.py reads the engine source and asserts exactly one product entry into the mode plus one compile-gated adapter entry, and that run_ablate_score has exactly one caller inside the mode; it fails if a second call site appears
nll is still dev's double-precision reduction	tests/test_ablate_mode.c drives a row through the real emitter whose logits are chosen so a single-precision intermediate differs, and compares the emitted text with == against an expectation computed in the test from dev's formula; substituting the float intermediate fails it
the header's config_sha256 really comes from the load path	the same suite writes a temp config.json, calls load_cfg, and compares the field against an independently computed digest and a pinned literal; it also asserts the field is left unset when the variable is not set
the manifest digest means the same thing in C and in Python	the same 64-character known answer is pinned in tests/test_ablate_mode.c:287 and in the tooling PR's suite, and reproduces outside both: printf 'coli-ablate-manifest/2\n0 3 2 0 0 1 2 3\n' | shasum -a 256 → c63a48c375b14ca60f26c7e3c5dd36b5929ffaf669a45511c93deee6e8bbd5ed
producer output survives a real round trip	on the Linux host, a real run over a real container produced a coli-ablate/2 artifact with three logit records that the offline checker accepted against the model's own config.json: [ablate-evidence] PASS manifest=4e18e3ab… items=1 targets=3, exit 0
the refusals are real, not decorative	tests/test_ablate_mode.c runs 73 named checks with no model and no weights — the loader, the writer, the three accepted framings, each named refusal, and the dispatch contract; tests/test_logprob_status.c runs 99, including published SHA-256 vectors at the block boundary and both padding branches

How the fleet comparison was set up, what it does and does not cover, and origin accounting

Container substitution, stated plainly. The CPU comparison did not run on the
fp8 container we first chose: an engine built from dev cannot load it (the
kv_b absorb path refuses fmt=8 without #1102), and it refuses a second
candidate outright because an overlay shard duplicates tensor names. We
therefore ran the CPU arms on a container the base engine does load: an
int3-spine/int4-expert build of the same model. The differential is
between two engines on one container, so the substitution changes which weights
were decoded, not what is being compared.

The CUDA arms carry another change. So that the fp8 container would load at
all on the GPU host, both arms there were built from local integration branches
that add #1102's fp8 decode on top: one is dev + #1102, the other is this
branch + #1102. Both arms carry it, so the difference between them is still
exactly this PR. Those integration branches are local only and are not proposed
anywhere.

Six launches, and what failed. Reaching the two green cells took six lane
launches. Four failed honestly and are worth naming because each was a refusal
we wanted: two container loads the base engine refuses by design, one cell whose
witness was written on a channel the server discards before it is readable (an
instrument defect on our side — a green there would have proved nothing, and it
failed instead of passing vacuously), and one arm refused by the engine's own
RAM guard while the previous arm was still releasing memory. The later CPU
re-run described above cost one more honest failure of the same kind: its first
launch failed its own content check because our cache-clearing record was written
in the wrong place in the log, which the check caught; the fixed relaunch is the
record cited here, and the failed launch's artifacts were kept.

What the comparison covers. Both hosts compared the generated continuation
of the same request across the two binaries with a positive-coverage assertion.

On the CPU host the comparison covers prefill as well as decode. The first
run of that cell had let the second arm resume the prompt's key/value state from
the first arm's on-disk cache, which meant only the dev binary had actually
computed a prefill; we re-ran the cell with the container's cache file moved
aside before each arm (its size and SHA-256 recorded each time it was moved),
so neither arm could inherit the other's state. Both arms then reported the same
line in their own server stderr — [API] KV slot 0 prefix 0/4 token, prefill 4
— which is the server saying it found nothing to reuse and computed all four
prompt tokens itself. With that, the two binaries agree on a run in which each
one performed the whole computation: match: true, positive_coverage: true,
both HTTP 200, the same 24 tokens.

On the GPU host the first run had let both arms resume a warm cache, so we
re-ran that cell the same way: the container's cache file moved aside before each
arm (size and SHA-256 recorded each time). Both arms reported
[API] KV slot 0 prefix 0/4 token, prefill 4 in their own server stderr, and the
two binaries agreed on a run in which each performed the whole computation:
match: true, positive_coverage: true, both HTTP 200, the same 24 tokens. The
cache file each binary then wrote to disk had the same SHA-256 on that host.

fp8_format.h pairing. The two new Makefile rules here list every header
their sources include, quant.h among them. If #1102 lands first, quant.h
gains fp8_format.h and these two rules should list it too, exactly as its
sibling rules do — a one-line follow-up in whichever order the two land.

requirement	instrument	result
default-path neutrality	old-vs-new binary differential on real containers, two hosts, two toolchains (GCC and NVCC), the key/value cache cleared before each arm on both hosts	identical output, positive coverage on both; every arm computed its own prefill
no unreachable-mode dispatch	source-level dispatch assertions in the gate test	one product entry, one compile-gated adapter entry
numeric equality with dev	committed row pin on a value where the two candidate reductions differ	equal, and the mutation fails
digest binding through the production load path	committed load_cfg round trip against an independent digest and a literal	bound; default path leaves it unset
named refusals, no partial execution	73 named checks with no model	green
round trip across all accepted framings	producer and offline checker, three framings, plus an on-disk difference assertion	one digest for all three
build hygiene	full clean build of the branch and its suites	see the checks run at the pushed head
Windows	not built here; the file APIs used come through the project's own compatibility header and the manifest is opened in binary mode, so line endings behave identically	unverified locally; CI's MinGW job is the closing evidence

Origin accounting. The mode's implementation, the two headers and their tests
are new content re-expressed onto current dev. One behaviour of dev's is
deliberately superseded (the old mode, as described at the top). One item is a
correction found during review rather than carried in: the offline checker
originally refused manifests with CRLF or without a final newline, which the
engine accepts — dev's leniency is the contract of record, so the
normalisation now lives in one place and is pinned by a known answer on both
sides. Two committed pins (the nll value and the load_cfg digest) close gaps
that nothing guarded before. The Makefile change completes the prerequisite
lists of the two new rules, matching every sibling rule. No surrounding code was
reformatted.

Origin accounting — additions after the rebases onto the current dev

• Dev-motion-caused: one commit lists evidence_digest.h among colibri's Makefile prerequisites, because upstream's
  new Makefile-prerequisite test (build: list the headers each engine includes as Makefile prerequisites #1284) requires every header a source includes to be declared. No behavior change.
• Correction found by re-review (this PR's own code): the shared logprob helper in c/sample.h subtracted two floats
  before promoting to double; on widely spread logit rows the error reached 0.25 (about 4e-4 on moderate rows, negligible
  on ordinary vocabularies). It now promotes the logit first; the value then agrees with a second double computation to a
  few units in the last place. No number this PR writes to disk changes, because the evidence writer computes its own
  double subtraction. logprob_target on dev keeps its existing arithmetic and is not touched here.

Durable vs current state: the mode's format, its refusals, the digest
binding and the dispatch guard are durable. The binary digests, container
choice, host models and the 24-token continuation are current state, measured
2026-09-04 against base e1efc68 on a Linux CPU host (GCC) and an NVIDIA GB10
host (NVCC); the cleared-cache re-runs on both hosts were measured 2026-09-05
with the same two binaries.