Skip to content

Score a run once its lookups have closed, not only once its stream is quiet - #265

Merged
maverox merged 2 commits into
mainfrom
work/settle-on-the-ordinals
Sep 30, 2026
Merged

maverox merged 2 commits into
mainfrom
work/settle-on-the-ordinals

Conversation

@maverox

@maverox maverox commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

A follow-up named in #258 body. Scoring now waits until the candidate's lookups have closed, not only until its observed stream goes quiet.

Why

#258 numbers every lookup as it arrives, and the scorer fails a run when a number has no observation. The lookup writes its observation when it finishes, and the candidate outlives the driver. So #258 also made scoring wait for the observed stream to stop growing for two seconds. That is a proxy for what the scorer then checks. A real boundary call on an execute site that is still running (a pool wait, a lock) leaves the stream quiet with its number missing. A stream read at that moment fails the run for a lookup that had not finished.

What changes

Each time the stream goes quiet, the runner counts the lookups it has not closed. It counts them exactly as the scorer counts a gap: a finished or cancelled lookup closes its number, and a gap does not. A stream that is quiet and closed is read at once. A stream that is quiet but open waits for its next write and is checked again, up to the same one-minute bound. The run log says whether the stream closed, stayed open with how many lookups unwritten, was still growing, or could not be read.

The check reads only each observation's number, one line at a time, rather than decoding every call. That is cheap on a large stream, and the stream is checked at most once per size it holds still at. Both the check and the scorer count gaps through one function, missing_ordinals, and a test holds the two counts equal over the same stream.

A stream that cannot be read, for example with a torn tail or a missing file, is never taken for closed. The settle keeps waiting and names it at the bound.

Limits, stated

  • A trailing lookup is invisible. A lookup still running with the highest number has no later finished lookup to reveal it, so it leaves no gap. This is a limit of counting by number, and it predates this change; the scorer shares it.
  • A lookup that never finishes now holds scoring for the full minute, where the quiet window alone held it for two seconds. That is the trade: the run is read once its lookups close, or at the bound.
  • A candidate built before lookups were numbered reports none missing, so it settles at the first quiet period, with the same timing as before.

Evidence

  • Red first. The test commit calls the new settle signature and does not build on its own.
  • Mutants, each against the whole workspace with --no-fail-fast: 6, all killed:
    • the closure never checked;
    • an open stream read as closed;
    • an open stream read as still growing;
    • the check reading the wrong records;
    • the gap count inverted;
    • an unreadable stream read as closed.
  • The settle tests do not depend on scheduling. The late-lookup case is decided by the stream's content. The still-growing case has the check itself write a line, so the stream is always growing at the bound, with no thread involved. An earlier version, in which a thread paced its writes against the quiet window, failed under unrelated load.
  • just verify: exit 0, clippy -D warnings clean, 1503 tests pass, web 79/79. The change is orchestrator-only.

…ce its stream is quiet

Scoring waited for the observed stream to stop growing for two seconds. That
is a proxy for what the scorer then checks: that every lookup the candidate
numbered wrote its observation. A real boundary call on an execute site that
is still running (a pool wait, a lock) leaves the stream quiet with its
lookup's number missing, and a stream read at that moment fails the run for a
lookup that had not finished.

The wait now settles on the property itself. Each time the stream goes quiet,
it counts the lookups the stream has not closed, exactly as the scorer counts
a gap, through one shared function over the same loader: a cancelled lookup
closes its number and a gap does not. A stream that is quiet and closed is
read at once. A stream that is quiet but open waits for its next write and is
checked again, up to the same one-minute bound. The run log now says whether
the stream closed, stayed open with how many lookups unwritten, or was still
growing when scoring read it.
@maverox
maverox merged commit ee7a7e9 into main Sep 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant