Score a run once its lookups have closed, not only once its stream is quiet - #265
Merged
Merged
Conversation
… the stream to go quiet
…ce its stream is quiet Scoring waited for the observed stream to stop growing for two seconds. That is a proxy for what the scorer then checks: that every lookup the candidate numbered wrote its observation. A real boundary call on an execute site that is still running (a pool wait, a lock) leaves the stream quiet with its lookup's number missing, and a stream read at that moment fails the run for a lookup that had not finished. The wait now settles on the property itself. Each time the stream goes quiet, it counts the lookups the stream has not closed, exactly as the scorer counts a gap, through one shared function over the same loader: a cancelled lookup closes its number and a gap does not. A stream that is quiet and closed is read at once. A stream that is quiet but open waits for its next write and is checked again, up to the same one-minute bound. The run log now says whether the stream closed, stayed open with how many lookups unwritten, or was still growing when scoring read it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A follow-up named in #258 body. Scoring now waits until the candidate's lookups have closed, not only until its observed stream goes quiet.
Why
#258 numbers every lookup as it arrives, and the scorer fails a run when a number has no observation. The lookup writes its observation when it finishes, and the candidate outlives the driver. So #258 also made scoring wait for the observed stream to stop growing for two seconds. That is a proxy for what the scorer then checks. A real boundary call on an execute site that is still running (a pool wait, a lock) leaves the stream quiet with its number missing. A stream read at that moment fails the run for a lookup that had not finished.
What changes
Each time the stream goes quiet, the runner counts the lookups it has not closed. It counts them exactly as the scorer counts a gap: a finished or cancelled lookup closes its number, and a gap does not. A stream that is quiet and closed is read at once. A stream that is quiet but open waits for its next write and is checked again, up to the same one-minute bound. The run log says whether the stream closed, stayed open with how many lookups unwritten, was still growing, or could not be read.
The check reads only each observation's number, one line at a time, rather than decoding every call. That is cheap on a large stream, and the stream is checked at most once per size it holds still at. Both the check and the scorer count gaps through one function,
missing_ordinals, and a test holds the two counts equal over the same stream.A stream that cannot be read, for example with a torn tail or a missing file, is never taken for closed. The settle keeps waiting and names it at the bound.
Limits, stated
Evidence
--no-fail-fast: 6, all killed:just verify: exit 0, clippy-D warningsclean, 1503 tests pass, web 79/79. The change is orchestrator-only.