Corgi unit value diversion - #870
Closed
frankmcsherry wants to merge 3 commits into
Closed
frankmcsherry wants to merge 3 commits into
frankmcsherry wants to merge 3 commits into
Conversation
Use unit_output() for output handling and resolves_input_payloads() for the current input resolution path. Count remains on that general path; a later diff-only implementation can opt out without changing presentation. Register PresentedIds through the existing value registration helper, restore the negative-weight rationale, assert bracket lengths, and rename the unit-value test module without changing its oracle tests. For context, four-process maintenance medians for diversion 566e753 against master-next 626c80c were Reach 100.03 -> 91.18 ms/epoch (-8.8%), pair-key Reach 120.35 -> 111.34 (-7.5%), and SCC 530.92 -> 531.08 (within run spread). Apple M4, one worker, mimalloc, release/LTO, averaging epochs 6-25; Reach 200k nodes/400k edges, SCC 100k/200k, 1k replacements/epoch. These measure the entire diversion, not inserts/gather in isolation, and predate this helper/comment cleanup. Compound input hashes remain; unit IDs, resolution pools, correction maps and output indices are avoided. Validation: 133 selected Rust tests passed, 2 existing ignores; git diff --check passed. Count's existing positive-only semantics are unchanged in both DDIR backends, despite differing from signed DD count.
frankmcsherry
added a commit
that referenced
this pull request
Sep 22, 2026
`merge_present` merges presented chunk runs straight into the bridge when they arrive in `((key, value_id), time)` order, and otherwise consolidates. It recognized leaf values as ordered but not `Unit` values, though all unit rows share one id. Unit-valued presentations (`distinct`'s output, and unit-valued inputs such as reach's `join(edges, ($2 ;)) | distinct`) paid for a sort. The merge also reserves its bridge up front, as the fallback's `extend` does, rather than growing it by doubling. The `ordered_ids` line is the same as in #870, which predates the `IdPool` refactor. Same-plan harness, 1 worker, median of 3 (before -> after): reach 2M edges, 1000 x 100 load 767 -> 679 ms churn 85.2 -> 76.4 ms/round scc 200k edges, 100 x 50 load 674 -> 666 ms churn 35.6 -> 35.4 distinct 1M rows, 1000 x 500 load 87.5 -> 84.9 ms churn 2.51 -> 2.45 count 1M rows, 1000 x 500 load 88.5 -> 85.7 ms churn 3.62 -> 3.54 Peak RSS: a 4M-row count or distinct load 1289 -> 1145 MiB (the reserve); reach 609 -> 640 MiB, not traced. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Divert computation in a reduce whose values are
UNIT, as would be the case in a distinct operator, or similar.