Repository navigation
perf(container): keep an in-place Or's union in the target container's own slot - #61
Merged
Merged
Conversation
An inline Or of two array containers built the union in a buffer and copied it back. When the target's slot has room for both cardinalities, merge backwards straight into it instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An array-with-array Or turned the result into a bitmap whenever the two cardinalities summed past 2452, before the union was known. Unions of overlapping arrays then left their slot for a new 8 KB bitmap even when they fit the slot as an array. The result now stays an array in the target's slot when it fits, and is otherwise returned in the buffer. A bitmap-sized slot always takes a non-empty union as a bitmap, so later Ors into it set bits in place without a buffer. Keeping more arrays in their slots made more inline Ors of an array with a bitmap hand back the bitmap operand itself. The masked Or loops in CopresenceByMask and AndMasked assumed a returned container sat in their spare buffer, and lost the union or wrote into the operand. They now read such a container in place and copy it into their own buffer only before writing into it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When both arrays hold at most 2044 values but their union exceeds it, the merge into an array was thrown away and the bitmap built from scratch. Values of one array outside the other's range cannot be in it, so counting them with at most one binary search per end bounds the union from below, exactly when either array is a run of consecutive values. When that bound already makes the stored result a bitmap, the bitmap is built directly. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bitsIntoArray found each value one 16-bit word at a time with a leading-zero count. It now reads four words per 64-bit load, skips eight zero loads with one test, and takes each value with a trailing-zero count after reversing the bits within each 16-bit lane. Words before the next multiple of 4 from the container start, the words after the last full load, and a load that does not fit in the output still go through the 16-bit loop. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Setting values one at a time makes neighbouring values queue on the same bitmap word. orValues now builds one mask per uint64 word from the array's values, ORs it in once and counts new bits by popcount. Arrays averaging under three values per word keep the per-value loop, which is faster there; BenchmarkOrValues sweeps that threshold. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An inline array ∪ bitmap Or returned every union as a new bitmap, which the caller then appended to the Bitmap even when the array's slot had room. A non-empty array's union is still built as a bitmap, but it is now stored in the array's slot when the slot holds it: as is when the slot is bitmap-sized, since compaction picks the type by cardinality anyway, and as an array when it fits a smaller slot. An empty array copies the bitmap's values straight into its slot when they fit as an array. Only a union that fits neither is returned. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
orValuesByMask and the inline bitmap-array And each built the bit of a value in its uint64 word with their own arithmetic. Both now call wordBit, which a test checks against setBit for every value. The And loop drops its separate lane and bit shifts. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
antas-marcin
approved these changes
Oct 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An in-place
Ornow keeps the union in the target container's existing slot whenever the slot holds it, instead of appending a new container to the bitmap's buffer. Array ∪ array and array ∪ bitmap follow one rule set; bitmap ∪ anything is unchanged.Motivation
Merging many bitmaps into one with in-place Ors grew its buffer needlessly. An array container's
Orpicked its result type from the sum of both cardinalities, and array ∪ bitmap always returned a new bitmap, so a union that fit the existing slot still left it as dead space and appended an 8 KB bitmap.Approach
The result type follows the real union cardinality: an array stays in its slot if it fits, a bitmap-sized slot always takes the union as a bitmap (later Ors set bits in place, and compaction picks the type by cardinality anyway), and only a union that fits neither is returned. When the slot has room for both arrays, the merge runs backwards straight into it with no buffer.
Knowing the real cardinality first would cost a merge that is often thrown away, so array ∪ array bounds the union from its range edges, with at most one binary search per end, and builds the bitmap directly when the bound rules out an array. Array ∪ bitmap builds the bitmap first and extracts it into the slot only when it fits; a counted merge into the slot was slower and was dropped.
Two helpers make the bitmap-first route cheap: listing a bitmap's values reads 64 bits at a time, and ORing array values into a bitmap builds one mask per uint64 word, keeping the per-value loop for sparse arrays (
BenchmarkOrValuessweeps the threshold).Key areas for review
container_opt.goorArrayAlt,unionMayFitArray,canFitArray,bitmapUnionResult: the type rule; slots from older data, including arrays past 2044 values, are decided from their headers.bitmap_opt.gomasked Or loops inCopresenceByMaskandAndMasked: an inline Or can now return the bitmap operand itself, so they read it in place and copy it only before writing.container.gobitsIntoArray64andorValuesByMask: unsafe uint64 views over container words.Risks
BenchmarkOrArrayShapesandBenchmarkOrArrayBitmapreproduce them.Testing
TestOrArrayAltResultandTestOrArrayBitmapAltResulttable-test every route against slot type and size, legacy slots included;TestOrMergesArraysInPlacecovers the public path;TestAndMaskedandTestCopresenceByMaskpin the operand aliasing.🤖 Generated with Claude Code