Skip to content

feat(bitmap): build a bitmap's zeroed layout and clone it with headroom for new keys - #62

Open
aliszka wants to merge 2 commits into
mainfrom
feat/zeroed-layout
Open

aliszka wants to merge 2 commits into
mainfrom
feat/zeroed-layout

Conversation

@aliszka

@aliszka aliszka commented Oct 27, 2026

Copy link
Copy Markdown
Member

Adds ZeroedLayout and its buffer variants, which build an empty bitmap shaped like a source, and CloneToBufWithHeadroom, which clones into a pooled buffer with room for new keys. Both families take their buffer from get on CompactedToBuf's terms.

Motivation

A caller that repeatedly ORs subsets of a known bitmap into a scratch bitmap pays for growth on every round: a new container moves everything after it, and a new key past the keys node's capacity moves every container. Starting from an empty bitmap with the source's keys, container sizes and types avoids both, since nothing the subset brings needs room the layout lacks. A caller that instead starts from a copy of the source, and adds values at keys beyond its last one, needs the same headroom CompactedToBufWithHeadroom gives, without compacting.

Approach

The source's layout is kept, not rebuilt: containers keep their position, size and type, dead space included, and only the keys node is resized, to the source's keys plus one spare slot (plus whatever the headroom variants buy). Every offset shifts by the change in the keys node's size, which may be negative when the source had spare slots. That keeps sizing O(1), from the source's length and keys alone, and lets the clone copy all containers at once. Repacking would make the result smaller but costs a pass over the containers, and the zeroed layout would then no longer match the source container for container.

The zeroed builds write only the keys node and the container headers. A pooled buffer may be dirty, so bitmap payloads are cleared; array payloads and dead space keep their old bytes, because nothing reads past an array's cardinality.

Key areas for review

  • bitmap.go:writeShiftedKeys: the offset shift, including the negative case wrapping in uint64.
  • bitmap_opt.go:shiftedKeysDst: sizing before get, headroom from the buffer's capacity after it.
  • bitmap.go:writeZeroedContainers: which bytes of a dirty buffer are cleared, and why the rest are safe.

Testing

  • TestZeroedLayout and TestLayoutToBuf build every variant from sources covering nil, empty, key 0 only, arrays, bitmaps, mixed, dead space, emptied containers, spare key slots and oversized capacity, from dirty buffers. They check one get call, the exact layout, and that OR-ing a subset back in keeps capacity, length and offsets.
  • TestLayoutToBufWithHeadroom checks that keys added after the last fit without moving any container except an overflowing last array.
  • TestLayoutToBufContract covers the panics and zero allocations for the Init variants.
  • BenchmarkZeroedLayout compares against cloning and clearing; BenchmarkCloneToBufWithHeadroom against CloneToBuf, each alone and followed by an OR of new keys.

🤖 Generated with Claude Code

@orca-security-eu orca-security-eu Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

aliszka and others added 2 commits October 27, 2026 10:35
ZeroedLayout returns an empty bitmap with the source's keys and, per key,
an empty container of the source's size and type at its position in the
source, so OR-ing a subset of the source into it never grows it. The
keys node is resized to one spare slot whatever the source had, and every
container offset shifts by the change in its size. Dead space is kept,
not repacked. Only the keys node and the container headers are written.

ZeroedLayoutToBuf and InitZeroedLayoutToBuf build into a buffer obtained
from get, on CompactedToBuf's terms: get is called once with the exact
size, and the buffer is adopted at its full capacity and may be dirty,
so they also clear bitmap payloads. Array payloads and dead space keep
their old bytes, since nothing reads them. Sizing takes the source's
length and keys alone, with no pass over its containers.

The WithHeadroom variants spend the buffer's surplus on extra key slots
as CompactedToBufWithHeadroom does, estimating new containers from the
source's own, so values ORed in at new keys after the last neither
expand the keys node nor reallocate the data.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CloneToBufWithHeadroom and InitCloneToBufWithHeadroom clone into a buffer
obtained from get, on CompactedToBufWithHeadroom's terms: the surplus past
the requested size buys key slots and leaves the rest as tail room, so
values ORed in at new keys after the last neither expand the keys node nor
reallocate the data.

They share the zeroed layout's buffer step and keys writer. The keys
node is resized to the source's keys plus one spare slot and the extra
slots, dropping any spare slots the source had, and every offset shifts
by the change in its size. The containers then follow in a single copy,
each keeping its position, size and type, dead space included, so the
clone matches the source container for container.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@aliszka
aliszka force-pushed the feat/zeroed-layout branch from a339d2c to 07aa085 Compare October 27, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant