Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: adjust allocated_size() in GenericByteViewBuilder #7104

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Chen-Yuan-Lai
Copy link
Contributor

Which issue does this PR close?

Closes #7099 .

Rationale for this change

As #7099 says, allocated_size() in GenericByteViewBuilder needs to be adjusted since allocated_size() in NullBufferBuilder returns bits, not bytes.

What changes are included in this PR?

Are there any user-facing changes?

  1. Divide the allocated_size by 8.
  2. Add related test

@github-actions github-actions bot added the arrow Changes to the arrow crate label Feb 10, 2025
@Chen-Yuan-Lai Chen-Yuan-Lai changed the title bugfix: adjust allocated_size() in GenericByteViewBuilde bugfix: adjust allocated_size() in GenericByteViewBuilder Feb 10, 2025
Copy link
Contributor

@Jefffrey Jefffrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@mbrobbel mbrobbel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should re-consider https://github.com/apache/arrow-rs/pull/7089/files#r1945293119, because having allocated_size return both bytes and bits may cause confusion; this PR is an example.

@tustvold
Copy link
Contributor

IMO allocated_size being in bits is a bug and should be fixed - #7089 (comment).

Allocations can't be made at bit granularity and so the notion this method should return in bits seems odd to me

@tustvold
Copy link
Contributor

I've filed #7121 to change the allocated_size to be in bytes, going to mark this as a draft in the meantime

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memory accounting in GenericBytesView overcounts allocation size
4 participants