Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove partial codec initialization #2881

Open
d-v-b opened this issue Mar 3, 2025 · 3 comments
Open

remove partial codec initialization #2881

d-v-b opened this issue Mar 3, 2025 · 3 comments

Comments

@d-v-b
Copy link
Contributor

d-v-b commented Mar 3, 2025

Some codec classes (like the blosc codec) can be partially initialized, where codec creation succeeds but the resulting object is not actually serializable because certain fields that are required for serialization are nullable for codec creation. I believe the logic for this pattern was to spare users the burden of specifying codec properties that depend on the data type of their array, like bloscshuffle. The cost of this pattern is code complexity, indirection, and irregularity of the codec API surface.

Now that we have easier methods for automatically specifying codecs (e.g, the "auto" value in create_array), I don't think we need to support partial codec initialization any more. So we should remove this functionality from the codecs that use it, and instead require codecs are fully valid for serialization when they are constructed.

@AdityaChaubeyIITM
Copy link
Contributor

Hello @d-v-b , I see 2 ways to implement it, either make details like typesize and shuffle compulsory while defining, or ask data type instead and then set typesize and shuffle. Which of the 2 would you suggest? Or is there any other way that I missed? Please let me know

@d-v-b
Copy link
Contributor Author

d-v-b commented Mar 5, 2025

hi @AdityaChaubeyIITM, I would recommend the first option -- making all details required for serialization compulsory.

@AdityaChaubeyIITM
Copy link
Contributor

Sure, I shall start working on this then.

AdityaChaubeyIITM added a commit to AdityaChaubeyIITM/zarr-python that referenced this issue Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants