-
-
Notifications
You must be signed in to change notification settings - Fork 361
feat: Implement ZEP 8 URL syntax support for zarr-python #3369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Add comprehensive ZEP 8 URL parsing and resolution system - Implement StoreAdapter ABC for extensible storage adapters - Add built-in adapters for file, memory, S3, GCS, HTTPS schemes - Support pipe-chained URLs like s3://bucket/data.zip|zip:|zarr3: - Add URLSegment parsing with validation - Integrate with zarr.open_group and zarr.open_array APIs - Include demo script and comprehensive test suite - Pass all existing tests + 35 new ZEP 8-specific tests
…into feature/zep8-url-support
One tricky thing about The "path" syntax is generally the default when running a regular s3-compatible server, but the "virtual host" syntax can commonly occur when someone defines a CNAME DNS entry to map their own domain or subdomain to an AWS S3 bucket. When designing this syntax for Neuroglancer, it seemed like it would be annoying to require users to use separate syntax to disambiguate the two cases. Instead, for operations where it matters (namely List), Neuroglancer just automatically determines which of the two cases applies by trying both ways and seeing which one succeeds, and then caching the result so that subsequent list operations don't require two requests. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3369 +/- ##
===========================================
- Coverage 94.92% 60.87% -34.06%
===========================================
Files 79 86 +7
Lines 9500 10231 +731
===========================================
- Hits 9018 6228 -2790
- Misses 482 4003 +3521
🚀 New features to boost your workflow:
|
Examples:: | ||
|
||
# Basic ZIP file storage | ||
zarr.open_array("file:data.zip|zip", mode='w', shape=(10, 10), dtype="f4") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like a bad idea to break file uris.
This PR implements support for the ZEP 8 URL syntax in Zarr Python.
Some examples of what now works:
TODO:
docs/user-guide/*.rst
changes/
closes #2943
fixes #2831
xref: zarr-developers/zeps#48
cc @jbms