Give what a proxy reads from a module of its own

`c2array` imported `proxy` at module level, and had to: `C2NDSource`
subclasses `ByteRangeNDSource`, which a class statement needs at import
time.  The suggested fix -- a function-local import -- could not work for
that reason, and moving the import in `__init__` could not either:
`embed_store` and `dict_store` are bound at lines 565-566 and both reach
`c2array` for `C2Array`, while `lazyexpr` reaches `proxy` directly for
`convert_dtype`.  So `proxy.py` was dragged in ahead of `schunk`,
`indexing` and `lazyexpr` by three independent paths, and could import
none of them at module level without `import blosc2` failing outright.

The half those modules actually want moves to `blosc2/proxy_source.py`:
the `ProxySource`/`ProxyNDSource` protocol, the frame reading (header,
offsets, chunk and block extents, splicing), `ByteRangeNDSource`,
`FsspecNDSource`, `convert_dtype` and the block-granularity thresholds.
It reaches into the package for nothing past what `blosc2_ext` binds
first.  `proxy.py` keeps the `Proxy` itself and imports the rest back,
re-exporting the four classes so `blosc2.proxy.X` still answers -- as
`blosc2.X` does, unchanged, which is what the docs autoclass.

`.proxy` is now bound after `.schunk` and `.indexing`, and the proof is
in the diff: `_reopen_cache`'s function-local `from blosc2.schunk import
_set_default_dparams`, which was function-local for exactly this reason,
is a module-level import again.

What moved for good are the tuning constants -- `BLOCK_MIN_CBYTES` and
its neighbours are `blosc2.proxy_source.BLOCK_MIN_CBYTES` now, since
re-exporting a constant by value would leave anyone setting it on
`blosc2.proxy` silently patching a copy nothing reads.  The two benches
and the tests that set them are updated to say where they live.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: FrancescAlted

RELEASE_NOTES.md

@@ -53,7 +53,10 @@ XXX version-specific blurb XXX
   ranges (`multipart/byteranges`, which no object store offers). A dataset the
   subscriber *computes* — a lazy expression, an HDF5 leaf, a `.b2z` member — is
   fetched a whole chunk at a time as before; which it is costs at most one small
-  request to find out, and is never asked twice. `blosc2.ByteRangeNDSource` is
+  request to find out, and is not asked again -- unless the subscriber could not
+  say, which a busy or unreachable one cannot: a 5xx or a connection that failed
+  is asked again on the next fetch rather than written off, since neither
+  downloaded anything to find out. `blosc2.ByteRangeNDSource` is
   the frame reader `FsspecNDSource` and the new `C2NDSource` share: subclass it
   with a `read_range(offset, size)` to give any transport the same treatment.
   Opening a remote frame through either of them now costs two requests instead
@@ -97,8 +100,22 @@ XXX version-specific blurb XXX
   bytes they read are held to that as well, so a remote array *replaced* while
   keeping its shape is noticed rather than served stale: `FsspecNDSource` uses
   fsspec's token and `C2Array` the subscriber's mtime, both free with metadata
-  they already fetch. Caches from earlier 4.11.1 development builds are not
-  adopted (the stamp moved to a `proxy-stamp` entry); pass `mode="w"` once.
+  they already fetch. A proxy that `blosc2.open` rebuilds over its own cache
+  gets the same treatment without the raise -- there is no `mode="w"` to offer
+  it -- so a cache whose stamp no longer matches starts as though nothing had
+  been fetched and fills again from the bytes served now; opened read-only there
+  is nothing to empty, and every read falls through to the source. Caches from
+  earlier 4.11.1 development builds are not adopted (the stamp moved to a
+  `proxy-stamp` entry); pass `mode="w"` once.
+
+* The source protocol moved to its own module, `blosc2.proxy_source`:
+  `ProxySource`, `ProxyNDSource`, `ByteRangeNDSource`, `FsspecNDSource` and the
+  frame reading behind them. They are still `blosc2.X` and still reachable as
+  `blosc2.proxy.X`, so nothing outside need change; what moved for good are the
+  block-granularity knobs, `blosc2.proxy_source.BLOCK_MIN_CBYTES` and its
+  neighbours. This is what lets `proxy.py` be imported after `schunk` and
+  `indexing`, rather than being dragged in ahead of them by every module that
+  wants a source.
 
 * Querying a `utf8()` column through its FULL index no longer materializes the
   index vocabulary. The query literal is turned into an alphabetical rank by

bench/ndarray/cat2-block-granularity.py

@@ -84,7 +84,7 @@
 import blosc2
 from blosc2 import c2array
 
-CHUNK_HEADER = blosc2.proxy._CHUNK_HEADER_LEN
+CHUNK_HEADER = blosc2.proxy_source._CHUNK_HEADER_LEN
 
 
 #
@@ -353,8 +353,8 @@ def timed_slice(open_array, item, mode, concurrency, latency, bandwidth):
     of what it used to do.  The frame index is read before the clock starts: it
     costs four small requests once per C2Array, not once per slice.
     """
-    threshold = blosc2.proxy.BLOCK_MIN_CBYTES
-    blosc2.proxy.BLOCK_MIN_CBYTES = 1 << 62 if mode == "chunks" else threshold
+    threshold = blosc2.proxy_source.BLOCK_MIN_CBYTES
+    blosc2.proxy_source.BLOCK_MIN_CBYTES = 1 << 62 if mode == "chunks" else threshold
     try:
         array = open_array()
         array.max_concurrency = concurrency
@@ -367,7 +367,7 @@ def timed_slice(open_array, item, mode, concurrency, latency, bandwidth):
         proxy[item]
         return time.perf_counter() - start, tally["requests"], tally["bytes"]
     finally:
-        blosc2.proxy.BLOCK_MIN_CBYTES = threshold
+        blosc2.proxy_source.BLOCK_MIN_CBYTES = threshold
 
 
 def connection_setup(urlbase, path, token, reps):
@@ -456,7 +456,7 @@ def open_array():
         found_out = (
             f"{opening['requests']} request found that out, and it is never asked again"
             if opening["requests"]
-            else f"nothing was asked.\n  Chunks under the {blosc2.proxy.BLOCK_MIN_CBYTES / 1e6:.0f} MB "
+            else f"nothing was asked.\n  Chunks under the {blosc2.proxy_source.BLOCK_MIN_CBYTES / 1e6:.0f} MB "
             "a round trip costs are not worth taking apart, and api/info\n  tells a computed "
             "dataset from a stored one for free"
         )

bench/ndarray/fsspec-block-granularity.py

@@ -223,8 +223,8 @@ def timed_open(urlpath, item, concurrency, blocks, latency=0.0, bandwidth=0.0):
     *bandwidth* bytes per second across it.  Both waits happen inside the
     proxy's thread pool, so they overlap the way real ones would.
     """
-    old = blosc2.proxy.BLOCK_MIN_CBYTES
-    blosc2.proxy.BLOCK_MIN_CBYTES = old if blocks else 1 << 62
+    old = blosc2.proxy_source.BLOCK_MIN_CBYTES
+    blosc2.proxy_source.BLOCK_MIN_CBYTES = old if blocks else 1 << 62
     try:
         array = blosc2.open(urlpath, lazy=True, max_concurrency=concurrency)
         traffic = [0, 0]
@@ -246,7 +246,7 @@ def counted(*args, _o=original):
         array[item]
         return time.perf_counter() - t0, traffic[0], traffic[1]
     finally:
-        blosc2.proxy.BLOCK_MIN_CBYTES = old
+        blosc2.proxy_source.BLOCK_MIN_CBYTES = old
 
 
 def run_moto(local, concurrency, reps, latency, bandwidth):

src/blosc2/__init__.py

@@ -588,20 +588,25 @@ def _raise(exc):
     result_type,
     can_cast,
 )
-from .proxy import (
-    Proxy,
+from .proxy_source import (
     ProxySource,
     ProxyNDSource,
-    ProxyNDField,
     ByteRangeNDSource,
     FsspecNDSource,
-    SimpleProxy,
-    jit,
-    as_simpleproxy,
 )
 from .indexing import Index
 
 from .schunk import SChunk, load, open
+
+# After .schunk and .indexing, which it may now import at module level: what a
+# proxy reads from lives in .proxy_source, which the early modules reach instead
+from .proxy import (
+    Proxy,
+    ProxyNDField,
+    SimpleProxy,
+    jit,
+    as_simpleproxy,
+)
 from . import linalg
 from .linalg import tensordot, vecdot, permute_dims, matrix_transpose, matmul, transpose, diagonal, outer
 from .utils import linalg_funcs as linalg_funcs_list

src/blosc2/c2array.py

@@ -23,10 +23,7 @@
 import blosc2
 from blosc2.b2objects import encode_b2object_payload, make_b2object_carrier, write_b2object_payload
 from blosc2.info import InfoReporter, format_nbytes_info
-
-# blosc2/__init__ imports this module before blosc2.proxy, so this pulls proxy in
-# early; it is safe because proxy only reaches into the package at call time
-from blosc2.proxy import REMOTE_MAX_CONCURRENCY, ByteRangeNDSource
+from blosc2.proxy_source import REMOTE_MAX_CONCURRENCY, ByteRangeNDSource
 
 _subscriber_data = {
     "urlbase": os.environ.get("BLOSC_C2URLBASE"),
@@ -782,7 +779,7 @@ def _open_block_source(self):
             # ever use a block, and the dataset keeps the behaviour it had before.
             # Inside the `try`, since `api/info` need not carry what these read.
             nchunks = math.prod(math.ceil(s / c) for s, c in zip(self.shape, self.chunks, strict=True))
-            if not nchunks or self.cbytes / nchunks < blosc2.proxy.BLOCK_MIN_CBYTES:
+            if not nchunks or self.cbytes / nchunks < blosc2.proxy_source.BLOCK_MIN_CBYTES:
                 return None
             source = C2NDSource(self, max_concurrency=REMOTE_MAX_CONCURRENCY)
             source.adopt_index(self._pending_index)

src/blosc2/lazyexpr.py

@@ -54,7 +54,7 @@
     write_b2object_user_vlmeta,
 )
 from .dsl_kernel import DSLKernel, DSLSyntaxError, DSLValidator, specialize_miniexpr_inputs
-from .proxy import convert_dtype
+from .proxy_source import convert_dtype
 from .utils import (
     check_smaller_shape,
     compute_smaller_slice,

src/blosc2/ndarray.py

@@ -6880,8 +6880,8 @@ def asarray(
     utf8_out, array = _asarray_string_dispatch(array, copy, kwargs)
     if utf8_out is not None:
         return utf8_out
-    dtype_ = blosc2.proxy.convert_dtype(array.dtype)
-    dtype = blosc2.proxy.convert_dtype(kwargs.pop("dtype", dtype_))  # check if dtype provided
+    dtype_ = blosc2.proxy_source.convert_dtype(array.dtype)
+    dtype = blosc2.proxy_source.convert_dtype(kwargs.pop("dtype", dtype_))  # check if dtype provided
     kwargs = _check_ndarray_kwargs(**kwargs)
     chunks = kwargs.pop("chunks", None)
     blocks = kwargs.pop("blocks", None)