Skip to content

Commit b1637af

Browse files
FrancescAltedclaude
andcommitted
Write a chunk of a remote array, and read which ones were written
A pre-sized array is filled a chunk at a time, by as many writers as there are chunks: `C2Array.update_chunk` posts one, and the subscriber refuses a slot that already holds anything. `written_chunks` reads what the fill recorded in the frame's own offsets, so progress costs one range read and no endpoint of its own. Reading the index turned out not to be the same question as reading blocks of a chunk: `serves_blocks` also weighs whether splitting a chunk would pay, which is nothing to do with whether the frame has offsets to read. Split the geometry half out, so a frame of small chunks can still say which of them were written. `invalidate_index` drops what was read of a frame that has since been written to -- the header as well as the offsets, since a write moves the frame's length and its payload extent, and the offsets are found through both. Nothing is read until the next lookup asks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 41ad9d6 commit b1637af

7 files changed

Lines changed: 103771 additions & 12 deletions

File tree

‎src/blosc2/__init__.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -572,7 +572,7 @@ def _raise(exc):
572572
from .ref import Ref
573573
from .b2objects import open_b2object
574574

575-
from .c2array import c2context, C2Array, C2NDSource, URLPath
575+
from .c2array import c2context, C2Array, C2NDSource, ChunkAlreadyWritten, URLPath
576576

577577
from .dsl_kernel import DSLSyntaxError, DSLKernel, dsl_kernel, validate_dsl, validate_dsl_jit
578578
from .lazyexpr import (
@@ -859,6 +859,7 @@ def _raise(exc):
859859
# Classes
860860
"C2Array",
861861
"C2NDSource",
862+
"ChunkAlreadyWritten",
862863
"Column",
863864
"CParams",
864865
"CTable",

0 commit comments

Comments
 (0)