Skip to content

Commit 8a93c64

Browse files
FrancescAltedclaude
andcommitted
Say in the docs that a remote array can be filled, not only read
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 85bd3f2 commit 8a93c64

2 files changed

Lines changed: 21 additions & 0 deletions

File tree

‎RELEASE_NOTES.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,17 @@ XXX version-specific blurb XXX
2222
which an object store has no way to serve), so constructors given a URL now say
2323
that instead of failing deep in C.
2424

25+
* A `C2Array` can be written to a chunk at a time, which is how several
26+
processes fill one remote array at once: `update_chunk()` (and its async
27+
`aupdate_chunk()`) posts one compressed chunk into a slot of a pre-sized
28+
array, and `written_chunks()` says which slots hold anything yet. The array is
29+
laid out with `blosc2.uninit()` and uploaded -- a couple of hundred bytes
30+
whatever its size -- and each slot is written once: a second write raises
31+
`blosc2.ChunkAlreadyWritten`, which is the whole of the coordination between
32+
writers. Writing into an empty slot appends to the frame and moves no other
33+
chunk, so a fill is cheap and a concurrent reader's cached offsets stay good.
34+
Needs a Caterva2 subscriber that serves the endpoint.
35+
2536
* `Proxy.fetch()` takes a `max_concurrency=` argument, and reads it from the
2637
source when the source has one, so `blosc2.open(url, lazy=True,
2738
max_concurrency=...)` overlaps its chunk fetches in a thread pool. Ordinary

‎doc/reference/c2array.rst‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,16 @@ HDF5 leaf) is fetched a whole chunk at a time, as everything was before. Which
1414
one this is takes at most one request to find out, and is decided once --
1515
:meth:`C2Array.block_source` is what answers it.
1616

17+
A stored remote array can also be *filled*, by as many writers at once as it has
18+
chunks. The array is laid out first -- ``blosc2.uninit`` writes a couple of
19+
hundred bytes whatever its size -- and then each writer posts the chunks it owns
20+
with :meth:`C2Array.update_chunk`. A slot nothing was written to is free, and a
21+
write claims it; a second write to the same slot raises
22+
:class:`blosc2.ChunkAlreadyWritten`, so two writers that both believe they own a
23+
chunk are resolved by the array rather than by anything either of them holds.
24+
:meth:`C2Array.written_chunks` reads how far the fill has got out of the frame's
25+
own offsets, which is one range read and no endpoint of its own.
26+
1727

1828
.. currentmodule:: blosc2
1929

0 commit comments

Comments
 (0)