@@ -344,3 +344,81 @@ Each was a short script run against local files; none needs a server.
344344 ` chunk[31] >> 4 & 0x7 ` ; cross-check with ` schunk.iterchunks_info() ` .
345345- ** Length is not a validator** : stat + md5 the file after a create, two ZERO
346346 writes and a regular write.
347+
348+ ## What landed
349+
350+ Phases 1, 2, 3 and 5 (2026-08-21), on ` cat2-concurrent-writers ` here and
351+ ` c2cache-monorepo ` in Caterva2. Phase 4 is still open, on purpose; phase 6's
352+ tests landed with the phases they cover, its bench did not.
353+
354+ | phase | where | commit |
355+ | ---| ---| ---|
356+ | 1. ` POST api/chunk ` | caterva2 ` server.py ` | * Accept one chunk at a time into a slot that holds none* |
357+ | 2. ETag | caterva2 ` server.py ` | the same commit |
358+ | 3. client writes | blosc2 ` c2array.py ` , ` proxy_source.py ` | * Write a chunk of a remote array, and read which ones were written* |
359+ | 5. completion and publish | caterva2 ` server.py ` | * Publish an array once every one of its chunks has landed* |
360+ | 5a. atomic publish | caterva2 ` server.py ` | * Move a published array into place instead of streaming into it* |
361+
362+ The shape held: a pre-sized ` uninit ` array, one write per slot, the offsets as
363+ the record, and a 409 as the whole of the coordination. 15 tests against a live
364+ subscriber (` caterva2/tests/test_chunk_writes.py ` ) and 10 against a stand-in
365+ (` tests/ndarray/test_c2array_writes.py ` ), which is where the client-side
366+ behaviour is pinned without a service.
367+
368+ ### What the work found
369+
370+ - ** Reading the index is not the same question as reading blocks.**
371+ ` C2Array.written_chunks() ` was gated on ` serves_blocks ` , which also weighs
372+ whether * splitting a chunk into blocks* would pay — a frame of small chunks
373+ reported that it served no blocks and so could not say which chunks were
374+ written either. The geometry half is now ` _reports_geometry ` and the index is
375+ read whatever the chunks cost. ` serves_blocks ` is unchanged for the block
376+ path, which reads it at ` Proxy ` build time and must keep costing nothing.
377+ - ** Invalidating the index means the header too.** A write moves the frame's
378+ length and its payload extent, and the offsets are found through both, so
379+ dropping the offsets alone left the next read looking for them at the old
380+ position. ` ByteRangeNDSource.invalidate_index() ` marks both stale and reads
381+ neither until something asks.
382+ - ** The completion scan had to move off ` iterchunks_info ` .** It reads a lazy
383+ chunk apiece — 3.5 µs each, so 17.8 ms on 5000 chunks, which would have made a
384+ fill cost the square of its length. The write-once check reads the one
385+ chunk's header instead (~ 3 µs), and the count comes from the offsets in one go
386+ (0.37 ms at 5000 chunks, 0.39 ms at 20000 — flat where the walk is linear).
387+ - ** A publish that streams into place is readable before it is whole.** The
388+ destination file exists from its first byte and a frame is not readable until
389+ its last, so a reader polling for the published array opened it mid-copy and
390+ got a NULL back. Found by the test doing exactly that, which had passed only
391+ while the copy won the race. Published under a name of its own and moved into
392+ place.
393+ - ** A stale blosc2 handle corrupts a write silently.** Two handles open over one
394+ frame, one of them writing, leaves the frame unreadable — `Invalid arguments
395+ for stdio write` under ` BLOSC_TRACE=1`, and nothing at all without it: the
396+ write itself does not raise. This is the hazard ` todo/locking-mwmr.md `
397+ documents, but the silence of it is worth knowing. Both the endpoint and the
398+ test stand-in are written to hold exactly one handle and to drop it before
399+ anything reads the file again.
400+
401+ ### Phase 4 is still open
402+
403+ ` C2Array.stamp ` is still ` mtime:cbytes ` , so a ` Proxy ` cache over an array being
404+ filled is still discarded on every write, though every chunk it holds is still
405+ exactly where it was. The ETag added in phase 2 is the * freshness* half of the
406+ question and does not answer this one: it changes on an append as readily as on a
407+ replacement, which is precisely the distinction wanted.
408+
409+ What would answer it is an identity that survives appends and not replacement.
410+ The frame header has no UUID to use. The candidate is a nonce written into
411+ vlmeta when the array is laid out — ` api/info ` already carries vlmeta, so it
412+ costs no request — with today's stamp as the fallback for an array created
413+ without one. That is a convention as much as a change, so it is left for a
414+ decision rather than settled here.
415+
416+ ### Left undone
417+
418+ - The bench of phase 6: ` bench/ndarray/cat2-block-granularity.py ` 's stand-in
419+ does not accept writes, so the write path has no measured numbers of its own
420+ against a loopback server.
421+ - Multi-worker deployment. ` locking=True ` covers the frame across processes and
422+ the ` .b2lock ` counter is read from disk, so the ETag is right there too; the
423+ per-path ` asyncio.Lock ` is not, and neither are the mtime-keyed open-array
424+ caches. Nothing here depends on it, and nothing here provides it.
0 commit comments