[mia-autodev]

Closes #33: For JaneliaSciComp/mia-workflow-data#213: dataset zenodo-5553771-detecdiv-yeast-trap-segmentation has no snippet that reads it.

Implemented by snippets: Claude Code running in a sandboxed job on the cluster. The agent's own
summary is the first comment.

• Merge: approve this PR, or comment /approve <head sha> (needed when running under a personal token).
  snippets merges once an approval matches the current head and checks are not failing.
• Request changes: submit a review with Request changes, or comment /revise <what to change>.
  Up to 3 revision rounds.
• Stop: close this PR.

[mia-autodev]

snippets: round 1 pushed e1e4c1403b91 to snippets/issue-33 · $3.90

The commit now holds exactly the four snippet files and the working tree is clean.

Summary

Added one snippet, snippets/tiff-zip-glob-rgb/, for issue #33 and committed it on snippets/issue-33 (commit 4d37823). Nothing else in the repository changed.

What the snippet does

• Takes URLs of the form <zip>::<glob>, expands them with io.list, sorts the matches, and presents the set as one zyx array (one file per z plane). A single file is yx. Plain members and plain URLs also work and may be mixed in binding order.
• Collapses the TIFF sample (RGB) axis, which is the gap the match agent reported. samples: grey returns sample 0 after checking all samples agree. samples: onehot returns uint8 class ids 1..S by argmax over the samples, 0 where none is set, and raises on multi-hot pixels. sample: k returns one plane for a binary mask.
• Opens lazily. open_array reads only the first file, so opening the 1,404-member set costs one zip directory read plus one member. Each z plane is fetched on read. All members here are deflated, so each is staged via io.fetch. Access is reported as plane.
• The README states that the mother/other order of samples 1 and 2 follows the Zenodo text and is not documented in the files.

Binding (examples/zenodo-5553771-detecdiv-yeast-trap-segmentation.yaml): raw is trainingdataset/images/*.tif with samples: grey, label is trainingdataset/labels/*.tif with samples: onehot, paired plane for plane since both directories have identical file names (verified for all 1,404). All 1,404 pairs are bound instead of 1.

Testing

• Synthetic tiles over the harness range server, loaded through the real loader: glob expansion and mixing, grey/onehot/sample collapses, z int and slice reads, single-file yx, stored vs deflated members, and every error path. Passed.
• ci/check_pr.py static and metadata checks as author mia-autodev[bot]: no errors.
• Real-data smoke with the pinned harness:

ok: true   shapes raw [1404, 60, 59]  label [1404, 60, 59]   warnings: []   bytes_fetched: 1,090,051

• Real-data harness crop --n 3 with the pinned harness: 3 crops made, capacity 1404, no errors. The label PNGs now hold class ids, e.g. {1: 3318, 2: 122, 3: 41}, on random z planes 1132, 323 and 533, rather than one one-hot plane.

Notes for the reviewer

• This repository's own harness copy still requires y and x ≥ 256, so it would report these 60×59 tiles as ineligible. The pinned harness on PATH crops a small raw at its own extent, which is what the worker runs, so crops are 59×59.
• Each plane costs two small HTTP requests to Zenodo. Zenodo returned no errors during these runs, but the match agent saw transient 504s and the harness has no retry.
• I could not run pixi run test because pixi is not on PATH, and the repo's existing harness tests cover the other snippets, not this one.

[mia-autodev]

Test crops at e1e4c1403b91ad20af08b5beba7fad3eb39ba69a: passed, 4 of 4 verified; the crop job fetched 1098341 bytes.

raw	overlay	caption
		i213-e34373623762-b0-000 · z=0 c=None · (1, 0)
		i213-e34373623762-b0-001 · z=1067 c=None · (1, 0)
		i213-e34373623762-b0-002 · z=836 c=None · (1, 0)
		i213-e34373623762-b0-003 · z=173 c=None · (0, 0)

Binding (sha256 23e46b652adfcbc3a1b97914f25beb368a2fcce779b88bf007cd70ddbab8694f):

record: zenodo-5553771-detecdiv-yeast-trap-segmentation
record_commit: 53cc3727d612ee9b5b23da7a8b8b08d11e0756f5
snippets_commit: e1e4c1403b91ad20af08b5beba7fad3eb39ba69a
raw:
- snippet: tiff-zip-glob-rgb
  urls:
  - https://zenodo.org/records/5553771/files/cellSegmentation.zip::trainingdataset/images/*.tif
  params:
    samples: grey
labels:
- snippet: tiff-zip-glob-rgb
  urls:
  - https://zenodo.org/records/5553771/files/cellSegmentation.zip::trainingdataset/labels/*.tif
  params:
    samples: onehot
  pairs_with: 0
  label_key: manual_gt-cell-semantic
  segmentation_type: semantic
reasons:
- 'tiff-zip-glob-rgb instead of tiff-zip-member: the tiles are plain RGB TIFFs (yxs
  60x59x3 uint8, record technical.arrays) whose sample axis is not a channel axis;
  tiff-zip-member exposes it as c, so the harness would pair one one-hot label plane
  with each raw sample and write it as a class map'
- 'urls with a member glob: trainingdataset/images/*.tif and trainingdataset/labels/*.tif
  (the record''s path_pattern) expand to the 1,404 members each (listing_complete:
  true), sorted by name, stacked as z so the whole set is bound rather than the one
  pair record.yaml samples; both globs give the same sorted basenames, so z index
  k of the label is the mask of z index k of the raw (the record''s pairing rule:
  same file name in images/ and labels/)'
- 'samples grey on raw: the brightfield crop is grey stored as three identical samples
  (alignment_notes samples_identical: true); sample 0 is returned as yx after the
  snippet checks the samples agree, so each image yields one plane, not three duplicates'
- 'samples onehot on label: every mask pixel is 255 in exactly one of the 3 samples
  (alignment_notes, checked in all 1,404 masks); argmax over the sample axis + 1 gives
  class ids 1 background, 2 mother, 3 other, the record''s class ids and the Zenodo
  text''s order'
- 'label_key manual_gt-cell-semantic: annotations.source manual; background/mother/other
  mark yeast cells; a one-hot map over 3 classes is a semantic map, not instances'
- 'segmentation_type semantic: the class ids 1..3 are written as they are (encode_label
  identity); the record notes the order of samples 1 and 2 (mother, other) is presumed
  from the Zenodo text, not documented'