Wire format metadata into images and transformations (#233)

[balbasty]

Relates to #233 (parent #124). This PR holds the review history of the format-metadata work. It has been split into a stack, and this PR now keeps only the wiring: connecting the metadata system to the existing image and transformation classes.

Base. claude/feat/233-metadata-system (#310). Once #310 merges, retarget this PR to main.

What's in it

• The metadata field on images and transformations (datamodel/base.py, _transformations/base.py). Each image or transformation holds its own copy of its metadata.
• Derivation through the image operations:
  • SingleScaleImage.__getitem__, reslice and __call__ derive the metadata from the image's own geometry, through the private hooks _select/_reslice.
  • MultiScaleImage.__call__ keeps the pyramid's metadata.
  • The OME-Zarr multiscale reader gives each level its metadata.
• Readers and writers of NIfTI, MGH, Zarr/OME-Zarr, x5, ITK, FLIRT, SPM and NiftyReg:
  • they read the metadata, and write it back with change detection;
  • they report losses through the check_raw read-back check;
  • they store values with preferred_storage.
• Save: io.save reports losses for the whole write.
• Tests: the image, I/O and round-trip tests that need .metadata, which come back from Format-metadata framework: vocabulary, formats and conversions (without wiring) #310.
• The user guide (docs/start/metadata.md), run as a doctest.

Behaviour after merging main

• Time step: the NIfTI pixdim[4] and the MGH TR come from the image's transformations. A metadata value that disagrees is reported, and is written only when the geometry has no time step.
• Equality: images and transformations compare by identity, as on main.

Stack

1. Design memo for the format-metadata framework #306: design memo
2. Add vocabulary enums for spaces, intents, manufacturers and microscopy #307: vocabulary enums
3. Add core helpers: shortest_decimal, EnumConverter, own_annotations #308: core helpers
4. Extract the FormatDispatcher mixin; accept non-tuple image indices #309: FormatDispatcher mixin and small fixes
5. Format-metadata framework: vocabulary, formats and conversions (without wiring) #310: the metadata system
6. this PR: wiring, stacked on Format-metadata framework: vocabulary, formats and conversions (without wiring) #310

Follow-ups: #297 (dispatchers vs formats), #293 (4-D reslice, fixed on main), #290 (lazy imports).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VmK1jLTwd2hMkvYnMWEyYw

[balbasty]

Suggested change

	raw: tx.Any = Field(None, repr=False) # format-private record (M6)
	raw: NoRepr[tx.Any] = None # format-private record (M6)

[balbasty]

Done in 66504fa. The memo's §3 snippet now reads raw: NoRepr[NoEq[tx.Any]] = None # format-private record (M6): bagof accepts the subscript form, and NoEq is there because the prototype leaves raw out of == too. The code keeps Annotated[..., NoRepr(), NoEq()] so that the field can carry its Doc.

---

Generated by Claude Code

[balbasty]

But some of the user-provided metadata (e.g. moving/reference in FLIRT) could end up in the FLIRT class' metadata, which is then not so opaque.

[balbasty]

Agreed, FLIRT isn't opaque. In c036367 FlirtMetadata subclasses FileBasedMetadata directly, with supports=("moving", "fixed") and raw: None. It keeps both fields in memory, through copies and conversions, and a write reports them as lost. OpaqueMetadata now means "stores nothing, not even in memory" (ITK .tfm/.mat, matrix text), and memo §3, §11 and M2 say so (66504fa).

---

Generated by Claude Code

[balbasty]

Suggested change

	| `diffusion_bvalues` | (N,) float, s/mm² | MRtrix `dw_scheme` col 4, NRRD `DWMRI_b-value` x gradient norms, BIDS `.bval` |
	| `diffusion_bvectors` | (N,3) float, unit vectors in **world (RAS) coordinates** | MRtrix `dw_scheme` cols 1-3 (already world), NRRD gradients via `measurement frame`, BIDS `.bvec` via the voxel-to-world rotation |
	| `bvalues` | (N,) float, s/mm² | MRtrix `dw_scheme` col 4, NRRD `DWMRI_b-value` x gradient norms, BIDS `.bval` |
	| `bvectors` | (N,3) float, unit vectors in **world (RAS) coordinates** | MRtrix `dw_scheme` cols 1-3 (already world), NRRD gradients via `measurement frame`, BIDS `.bvec` via the voxel-to-world rotation |

(Are there other bval/bvec that could be confused with the diffusion ones?)

[balbasty]

Renamed to bvalues/bvectors in the code (c036367) and in memo §4.3 (66504fa). None of the formats we read has b-values or b-vectors with a non-diffusion meaning: NRRD DWMRI_b-value, MRtrix dw_scheme and BIDS .bval/.bvec are all diffusion. The DiffusionMetadata group now gives them that context.

---

Generated by Claude Code

[balbasty]

I do think that the data type (ITK's precision) should be in the metadata, since the dtype of the loaded volume may different from the dtype on disk.

[balbasty]

Added data_type in c036367. It is a numpy dtype in native byte order (byte order stays an encoding detail), in DisplayMetadata, with grid scope so that derive(grid_changed=True) clears it.

• It maps to NIfTI datatype and MGH type.
• Zarr/OME-Zarr derive it from their arrays.
• ITK precision stays in the raw record for now.

One deviation from "dtype= > data_type > array dtype": data_type is used only when it is the same kind as the array (an integer type for integer values, a float type for floats). Otherwise a resampled uint8 label map would be quantised, since data-model operations don't call derive() yet. A hand-set type that can't be used is reported as approximated (memo §4.4, §6.2 and M1, 66504fa).

---

Generated by Claude Code

[balbasty]

I'd feel better if it was a list of (positively) supported fields. Makes it easier to extend the vocabulary.

[balbasty]

Done in c036367. supported_fields is now the primary ClassVar. It is built from supports=, which also accepts group classes. unsupported_fields is derived from it (the rest of the vocabulary) and kept only because the write and conversion loops need the complement. A field added to the vocabulary is unsupported everywhere until a format opts in.

---

Generated by Claude Code

[balbasty]

Why is this class-specific?

[balbasty]

It didn't need to be; it was only on the class because it was computed once the class existed. In c036367 it is a module constant, VOCABULARY (the field names, group by group), and GROUPS maps each group class to its field names.

---

Generated by Claude Code

[balbasty]

Suggested change

	Maybe[str],
	Maybe[tx.Union[SpaceEnum, str]],

So that known spaces are accessible somewhere.

[balbasty]

Done in c69c662/c036367. space is Maybe[Union[Space, str]], where Space holds the NIfTI code names and the BIDS standard and non-standard spaces. bagof does not turn a str into the enum member of such a union on its own, so a small converter does it. An unknown label stays a string. input_space and output_space use the same type.

---

Generated by Claude Code

[balbasty]

Suggested change

	Maybe[str],
	Maybe[tx.Union[IntentEnum, str]],

Same idea

[balbasty]

Done in c69c662/c036367: intent is Maybe[Union[Intent, str]], and Intent lists the NIfTI intent names, spelled as in nibabel. This also settles open question 6 of the memo (66504fa).

---

Generated by Claude Code

[balbasty]

I know BIDS specifically asks for [-] i/j/k, but in our case, should we use axes names from the image system? Or even accept an oriented vector in some coordinate system (helps with transformed images, in which case the PE is not axis-aligned anymore)

[balbasty]

Done in c036367, as an oriented vector in a named system: EncodingDirection(vector, space=None). None means the image's voxel axes, which is the BIDS frame.

• It accepts "j-" and compares equal to it, so BIDS/MRtrix strings and NIfTI dim_info convert without loss; NIfTI's missing polarity is still reported as approximated.
• An oblique direction is reported as lost where a format can only store an axis.
• derive(grid_map=...) maps a direction in voxel axes through the old-to-new voxel map instead of clearing it. A direction labelled with a world space is kept as is, since resampling doesn't move it.

Axis names are the special case: an axis-aligned vector in voxel space.

---

Generated by Claude Code

[balbasty]

I'd prefer a type alias or a __class_getitem__.

Something like

class _MetadataField:
    def __class_getattr__(cls, hint, *args):
        return Annotated[
             hint,
             ConvertTo(_Copied(hint)),
             KwOnly(),
             NoRepr(),
             NoEq(),
             *args,
        ]

metadata: _MetadataField[Metadata, tx.Doc("some metadata")]
nifti_metadata: _MetadataField[NiftiMetadata, Factory(), t.Doc("some other metadata")]

[balbasty]

Done in c036367 with your __class_getitem__ shape. MetadataField[hint, *annotations] gives Annotated[hint, ConvertTo(_EnsureCopy(hint)), KwOnly(), NoRepr(), NoEq(), *annotations], used as MetadataField[NiftiMetadata, Factory(NiftiMetadata), tx.Doc("...")]. metadata_annotation and the io-side metadata_field helpers are removed.

---

Generated by Claude Code

[balbasty]

I like instance.to(type[Metdata]) better, symmetric to images/transformations.

[balbasty]

Added in c036367: metadata.to(cls=None, *, on_loss=None, report=None, **values).

• cls is a class or a format name; None keeps the class, as Transformation.to does.
• Pass report= to get the losses back, as update_raw already allows. The loss policy then applies only if on_loss is given too.

The module-level convert and the convert_metadata export are gone.

---

Generated by Claude Code

[balbasty]

Why don't we use nibabel's MGHFooter?

[balbasty]

nibabel (5.4.2) has no MGHFooter. The footer fields (tr, flip_angle, te, ti, fov) are part of MGHHeader, whose hf_dtype covers header and footer, and that header is what we store. Our own code only handles the tag stream after the footer (TAG_CMDLINE, the command history), which nibabel neither reads nor writes. The MghRaw docstring and the memo now say so (c036367, 66504fa).

---

Generated by Claude Code

[balbasty]

I am assuming raw must be of a type compatible with the metadata type? Otherwise _decode will fail, no? When would a key decoded from a compatible raw not be compatible with the class itself?

[balbasty]

Right: raw is always the format's own, so this check never fires because of the data. It fires when a format's _decode and its supports= disagree, i.e. _decode returns a field the class doesn't declare. Without the check, that value would be dropped silently. I kept it as a cheap check against programmer errors and reworded its docstring (c036367). The same check now also refuses a Lazy value for a field not declared in lazy=.

---

Generated by Claude Code

[balbasty]

not sure about the difference with from_raw. especially because we use record or raw for the same thing. It would be good to have a more consistent vocabulary (update_from_raw?)

[balbasty]

Agreed; it's one word now, "raw" (c036367).

• from_raw builds.
• update_from_raw (was with_record) rebinds to a new raw record and keeps this object's changes.
• update_raw writes the changes into a raw record.

The rest follows: _check_raw, MghRaw, OmeZarrRaw, and "raw record" in the docstrings, the user guide and the memo.

---

Generated by Claude Code

[balbasty]

Fifth round: two hooks, and the metadata dispatcher (07e3c68…e9fbbbb)

A format now implements two private hooks, _decode_raw(raw, *, image=None) -> dict and _encode_raw(raw, changed, *, image=None, report) -> raw, plus the public parser methods for its files.

The record type is a type argument. It is declared as FileBasedMetadata[RawT], for example NiftiMetadata(..., FileBasedMetadata[nb.Nifti1Header]). A blank record is RawT().

These private methods are gone:

• _default_raw, replaced by the type argument above.
• _geometry, replaced by a generic read-back check. The public check_raw(), called by all five writers, decodes the finished record and reports any field that doesn't read back as the user set it.
• _check_raw, replaced by check_writable(*, image=None, raw=None).
• _derive_raw, replaced by NIfTI overriding _reslice with super().
• _read_raw/_write_raw, replaced by public from_fileobj, from_filename, sniff_h5/from_h5, sniff_node/from_node and to_file.

The metadata dispatcher is split from the parsers:

• MetadataParser(FileParser) is a plain parser mixin.
• FileBasedMetadata(FormatDispatcher) owns the metadata registry. It isn't a FileBasedObject, so io.load never returns metadata.
• Metadata.load delegates to it.

The io-wide part of the dispatcher split is #297.

Open points:

1. BIDS registration: BidsSidecar isn't a FileBasedMetadata, so it is added to the registry by hand (FileBasedMetadata._REGISTRY.add(...) in io/metadata/bids.py). Keep that, add a small helper, or turn the sidecar reader into a metadata class?
2. _import: the key/value conversion hook on Metadata is now the only other private override point. No format uses it yet. Keep it, or drop it?
3. Lazy export: FileBasedMetadata is exported lazily from brainhops.datamodel.metadata, because _filebased now imports brainhops.io.

---

Generated by Claude Code