Refactor(io): NIfTI images hold a record and a lazy raw array (#415, pass 1b)

[balbasty]

Pass 1b of #415. It is stacked on #417 (pass 1a), so the base is claude/feat/415-core-records. Design: #416 (docs/design/format-records.md).

NIfTI images now hold a record of their file instead of inheriting NiftiReaderWriter.

What changes

• brainhops.io.common.nifti gains two public classes and a private module:
  • NiftiRaw: the header record, with extensions. It reads and writes the header only, sniffs files, and its to_filename gzips on .gz. It is built on the module helpers read_nifti, which returns the record and a lazy proxy, and write_nifti.
  • NiftiMetadata(MetadataFormat): a registered metadata format holding a NiftiRaw. NiftiMetadata.load(path), like MetadataFormat.load, reads only the header and never touches the voxels.
  • _views.py: the image's to_model/to_disk pair. A NumPy array decoded from a file is read-only.
• NiftiImage holds a metadata (a NiftiMetadata) and raw, the data as stored: nibabel's ArrayProxy after a load, or an array once data has been assigned. data is a cached view of raw, and transformations is a cached view of the header, cleared when metadata is reassigned.
• Write order: the base header is a copy of the record. Its structural fields are reset, the geometry is encoded from the model, then like= and the overrides are applied. The header is written first, then the data. An untouched proxy is written unscaled with the record's dtype and scl_*, so the file is copied exactly.

Visible behaviour changes

• Saving starts from the file's own header. descrip, aux_file, the intent, cal_*, slice_*, toffset, dim_info and the extensions survive a save, and NIfTI-2 files stay NIfTI-2. Before, each save started from a fresh header.
• An untouched scaled file is no longer rescaled. The same applies to a save through io.save.
• A record whose spatial unit is unknown is written back as unknown, not as mm.
• Data read from a file is read-only. Assign data to change voxels; an in-place edit raises instead of being silently lost.
• img.transformations is the same cached list each time, so in-place appends persist.
• Removed from NiftiImage:
  • the attributes image and header, now raw and metadata.raw.header;
  • sniff_nibabel, now NiftiRaw.sniff_*;
  • the image=/header= constructor arguments;
  • the setter of system, which is now read-only.
• like= accepts a NiftiImage, and the FSL FLIRT/FNIRT geometry reads its record.
• NiftiReaderWriter stays exported, for the NIfTI transformations, until pass 1c.

Tests

• NiftiImage and NiftiMetadata move into the conformance EXEMPLARS. All 12 checks pass on .nii and on .nii.gz.
• Extra cases cover an int16 file with scl_slope (both suffixes, and through io.save), a NIfTI-2 file, header-only reads of a truncated file, gzipped metadata, and read-only data.
• The sniff, write, remote, reslice, units, MRO and MGH tests are updated for the removed names.

Review

Fable reviewed this PR, and its fixes are in e02244f: read-only views with the proxy always written, gzipped metadata on .gz, and single header parses. The tensions found are logged in #416 (items 24–32 and updates to 4, 6, 13 and 18).

Checks

• pytest (from /tmp, Python 3.8): 5513 passed, 95 skipped, 1 xpassed.
• ruff check, ruff format --check and codespell are clean.

Part of #415.

Changes after review

After the first review, the stack gained these changes. They are merged into every PR from #417 up, and the design note (#416) is updated to match.

• Review comments on Feat(io): metadata records and the format conformance tests (#415, pass 1a) #417–Refactor(io): NIfTI transformations hold a record and a lazy raw array (#415, pass 1c) #419.
  • smartproperty is hardened: every setter drops its own cache, and invalidates= takes names or invalidators.
  • The view modules moved next to the formats that use them.
  • from_any is defined once on TransformationFormat. One relabelling converter is registered explicitly for each format family; there is no converter priority.
  • The redundant KwOnly markers on transformations are removed.
• bagof-magic 0.3.dev3.
  • A format that decodes its transformations takes them over privately: _transformations = Inherit("transformations", default=None). None decodes from the record, and [] is honoured.
  • Forget replaces the ClassVar workarounds, inherited defaults use x: ... = v, and unset="empty" is gone.
• Pending data and consolidate().
  • Assigning data or transformations never touches raw or metadata.
  • consolidate(*, inplace=False, **options) is the only public encoding API. to_raw() is removed.
  • save, to_bytes and to_fileobj write a consolidated copy and leave the object unchanged. The exception is a save over the file the object itself reads (Saving an uncompressed NIfTI or MGH image over its own file destroys its data #426), after which the object equals load(path).
  • Transformations are compared with a fresh decode at consolidation, so reading them alone keeps the file byte-identical, and in-place edits are written.
  • A copy consolidated with a storage override holds exactly what the file holds.

The conformance harness gained checks 15–19, which cover consolidation, saving over the object's own file, storage overrides, and the transformations list.

🤖 Generated with Claude Code

https://claude.ai/code/session_01M3KmPTJ2CihFMCFaJ4twx8

---

Generated by Claude Code

---

Generated by Claude Code

[balbasty]

I think invalidated can be a string (not in a tuple), no?

[balbasty]

Yes. A string was already accepted, so only the call sites changed. They now read invalidates="transformations" (eded01e). invalidates= also takes an invalidator object, or a tuple mixing names and invalidators (80efb0d).

---

Generated by Claude Code

[balbasty]

Does it make sense to invalidate itself? I thought all smart properties cleared their cache when assigned a new data by a user.

[balbasty]

You were right for the default setter, but not for a setter given with fset= or @x.setter. Those did not drop their own cache, which is why data named itself here.

This is fixed in smartproperty (80efb0d): every setter now drops the property's own cache before it runs. data is now a plain @lazyproperty with a setter, and no longer invalidates itself (eded01e).

---

Generated by Claude Code

[balbasty]

I really don't like this unset=empty thing, seems so fragile (what if I actually want no transformations?). Maybe we should go back to allowing None transformations? Discuss.

[balbasty]

Agreed that unset="empty" is fragile, and it is also wrong today. Assigning [] to a loaded NiftiImage is silently ignored: the getter reads [] as unset and decodes the header again. As a result an image can never drop its geometry, which breaks the "honoured or refused" rule of the memo.

The root cause is the data model. Its default transformations = () is written through the format's setter by the constructor, so the format cannot tell the default apart from an empty list someone chose on purpose.

Proposal: allow None in the data model, the way Transformation already stores _input and _output.

• SingleScaleImage gets _transformations: Optional[List] = None, behind a transformations property that returns [] when nothing is stored.
• Formats use a plain @smartproperty(cache=True).
• None means "decode from the record".
• [] means "no geometry" and is written as such.
• replace() stays lazy, because it copies None.

There is one explicit consequence, in _FileBasedModelMixin.from_instance:

• Within a format it passes transformations=None, because the record carries the geometry, as raw carries the data.
• Across formats it passes the decoded list.

The constructor signature is unchanged: transformations stays positional. The model part would go in #417, and the format part in #418 and the later image PRs.

I'm holding this one until you confirm. The other comments are being addressed now.

---

Generated by Claude Code

[balbasty]

Decision recorded: None only on the classes that implement a property on top of it, i.e. the formats. The data model is unchanged.

That idiom cannot be written yet, because of bagof:

• A format that redeclares transformations: Default[Optional[List], None] beside @smartproperty def transformations gets the property itself as the field default. bagof treats a class attribute as the default, as dataclasses do (magic/_magic.py:1965).
• A private _transformations field is refused, because it shares the public name of the model's transformations.

The NIfTI transformations need nothing: their sequence bases already store _transformations: Optional = None. So NiftiImage keeps unset=(None, "empty") until one of these is decided:

• (a) A bagof change: when an annotation gives an explicit default, use it rather than a property of the same name defined on the class. Recommended.
• (b) The readers pass transformations=None explicitly. Then NiftiImage(metadata=md) built by hand would no longer decode its geometry.

---

Generated by Claude Code

[balbasty]

Done with bagof-magic 0.3.dev3 (520c2d7 here; merged up the stack, with the other image formats in #424, #425 and #427). Each format that decodes its transformations takes the model's field over privately:

_transformations: Optional[List[Transformation]] = Inherit("transformations", default=None)

@smartproperty(cache=True)
def transformations(self): ...  # None decodes from the record; a stored list, even [], is used as is

This applies to NiftiImage, MghImage, AfniImage, MrtrixImage, NrrdImage and MincImage. The data model is unchanged, and so is the signature (data, transformations, *, raw, metadata).

• [] is honoured. It is kept on the image and on its copies, and written with the same geometry as an image built from bare data. In Refactor(io): NRRD and MINC images hold a record and a lazy raw array; saves over a read file keep its data (#415, pass 3c) #427, the NRRD writer no longer brings back the old spacings for [].
• None decodes from the record. replace and from_instance within a format stay lazy.
• Copies to the model or to another format carry the decoded list.

Conformance checks 13 and 14 cover this on every image exemplar. unset="empty" remains only in OmeZarrImage, which takes its fields over the same way next.

---

Generated by Claude Code