Refactor(io): AFNI and MRtrix images hold a record and a lazy raw array (#415, pass 3b)

[balbasty]

Pass 3b of #415. It is stacked on #424 (pass 3a, MGH) → #423 → #422 → #421 → #419 → #418 → #417. Design: #416 (docs/design/format-records.md).

AFNI (.HEAD/.BRIK) and MRtrix (.mif/.mih) images now hold a record of their header and a lazy array, instead of inheriting AfniReaderWriter and MrtrixReaderWriter.

What changes

• AFNI (io.common.afni):
  • AfniRaw, formerly AfniHeader, holds the attributes of the .HEAD file and the text they were read from.
  • AfniMetadata(MetadataFormat) is new and registered.
  • AfniFormat is the shared file-name logic.
  • The BRIK is read by a lazy proxy, .BRIK or .BRIK.gz, that holds no open file.
• MRtrix (io.common.mrtrix):
  • MrtrixRaw, formerly MrtrixHeader, holds the keys and the bytes they were read from.
  • MrtrixMetadata(MetadataFormat) is new and registered.
  • The data is read by a lazy proxy that applies the stored layout.
• AfniImage and MrtrixImage:
  • Each holds raw and metadata.
  • data is a cached view of raw. It follows the array backend, is lazy with Dask, and is read-only with NumPy.
  • shape and ndim come from raw without reading the voxels. system is read-only.
  • The axis changes that depend on the record stay inside the proxies: AFNI drops the axis of a single sub-brick, and MRtrix applies its layout.
• Stored header text is written back as read for as long as a re-parse shows it still describes the record. This keeps AFNI's spacing and MRtrix's comments and padding.

Write precedence

• AFNI, to_raw(view, datatype, attributes):
  1. The attributes that describe how the BRIK stores the voxels come from the data. An untouched BRIK keeps its own.
  2. The geometry is kept as stored when the voxel-to-DICOM matrix and the shape are exactly unchanged. Otherwise it is encoded again.
  3. The view comes from view=, then the file name, then the world space, then the record.
  4. attributes= is applied last. It refuses the storage attributes and the geometry attributes (ORIGIN, DELTA, ORIENT_SPECIFIC, IJK_TO_DICOM, IJK_TO_DICOM_REAL, SCENE_DATA) with WriterError.
  5. If nothing changed, the held record is returned. Otherwise the dataset gets a new identity code.
• MRtrix, to_raw(layout, datatype, scaling, keyval):
  1. The storage keys come from the lazy array when the options match what it stores. Otherwise they come from the options.
  2. vox and transform are kept when the geometry and the shape are exactly unchanged.
  3. keyval= is merged in. It refuses dim, vox, layout, datatype, transform, scaling and file.
  4. If the header is unchanged, the held record is returned.

Visible behaviour changes

• Removed: AfniReaderWriter, AfniHeader, MrtrixReaderWriter, MrtrixHeader, and the .header, ._header and .dataobj attributes of both image classes. The record is image.metadata.raw. MrtrixRaw.from_fileobj returns the record instead of a (header, nbytes) tuple.
• data is cached, read-only and follows the array backend. Before, it was always a NumPy array or a memory map.
• An untouched dataset is written byte for byte. This includes both AFNI files and nibabel's AFNI test files; compressed files are identical after decompression. AFNI keeps IDCODE_STRING and IDCODE_DATE when untouched; before, every save gave a new identity.
• Overrides that the data or the geometry decide are refused with WriterError. Before, MRtrix dropped them silently.
• Loading a compressed file no longer reads the voxels.
• Saving under another BRIK compression keeps the image readable. A save that removes the BRIK the image reads points the image at the new BRIK.
• An AFNI dataset can be saved over its own files.

Tests

• AFNI is a conformance exemplar as .HEAD, with .BRIK and .BRIK.gz variants, and MRtrix as .mif, with a .mif.gz variant. All 12 checks pass.
  • A new harness field, Exemplar.header_file, serves formats made of two files.
  • AFNI refuses from_bytes, because one stream cannot hold a dataset.
  • .mih is tested separately, because its header names its data file.
• New tests count voxel reads. Load, io.load, shape, ndim, repr, transformations, system, to_raw and pickling read none. Accessing data under Dask reads none, and computing it reads the file once.
• Other new tests cover:
  • each write option and each refusal;
  • NaN attributes;
  • saves that remove the BRIK the image reads.

Review

Fable reviewed this pass with about 240 probes. It required two fixes:

• Data loss on save: a save under another compression removed the BRIK that the loaded image still read.
• Double read under Dask: Dask read the whole file twice, because the proxies decoded everything for its empty probe.

Both are fixed in c7e2a4e, with its nits. The tensions are logged in #416. One is open and shared with MGH: replace turns the proxy into an array.

Checks

• pytest (from /tmp, Python 3.8): 5968 passed, 95 skipped, 1 xpassed.
• ruff check, ruff format --check and codespell are clean.

Part of #415.

Changes after review

After the first review, the stack gained these changes. They are merged into every PR from #417 up, and the design note (#416) is updated to match.

• Review comments on Feat(io): metadata records and the format conformance tests (#415, pass 1a) #417–Refactor(io): NIfTI transformations hold a record and a lazy raw array (#415, pass 1c) #419.
  • smartproperty is hardened: every setter drops its own cache, and invalidates= takes names or invalidators.
  • The view modules moved next to the formats that use them.
  • from_any is defined once on TransformationFormat. One relabelling converter is registered explicitly for each format family; there is no converter priority.
  • The redundant KwOnly markers on transformations are removed.
• bagof-magic 0.3.dev3.
  • A format that decodes its transformations takes them over privately: _transformations = Inherit("transformations", default=None). None decodes from the record, and [] is honoured.
  • Forget replaces the ClassVar workarounds, inherited defaults use x: ... = v, and unset="empty" is gone.
• Pending data and consolidate().
  • Assigning data or transformations never touches raw or metadata.
  • consolidate(*, inplace=False, **options) is the only public encoding API. to_raw() is removed.
  • save, to_bytes and to_fileobj write a consolidated copy and leave the object unchanged. The exception is a save over the file the object itself reads (Saving an uncompressed NIfTI or MGH image over its own file destroys its data #426), after which the object equals load(path).
  • Transformations are compared with a fresh decode at consolidation, so reading them alone keeps the file byte-identical, and in-place edits are written.
  • A copy consolidated with a storage override holds exactly what the file holds.

The conformance harness gained checks 15–19, which cover consolidation, saving over the object's own file, storage overrides, and the transformations list.

🤖 Generated with Claude Code

https://claude.ai/code/session_01M3KmPTJ2CihFMCFaJ4twx8

---

Generated by Claude Code

---

Generated by Claude Code