Skip to content

Feat(io): metadata records and the format conformance tests (#415, pass 1a) - #417

Open
balbasty wants to merge 6 commits into
mainfrom
claude/feat/415-core-records
Open

balbasty wants to merge 6 commits into
mainfrom
claude/feat/415-core-records

Conversation

@balbasty

@balbasty balbasty commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Pass 1a of #415 (design: #416, docs/design/format-records.md). This PR adds the core that the format records build on. No format is migrated yet; NIfTI follows in stacked PRs.

What it adds

  • brainhops.datamodel.metadata.Metadata: the root of the metadata model API. It has no fields yet; the vocabulary arrives with Design: generic metadata representation and cross-format metadata conversion #233.
  • brainhops.io.metadata.MetadataFormat: the file-based metadata dispatcher, built like ImageFormat and TransformationFormat (isolated registry, register_parser(Metadata), _FileBasedModelMixin).
    • Its field raw is the format-private record, hidden from repr.
    • from_raw(raw, **kw) builds a metadata object from a record.
    • to_raw() returns a copy of the record, because records are never changed in place.
    • load and sniff pick a registered <X>Metadata and read only the header.
    • The package docstring describes the pattern every format object follows: raw holds the data as stored, data is a cached view of it, and the write order is fixed.
  • smartproperty setters keep invalidates. smartproperty now returns a private property subclass whose setter/getter/deleter keep the invalidator, so a format can write @data.setter without silently losing cache invalidation. Two tests cover it. All 40 existing @x.setter sites were checked and none changes behaviour.

Changes to existing behaviour

  • SingleScaleImage.data is hidden from repr. Printing a lazily loaded image must not decode the whole file. This is a breaking change.
  • from_instance within one format keeps raw and leaves data unset. A copy then stays lazy, and io.save of an untouched object stays byte-identical, since it goes through from_instance. For this to work, DataModelBase._build no longer reads a source field that the caller already passes as a keyword. A field may be a property that reads a file.
  • _foreign_format_fields no longer counts dispatchers. Without this, every metadata format would have copied another format's record.

Tests

  • tests/test_io_conformance.py:
    • A registry table (EXEMPLARS, NOT_MIGRATED) must cover every registered format. Each pass moves formats from the second to the first.
    • It runs the 12 checks of the design note against a test-only format (ToyRaw → ToyMetadata → ToyImage, with a lazy proxy and a transposed view). The checks cover no parser state, lazy loading, setting data, round-tripping the view, byte-identical untouched saves (also through io.save), header-only metadata reads, pickling, building from data=, replace/from_instance, editing the record, the public surface and the reader/writer contract.
  • tests/test_io_public_names.py: pins __all__ of the packages the passes will touch, and checks that their submodules are private.

Review

Fable reviewed this PR, and its fixes are in 307991b. The tensions it found are logged in the design note (#416).

Checks

  • pytest (from /tmp, Python 3.8): 5477 passed, 95 skipped, 1 xpassed. On main: 5418 passed.
  • ruff check, ruff format --check and codespell (with CI's ignore-words file) are clean.

Part of #415.

🤖 Generated with Claude Code

https://claude.ai/code/session_01M3KmPTJ2CihFMCFaJ4twx8


Generated by Claude Code

claude added 2 commits October 9, 2026 21:41
)

Add the core of the format-record design, with no format migrated yet.

- `brainhops.datamodel.metadata.Metadata`: the model-API root of
  metadata, without fields until the vocabulary of #233 arrives.
- `brainhops.io.metadata.MetadataFormat`: the dispatcher of metadata
  formats, with its own isolated registry. It holds the record of a file
  in `raw`, hidden from the representation, and offers `from_raw` and
  `to_raw`, which returns a copy so that records are never changed in
  place. It inherits `_FileBasedModelMixin`, as the other dispatchers do.
- `_foreign_format_fields` no longer counts dispatchers among the formats
  that declare a field, so copying metadata from another format resets
  `raw`.
- `smartproperty` returns a property whose `setter`, `getter` and
  `deleter` decorators keep `invalidates`, so a format can write its data
  setter with `@data.setter`.
- `tests/test_io_conformance.py`: the EXEMPLARS and NOT_MIGRATED tables,
  a check that they name every registered format, the twelve checks of
  the design, and a test-only toy format that runs them end to end.
- `tests/test_io_public_names.py`: the expected public names of the
  metadata, NIfTI, LTA and ITK packages.
- API pages for the two new modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M3KmPTJ2CihFMCFaJ4twx8
…#415)

- `_FileBasedModelMixin.from_instance` does not pass `data` when it copies
  an object of the same format that has a `raw`, so the copy holds the
  same lazy `raw`. `data` still takes precedence over `raw` when both are
  given, and `replace` still decodes the data.
- `DataModelBase` copies no longer read a source field that the keyword
  arguments already set, so that the data view is not read for nothing.
- `SingleScaleImage.data` is hidden from the representation, because
  reading it may load a whole file.
- The conformance checks test that `from_instance` and `io.save` keep the
  proxy, that `repr` reads no data, and compare public field names.
- The description of format objects, and the rule that a lazy format
  overrides `from_filename`, move to the `brainhops.io.metadata` package
  docstring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M3KmPTJ2CihFMCFaJ4twx8
Comment thread src/brainhops/io/base/_base.py Outdated
for klass in cls.__mro__:
if not (isinstance(klass, type) and issubclass(klass, DataModelBase)):
continue
if "_REGISTRY" in klass.__dict__:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add is_format_registry, next to the registry decorators, and use it here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f4db838. is_format_registry(cls) now sits next to format_registry and register_format, and it replaces the "_REGISTRY" in klass.__dict__ checks in _base.py, _dispatch.py and four tests. It is not added to __all__, which keeps the public surface unchanged.


Generated by Claude Code

… or invalidators (#415)

Every setter of a smartproperty, whether the default one, one given with
fset= or one added with @name.setter, now deletes the cached value of
the property before it runs, so a property never names itself in
invalidates. invalidates takes a name, an Invalidator, or a tuple mixing
both. lazyproperty takes invalidates= instead of unset=, and accepts a
setter. drop_cache(obj, *names) is the one way for code outside the
module to delete a cached value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M3KmPTJ2CihFMCFaJ4twx8
… carries the data (#415)

store_data_argument(obj, arguments) stores a data= constructor argument
again through the data setter, after the default of raw has erased it.
Within a format, from_instance no longer copies data when raw or the
metadata record is set, so a copy decodes its data from the same record.
The conformance exemplar uses the helper and a lazyproperty data view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M3KmPTJ2CihFMCFaJ4twx8
The NIfTI affines and fields decide a within-format copy in their own
from_instance, because the data model's from_instance precedes the
format base in their method resolution order. No format reaches the
rule with metadata but no raw, so the wider rule had no effect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M3KmPTJ2CihFMCFaJ4twx8

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants