getitem call fails when metdata.fill_value is of type numpy.ndarray

[pfariborz]

Zarr version

v3.0.2

Numcodecs version

v0.14.1

Python Version

3.12

Operating System

Mac

Installation

using pip in virtual environment

Description

Call to getitem on an array is failing at this stage in the code in _get_selection line 1274 in array.py:

out_buffer = prototype.nd_buffer.create(
                shape=indexer.shape,
                dtype=out_dtype,
                order=self._config.order,
                fill_value=self.metadata.fill_value,
       )

With error:

  File "/opt/anaconda3/envs/aq_data/lib/python3.12/site-packages/zarr/core/array.py", line 1345, in getitem
    return await self._get_selection(indexer, prototype=prototype)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/aq_data/lib/python3.12/site-packages/zarr/core/array.py", line 1274, in _get_selection
    out_buffer = prototype.nd_buffer.create(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/aq_data/lib/python3.12/site-packages/zarr/core/buffer/cpu.py", line 159, in create
    ret.fill(fill_value)
  File "/opt/anaconda3/envs/aq_data/lib/python3.12/site-packages/zarr/core/buffer/core.py", line 479, in fill
    self._data.fill(value)
TypeError: only length-1 arrays can be converted to Python scalars

The metadata.fill_value for this array is: [0.+0.j 0.+0.j] of type: <class 'numpy.ndarray'>

Was able to work around this by manually running the code below:

        indexer = zarr.core.indexing.BasicIndexer(
            selection, z_shape, z_array.metadata.chunk_grid
        )
        prototype = zarr.core.buffer.core.default_buffer_prototype()
        out_buffer = prototype.nd_buffer.create(
            shape=indexer.shape,
            dtype=z_array.dtype,
            order=z_array._config.order,
        )
        return await z_array._get_selection(
            indexer, prototype=prototype, out=out_buffer
        )

The only difference between this code and the zarr source is the lack of fill_value in the out_buffer definition. The dtype of the array is complex128.

Steps to reproduce

Run a getitem call on an array with a fill_value that is of numpy.darray type.

Additional output

No response

[d-v-b]

Sorry about this bug, and thanks for the report. Zarr uses special JSON encoding for complex values that's relevant when reading / writing the fill value from metadata. It looks like the fill value attached to the metadata object is in the JSON form, when it should be a numpy scalar instead. A short-term fix would be to normalize the fill value before using it as an array fill value, and a longer-term fix would be to only use the JSON-serializable form when writing to JSON.