Skip to content

ZFP with Zarr breaks when data is a view #812

@tasansal

Description

@tasansal

With Zarr v3 and using zarr.codecs.ZFPY as serializer, the ZFP compressor logic in numcodecs complains that it is not F-order data. The chunk data neither C/F contiguous when it is a sliced view of an existing array.

Reproducer

import numpy as np
import asyncio
import zarr
from zarr.storage import MemoryStore
from zarr import create_array
from zarr.codecs import numcodecs

zfp = numcodecs.ZFPY(mode=4, tolerance=0.001)
store = MemoryStore()

zfp_arr = create_array(
    store,
    shape=(100, 100, 100),
    chunks=(50, 50, 100),
    dtype="float32",
    serializer=zfp,
    compressors=None
)

rng = np.random.default_rng()
data = rng.random(zfp_arr.shape)
zfp_arr[:] = data

Raises:

ValueError: The zfp codec does not support F order arrays. Your arrays flags were   C_CONTIGUOUS : False
  F_CONTIGUOUS : False
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
.

If we look at the chunk slices' numpy flags:

>>> data[:50, :50, :].flags.c_contiguous
False
>>> data[:50, 50:, :].flags.c_contiguous
False
>>> data[50:, 50:, :].flags.c_contiguous
False
>>> data[50:, :50, :].flags.c_contiguous
False

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions