No such file or directory when replacing an array

[csparker247]

Zarr version

3.0.4

Numcodecs version

0.15.1

Python Version

3.12.8

Operating System

Mac

Installation

using pip to a venv

Description

I'm trying to replace (or create if it doesn't exist) an array that's inside an existing, local zarr group. Since the array I want to add may or may not be the same size as the existing array, I'm trying to delete the existing entry and create a new array fresh using del. Yes, I know I can check the existing shape to avoid an unnecessary deletion. Similar to #2334, the call to del raises a FileNotFoundError:

Traceback (most recent call last):
  File "/Volumes/REDACTED/test.py", line 13, in <module>
    del root['labels']
        ~~~~^^^^^^^^^^
  File "/Users/REDACTED/venv/lib/python3.12/site-packages/zarr/core/group.py", line 1900, in __delitem__
    self._sync(self._async_group.delitem(key))
  File "/Users/REDACTED/venv/lib/python3.12/site-packages/zarr/core/sync.py", line 208, in _sync
    return sync(
           ^^^^^
  File "/Users/REDACTED/venv/lib/python3.12/site-packages/zarr/core/sync.py", line 163, in sync
    raise return_result
  File "/Users/REDACTED/venv/lib/python3.12/site-packages/zarr/core/sync.py", line 119, in _runner
    return await coro
           ^^^^^^^^^^
  File "/Users/REDACTED/venv/lib/python3.12/site-packages/zarr/core/group.py", line 755, in delitem
    await store_path.delete_dir()
  File "/Users/REDACTED/venv/lib/python3.12/site-packages/zarr/storage/_common.py", line 161, in delete_dir
    await self.store.delete_dir(self.path)
  File "/Users/REDACTED/venv/lib/python3.12/site-packages/zarr/storage/_local.py", line 216, in delete_dir
    shutil.rmtree(path)
  File "/opt/homebrew/Cellar/[email protected]/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/shutil.py", line 759, in rmtree
    _rmtree_safe_fd(stack, onexc)
  File "/opt/homebrew/Cellar/[email protected]/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/shutil.py", line 703, in _rmtree_safe_fd
    onexc(func, path, err)
  File "/opt/homebrew/Cellar/[email protected]/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/shutil.py", line 700, in _rmtree_safe_fd
    onexc(os.unlink, fullname, err)
  File "/opt/homebrew/Cellar/[email protected]/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/shutil.py", line 698, in _rmtree_safe_fd
    os.unlink(entry.name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'foo/01234/labels/c/1/0'

If I try to run my script multiple times after this failure, I get the following:

1. Prints UserWarning: Object at labels is not recognized as a component of a Zarr hierarchy. but succeeds.
2. Fails with the FileNotFound error.
3. UserWarning...
4. UserWarning...
5. (and all attempts after this) Succeeds.

This leads me to believe that there is some race condition with the OS or filesystem, and eventually something starts caching the correct operations so that the deletion succeeds every time.

Steps to reproduce

If I run the following script with my full zarr, I get the errors as described. Unfortunately, if I try to create a new, minimal dataset from scratch with only one entry, this script does not reproduce the error.

import zarr
import numpy as np

# create a replacement array
labels = np.ones((47, 512, 512), np.uint8)

# open the zarr and select a sub-group
z = zarr.open('foo', mode='r+')
root = z['01234']

# delete the labels array
if 'labels' in root.keys():
    del root['labels']

# create and assign the new labels array
a = root.create_array('labels',
                      dtype=labels.dtype,
                      shape=labels.shape,
                      chunks=(12, 64, 64),
                      shards=(24, 512, 512),
                      config={'write_empty_chunks': False})
a[...] = labels

Additional output

No response

[d-v-b]

hi @csparker247, thanks for the bug report. It's possible that this was previously reported here. In any case, we definitely need to fix this. I don't have the bandwidth for it now but hopefully another member of the python dev team can look into it

[csparker247]

Thanks. It doesn't look exactly like the same thing, but it certainly seems to be related behavior. I wasn't aware of the overwrite parameter, either, so now I'm curious if using it will give me the same errors...