Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single Writer Multiple Reader (SWMR) support? #136

Open
cnydw opened this issue Jun 12, 2024 · 2 comments
Open

Single Writer Multiple Reader (SWMR) support? #136

cnydw opened this issue Jun 12, 2024 · 2 comments

Comments

@cnydw
Copy link

cnydw commented Jun 12, 2024

While writing to my hdf5 file with swmr_mode on, I tried to open the hdf5 file with the extension, but I got the error Request failed with status code 500 on the frontend and OSError: Unable to synchronously open file on the backend.

Example can be found on https://docs.h5py.org/en/stable/swmr.html#using-the-swmr-feature-from-h5py

If we add the option "swmr": True to

h5py_options={"locking": False} if H5PY_HAS_FILE_LOCKING_ARG else {},

it can enable the swmr feature for the extension.

@loichuder
Copy link
Member

Thanks for the report. I think it makes sense to add support for SWMR but if possible, I'd like to make this a setting of the extension.

We had some troubles in the past with SWMR so I would prefer to not enable it for all users.

A quick search shows me there is a tutorial for front-end settings but I didn't find anything about backend settings. I am sure it must exist though.

A PR is welcomed, else I will do a more thorough search later to see how to do it.

@t20100
Copy link
Member

t20100 commented Jun 17, 2024

One issue is that the SWMR mode requires HDF5>=v110, so I don't think you can open files created with older versions of the HDF5 library with swmr=True (needs to be tested). However it should be possible to catch the exception raised when trying to open a file that is already opened in SWMR mode and retry opening it with swmr=True.

Another limitation is that the h5web viewer caches the data, so AFAIK, updates of the HDF5 file will not be taken into account by the viewer once the data has been loaded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants