-
-
Notifications
You must be signed in to change notification settings - Fork 282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FSStore silently returns bogus chunk data when FSMap passes an exception #1578
Comments
If I understand this issue correctly, the problem is that zarr is fine with one particular exception (the one raised when a chunk is missing), as that gets handled via the |
cc: @martindurant |
@d-v-b I think you understand correctly. Why can't zarr's FSStore leave |
cc #1237 tries to fix this, but I don't remember why it's not complete. |
@martindurant I rebased your PR on main (locally, I mean, since I'm not a zarr dev) and saw the same pytest failures reported by the checks on your PR. Comments added there. |
Zarr version
2.16.1
Numcodecs version
0.11.0
Python Version
3.10.12
Operating System
Linux
Installation
using conda
Description
The
zarr.storage.FSStore
chunk store currently ignores an important exception that the underlyingfsspec.mapping.FSMap
is able to provide when callinggetitems
. The FSMap detects when the reference file system is not reachable (because of network problems or missing/invalid credentials) and returns an exception. But because the FSStore.getitems method sets theon_error
argument to "omit", the FSStore treats the values as missing. They're not missing, there was an error that the user needs to know about.Steps to reproduce
The returned value is bogus if the credentials are invalid, as above, but no error is raised. The correct value for "time" is
array([9685260.])
.Additional output
No response
The text was updated successfully, but these errors were encountered: