Storage backend information

[mullenkamp]

Hi,

I was thinking of making my own storage backend for zarr. I read that all that's needed is a class that implements the MutableMapping methods.
I was looking through some of the code in zarr, and I found classes that use methods called setitems, getitems, and delitems. These are the FSStore and FSMap classes. I think I understand what these three extra methods do, but I was wondering what other methods would zarr be able to utilise (or require)?

Thanks

[d-v-b]

setitems, delitems, and getitems exist because some storage backends support batched operations, e.g. deleting many keys with a single API call, and in principle using the batched API is more efficient than repeatedly calling the non-batched API (note that this efficiency claim is an assumption, and it depends on a lot of details). If you are are targeting a backend that supports batched operations for getting, setting, or deleting, then it might make sense to implement the respective methods, but they are not required by zarr.

[jhamman]

It is worth noting that we're moving away from the MutableMapping API in V3. The abstraction was always a bit leaky, evidenced by the addition of new methods like getitems.

In V3, we're moving toward a simple API tailor fit for Zarr-Python. It should be just as easy to implement storage backends with this API as a Mutable Mapping. A few pointers in case you are interested in trying out the new API:

• https://github.com/zarr-developers/zarr-python/blob/main/v3-roadmap-and-design.md#store-api
• Prototype of object-store-based Store implementation #1661

[mullenkamp]

Thank you both very much for the replies. That's very helpful.
V3 will be a big change ;)