Feat/store paths

[d-v-b]

Adds a path attribute to all the store classes. Key operations like get, set, list, list_prefix etc are now scoped to the subtree rooted by the path attribute of the store, which is how RemoteStore works in the v3 branch. This change is motivated by the desire for a consistent, scalable store API.

Key changes:

• renamed LocalStore.root to LocalStore.path, and changed the type of the attribute from pathlib.Path to str
• added a path attribute to MemoryStore
• changed the semantics of the path attribute of ZipStore, and introduced a new attribute file_path that does what ZipStore.path used to do (point to the location of the Zipfile in the parent file system).

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

Just a note, this fixes a failure in some xarray tests that write an array at the top level of the store, open a group at the root and then get the array

Edit: whoops, never mind. I missed that this only affects zarr_format=2 when testing. I'll make a new issue.

[jhamman]

Thanks for working on this @d-v-b. I'm not 100% convinced the with_mode approach is going to be substantially cleaner than our StorePath right now but I'm curious to see how this plays out.

[jhamman]

Suggested change

	def with_path(self, path: str) -> Self:
	@abstractmethod
	def with_path(self, path: str) -> Self:

Or do you think think we can do this in a generic way that applies to all stores?

[d-v-b]

I don't think we can make it generic unfortunately, because each store class might have its own extra attributes. maybe we could use __getstate__ to get around this?

[jhamman]

this store highlights the fact that not all stores have a meaningful path.

[d-v-b]

I'm not sure I agree -- if store_dict really is an arbitrary mutable mapping, there's no guarantee that we want all IO operations to be scoped to the top-level key space, and that's exactly what the path attribute is for.

[jhamman]

This feels quite confusing. IIUC, you are allowing the caller to specify a path within the zipfile as opposed to putting the root of the store at the root of the zip, is that right?

[d-v-b]

exactly! this is required to make ZipStore congruent to RemoteStore

[joshmoore]

though this may have some impact (simplifying) zarr-developers/zarr-specs#311

[d-v-b]

worth mentioning some changes to the reprs:

• In main, MemoryStore and GPUMemoryStore reprs are <store class name>://<id of ._store_dict>. This PR changes that to <store class name>://<id of ._store_dict>/<store.path>
• in main, LocalStore repr is file://<store.root>, which is allegedly wrong for a file url. In this PR, it's file:///<store.path>
• ZipStore is a bit weird, because it's a nested store: we need a wrapping file system to get to the zip file, and then inside the zip file we have another file system. In this PR i'm using the syntax introduced by @jbms for this purpose, which is basically a sequence of URLs separated by the | character, e.g. file:///<store.file_path>|zip://<store.path> (Jeremy please correct me if I'm wrong here).