Google Cloud storage

[Alexander-Barth]

I am using your package on the pangeo cloud (on google cloud) with the STAC package (https://github.com/JuliaClimate/STAC.jl). In order to use the Google's storage REST API, I needed to implement a couple of methods and a custom storage type (https://github.com/JuliaClimate/STAC.jl/blob/main/examples/Julia-pangeo-STAC-Zarr.ipynb). Thanks to the nice API of Zarr.jl this can be done without changing anything in Zarr.jl but I am wondering if it would not be better to integrate this in your package. I would be happy to create a PR.

Initially, I wanted to use GoogleCloud.jl, but there are some open PRs that need to be merged first (e.g.
JuliaCloud/GoogleCloud.jl#32 )

[meggart]

Thanks a lot for offering help here. Yes, a GCStore would be very much appreciated. So far I only experimented with publicly and freely accessible buckets that were simply accessable through AWS.jl with AnonymousGCS. It also looks like with the current implementation you would not even add a dependency, so feel free to create the PR.

One thing we might think about would be if we can use the gs:// prefix in general for your GCStore. Would it be possible to access public and free GCS buckets like the pangeo CMIP6 data without any access token using your GCStore? If yes, then we could simply delete this code https://github.com/meggart/Zarr.jl/blob/bbdc21a7b15d4fdcd124addcdb370c3e2b69f70d/src/Storage/s3store.jl#L98-L106 and have all GCS-related stuff going through your Store. Otherwise we can simply have both options.

[Alexander-Barth]

This is good idea. I just tried to access:

zds = zopen("gs://cmip6/CMIP6/CMIP/AS-RCEC/TaiESM1/1pctCO2/r1i1p1f1/Amon/hfls/gn/v20200225/")

via this url:

zds2 = zopen("https://storage.googleapis.com/cmip6/CMIP6/CMIP/AS-RCEC/TaiESM1/1pctCO2/r1i1p1f1/Amon/hfls/gn/v20200225")

and so far, it does not work (jet). I guess what is missing in my code is separation of the bucket name (cmip6) and the object prefix CMIP6/CMIP/AS-RCEC/TaiESM1/1pctCO2/r1i1p1f1/Amon/hfls/gn/v20200225. I will look into this!

[meggart]

I think this is fixed and can be closed. Thanks again for your help