DOC: Add a chapter on how to use icclim.create_optimized_zarr_store

[bzah]

Description

The new rechunking feature is available but the documentation for it is limited.

We could either create a dedicated chapter (rst document) or add a section in our dask.rst.

The doc should describe

• why this feature was needed
• how to use it
• what are the limits

Additionally, a didactic notebook using this feature would be great.

[pagecp]

Yes I think a dedicated chapter + a notebook is needed.

[bzah]

I have added a section in "the art of chunking"
I think it make sense to have this within the dask/chunking/perfs chapter.

[scorbarieu]

hi, happy to join the icclim community. I check the section on the art of chunking and gave it a try on TX90p. It seems I have an issue with chunking parameter which is not recognised:

CODE
using icclim 5.1.0

with icclim.create_optimized_zarr_store(
    in_files= ds,
    var_names="tasmaxAdjust",
    target_zarr_store_name="opti.zarr",
    keep_target_store = False,
    chunking={"time": -1, "i":"auto", "j":"auto" },
) as opti_tas :
     icclim.index(
        index_name="TX90p",
        in_files=opti_tas,
        slice_mode="month",
        base_period_time_range=[base_dt1, base_dt2],
        time_range=[dt1, dt2],
        out_unit='%',
        out_file=out_f_TX90p,
    )

ERROR

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [42], in <cell line: 6>()
      1 start_time = time.time()
      2 #catch wanring does not work like this
      3 #with warnings.catch_warnings():
      4 #    warnings.filterwarnings(action='ignore', message='All-NaN slice encountered')
----> 6 with icclim.create_optimized_zarr_store(
      7     in_files= ds,
      8     var_names="tasmaxAdjust",
      9     target_zarr_store_name="opti.zarr",
     10     keep_target_store = False,
     11     chunking={"time": -1, "i":"auto", "j":"auto" },
     12 ) as opti_tas :
     13      icclim.index(
     14         index_name="TX90p",
     15         in_files=opti_tas,
   (...)
     20         out_file=out_f_TX90p,
     21     )
     23 #ds_tx90p = icclim.index(index_name='TX90p', in_files=ds, var_name='tasmaxAdjust', slice_mode='month', base_period_time_range=[base_dt1, base_dt2], time_range=[dt1, dt2], out_unit='%', out_file=out_f_TX90p, logs_verbosity='HIGH')

File /usr/local/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py:263, in contextmanager.<locals>.helper(*args, **kwds)
    261 @wraps(func)
    262 def helper(*args, **kwds):
--> 263     return _GeneratorContextManager(func, args, kwds)

File /usr/local/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py:87, in _GeneratorContextManagerBase.__init__(self, func, args, kwds)
     86 def __init__(self, func, args, kwds):
---> 87     self.gen = func(*args, **kwds)
     88     self.func, self.args, self.kwds = func, args, kwds
     89     # Issue 19330: ensure context manager instances have good docstrings

TypeError: create_optimized_zarr_store() got an unexpected keyword argument 'chunking'

[bzah]

Hi @scorbarieu, this is an issue with our documentation I was just fixing a few minutes ago !
The "chunking" attribute is only available in icclim 5.2 which is yet to be released.

You should check the stable version (5.1.0) of RTD here: https://icclim.readthedocs.io/en/stable/

In the future, "stable" documentation should be displayed by default on readthedocs and "latest" will be available on the version selector.

---

Note that, without "chunking" parameter, the create_optimized_zarr_store behavior is to create a zarr store where time is unchunked and other dimensions are "auto" chunked.

[scorbarieu]

Oh thanks sorry I did not pay attention! Thanks I will try that!