Fixes serialisation issue #788

[pawel-wolff]

Description

Fixes the issue #788.

Added a notebook which sets up a local dask cluster and converts a small GRIB dataset on dask workers.

The fix was tested on a larger-than-memory GRIB dataset.

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.66%. Comparing base (66cddef) to head (3bbd91c).
⚠️ Report is 138 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #791      +/-   ##
==========================================
- Coverage   90.02%   85.66%   -4.36%     
==========================================
  Files         158      172      +14     
  Lines       12086    13175    +1089     
  Branches      593      637      +44     
==========================================
+ Hits        10881    11287     +406     
- Misses       1023     1698     +675     
- Partials      182      190       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sandorkertesz]

@pawel-wolff, thank you for this fix!

My comments:

Please add the new notebook to docs/examples/index.rst so that it would appear in the docs sidebar.

I also suggest you change this sentence in the notebook:

It contains 2 fields per day (at 0 and 12 UTC).

to

It contains 2 fields per day (at 0 and 6 UTC).

since the file contains these times only. I would like to ask you to do the same change in xarray_engine_chunks.ipynb too.

[sandorkertesz]

@pawel-wolff, just another request: please add xarray_engine_chunks_on_dask_cluster.ipynb to the SKIP list in tests/documentation/test_notebooks.py. We need to add some control as to when to run this notebook test in the future but the best is to skip it now.