Fix handling empty slices of a tensor

[pawel-wolff]

Description

This PR fixes a problem in handling empty slices of Xarray objects whenever the underlying data in managed by a field list through a FieldListTensor object (e.g. when a GRIB file is converted to Xarray via .to_xarray(lazy_load=True, ...) method). Consider the following code:

from earthkit.data import from_source
from earthkit.data.testing import earthkit_remote_test_data_file

ds_ek = from_source("url", earthkit_remote_test_data_file("xr_engine/level/pl.grib"))
ds = ds_ek.to_xarray(lazy_load=True)

ds2 = ds.sel(level=slice(350, 360))  # in ds2, the level-dim has size 0
ds2.load()  # crashes

ds3 = ds.load().sel(level=slice(350, 360))   # this works, because the full ds is loaded into memory 
                                             # before extracting an empty slice

The problem with ds2.load() comes from the fact that the underlying field list is empty (i.e. None, see

earthkit-data/src/earthkit/data/core/fieldlist.py

Line 1108 in 195942d

return None

).

This PR fixes this issue by properly handling the case of an empty field list, see:

earthkit-data/src/earthkit/data/indexing/tensor.py

Line 433 in dbc9810

def _to_array(self, method, index=None, **kwargs):

A related code was refactored slightly to remove some code duplication between to_numpy, to_array methods within FieldListTensor and FieldListSparseTensor classes.

A relevant test was added.

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.98%. Comparing base (195942d) to head (caa73ce).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #802      +/-   ##
===========================================
+ Coverage    85.96%   85.98%   +0.01%     
===========================================
  Files          173      174       +1     
  Lines        13450    13467      +17     
  Branches       674      676       +2     
===========================================
+ Hits         11562    11579      +17     
  Misses        1698     1698              
  Partials       190      190              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sandorkertesz]

This should be array format agnostic. The array_namespace is ideally in the kwargs.

[sandorkertesz]

The reshape on the array namespace should be used.

[pawel-wolff]

Do you mean using the top-level function xp.reshape(arr, new_shape) for an array namespace xp rather than arr.reshape(new_shape) ?

[sandorkertesz]

Yes, exactly. We try to use all these methods on the array namespace (when it supports the methods). Both empty and reshape are part of the array api.