Conversation
Greptile SummaryThis PR cleans up the ECMWF open-data sources by consolidating a confusing dual-method design ( Key changes:
|
| Filename | Overview |
|---|---|
| earth2studio/data/ecmwf.py | Removes unused async fetch overrides from all child classes, renames _call→fetch, sample→ensemble dimension, and fetch_wrapper→_download_wrapper; adds drop_vars calls to cleanly strip reduced coordinates. One stale # type: ignore[override] comment remains. |
Last reviewed commit: 838660f
earth2studio/data/ecmwf.py
Outdated
| pass | ||
|
|
||
| def _call( # type: ignore[override] | ||
| def fetch( # type: ignore[override] |
There was a problem hiding this comment.
Stale # type: ignore[override] annotation
The # type: ignore[override] comment on fetch is a leftover from when the base class had an @abstractmethod async def fetch that this method was overriding (with a different, incompatible sync signature). Now that the abstract async def fetch has been removed from the base class entirely, there is nothing being overridden here and the suppression is no longer meaningful.
| def fetch( # type: ignore[override] | |
| def fetch( |
The non FX ones are useful when someone just wants the analysis states used by the forecast... in the case of when a model needs history the non FX version is useful. The FX is when the lead_time needs to reference the actual forecast steps.
Right now none of the other ensemble classes output multiple ensembles. I think I prefer to keep the consistency of always returning [time, variable, lat, lon] over adding in the potential for the method to return an additional ensemble dim. But if this would be particularly useful, perhaps we can change the behaviors but other ENS models like GEFS would need a similar update. |
| pass | ||
|
|
||
| @abstractmethod | ||
| async def fetch( # type: ignore[override] |
There was a problem hiding this comment.
fetch should still be async here. Call is the synchronous. If there isnt really async implementation then this should then just pass through to the call function.
NickGeneva
left a comment
There was a problem hiding this comment.
Thanks Stefan!
The updates to the xarray dim and also the rename of the _fetch_wrapper seem good. But the removal of the async fetch has some consistency issues with the interface for the data sources.
|
I have renamed |
Earth2Studio Pull Request
Description
The ECMWF data sources had some additional, unused code and xarray coords were not cleanly removed. Changes:
Two questions:
Checklist
Dependencies