Skip to content

Pull test data from CSCS object store#1191

Draft
msimberg wants to merge 4 commits intoC2SM:mainfrom
msimberg:rgw-test-data
Draft

Pull test data from CSCS object store#1191
msimberg wants to merge 4 commits intoC2SM:mainfrom
msimberg:rgw-test-data

Conversation

@msimberg
Copy link
Copy Markdown
Contributor

Depends on #1182. Still needs grid URLs updated and all of the data added to the object store. We can at the same time decide if the directory structure needs to be changed (though no particular need for that).

I've only added the r2b4 ape experiment data to the test bucket.

Copilot AI and others added 4 commits March 30, 2026 10:18
- Replace `wget` library with `pooch` in data_handling.py
- Use `pooch.retrieve()` with `pooch.Untar(extract_dir=".")` for downloading
  and extracting tar archives
- Remove manual temp directory management and tarfile handling
- Remove the private `_perform_download()` helper function
- Update dependency in model/testing/pyproject.toml: wget>=3.2 -> pooch>=1.7.0
- Update root pyproject.toml: wget>=3.2 -> pooch>=1.7.0
- Update uv.lock to reflect dependency changes

Agent-Logs-Url: https://github.com/C2SM/icon4py/sessions/97eae63c-abce-4bf1-90d6-1f7c34967217

Co-authored-by: jcanton <5622559+jcanton@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown

Mandatory Tests

Please make sure you run these tests via comment before you merge!

  • cscs-ci run default
  • cscs-ci run distributed

Optional Tests

To run benchmarks you can use:

  • cscs-ci run benchmark-bencher

To run tests and benchmarks with the DaCe backend you can use:

  • cscs-ci run dace

To run test levels ignored by the default test suite (mostly simple datatest for static fields computations) you can use:

  • cscs-ci run extra

For more detailed information please look at CI in the EXCLAIM universe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants