Skip to content

ci: sccache S3 caching + parallel test jobs - #47

Merged
sametd merged 2 commits into
mainfrom
ci/sccache-s3-caching
Apr 17, 2026
Merged

ci: sccache S3 caching + parallel test jobs#47
sametd merged 2 commits into
mainfrom
ci/sccache-s3-caching

Conversation

@sametd

@sametd sametd commented Apr 17, 2026

Copy link
Copy Markdown
Member

Summary

  • Add sccache with S3-compatible backend (ECMWF object store) for shared compilation caching across all CI jobs
  • Split monolithic Linux test job into 4 parallel jobs: Rust, Python 3.13, xarray+zarr, free-threaded 3.13t
  • Add cmake compiler launchers for C++ caching
  • Install sccache via brew on macOS, bake into Docker image for Linux
  • Report sccache stats at end of every job

Expected impact

  • Cold run: ~20 min (same as before, slight S3 upload overhead)
  • Warm run: ~10-14 min (30-50% reduction)

Docs Preview
https://sites.ecmwf.int/docs/tensogram/pull-requests/PR-47

sametd added 2 commits April 17, 2026 23:35
Configure sccache with ECMWF S3-compatible object store for shared
compilation caching across all CI jobs. Split the monolithic test job
into four parallel jobs (Rust, Python 3.13, xarray+zarr, free-threaded)
to reduce the Linux critical path. Add sccache stats reporting and
cmake compiler launchers for C++ caching.
@sametd
sametd merged commit d3c1405 into main Apr 17, 2026
27 checks passed
@sametd
sametd deleted the ci/sccache-s3-caching branch April 17, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant