Repository navigation
e2e: the ReproNim/containers clone should be an optional cache, not a prerequisite #147
Description
What
The e2e suite's two SimBIDS app configs (mechababs/testing/e2e/pipelines/SimBIDS-0.0.3+{anchor,chain}.yaml) carry container.source: ../containers, a path that resolves against the study root, so the suite's studies and a hand-seeded ReproNim/containers clone must share a parent (MECHABABS_E2E_WORKDIR).
conftest.simbids_sif skips the whole suite if the SIF is not already there, and the seed is a two-command recipe the skip message prints.
The shipped example config (examples/bids-app-configs/SimBIDS-0.0.3.yaml) names the same dataset by URL, https://github.com/ReproNim/containers.git, which is what a production config carries and what scaffold.container_source passes through verbatim.
So the e2e exercises a different container.source shape than the one users are told to write.
Why the path is there
babs installs container.source into every derivative it inits.
With a URL that is a fresh clone of ReproNim/containers plus a datalad get of the SIF per cell, per test run; the sibling clone makes it a local clone from an already-populated dataset.
It is a cache, and it is worth keeping as one.
Fix
- Default the e2e configs to the URL, matching the example.
- Add an env-var override (
MECHABABS_E2E_CONTAINERS=<path>) that the fixtures substitute into the app configs when set, so a local clone anywhere serves as the cache. The path need not be a sibling of the studies; the../containersassumption goes. simbids_sifstops being a skip gate: with no override, babs does the pull; with one, the fixture checks the SIF is present there.run_in_podman.shforwards the variable and bind-mounts the path when set, the way it already handlesBABS_SPECand the workdir.
Cost without the override: one clone of ReproNim/containers per cell in CI (~4 cells across the three scenarios). Measure it once; if it dominates the ~3 min podman run, the podman script can seed the cache itself instead of the user.
Context
Austin, 2026-08-22, at the chunk-3 review: "let babs do the pull itself; add an env-var override for quick tests; kill the ../containers sibling assumption."
The examples half of that is done (#133 moved them to the ReproNim URL); this is the e2e half.
Data point, 2026-09-03 (#138, the CI workflow): the seed step costs 0 s on an actions/cache hit (4 s restore, ~330 MB) and 28 s of host-side git-annex + datalad install that exists only for seeding; the image pull is 38 s regardless.
Austin, same day, asked about a URL-for-CI + env-var override and then dropped it ("nevermind"); the CI-side argument against was per-cell clone + SIF fetch inside the container vs a cached one-time seed, and that the seed is verbatim test-cluster's documented host prep.
The 2026-08-22 directive above stands as written.
Activity