A reproducible scientific imaging pipeline using Python/OpenCV, Nextflow DSL2, Apptainer, and Slurm. Versioned inputs, deterministic scientific outputs, and run records let researchers trace measurements and repeat an analysis.
See the Product Requirements Document for the MVP scope, architecture, testable requirements, acceptance criteria, and delivery plan.
Two routines run through the same workflow: the synthetic demo-cv-v1 image routine (also exercised on real BBBC039 microscopy) and fsl-bet-volumetry-v1, which runs containerized FSL brain extraction and volumetry over public OpenNeuro T1w MRI on Slurm, with byte-identical reruns.
This is a development candidate. TASKS.md records verification and open release gates. No software/data DOI has been published. The supplied routine is the documented synthetic demo-cv-v1 reference; no domain routine was supplied. Open OnDemand is an optional future integration.
Prerequisites: a Linux x86-64 host, or Docker Desktop with Linux containers; Git; Python 3.12 for the build helpers. Allow at least 100 GiB free disk. The PRD benchmark environment requires 16 GiB RAM and eight logical CPUs. Installation/download time is separate from the tutorial.
Follow installation to create the disposable Linux lab and SIF. Run tutorial commands in the lab as researcher, from /workspace:
docker exec -it -u researcher reprohpc-lab bash
python reprohpc doctor --profile local
SIF=/scratch/reprohpc.sif
SIF_SHA=$(sha256sum "$SIF" | cut -d ' ' -f1)
python reprohpc run --profile local --params-file params/demo.yaml \
--sif "$SIF" --sif-sha256 "$SIF_SHA" \
--outdir /scratch/tutorial/first --work-dir /scratch/tutorial/work \
--launch-dir /scratch/tutorial/launch
python reprohpc verify --run /scratch/tutorial/first
python reprohpc compare --expected tests/expected/demo --actual /scratch/tutorial/firstThese commands execute all four workflow steps through Apptainer. A successful invocation prints Status: success, an output path, and a resume command. Copy the result directory to the host with docker cp reprohpc-lab:/scratch/tutorial/first artifacts/tutorial-result and open artifacts/tutorial-result/report/index.html. Keeping the directory structure preserves links to the tables and provenance.
The results workspace provides an overview, searchable image explorer, sortable measurements, and scientific provenance. Open the report directly or serve the completed run locally; all views work offline.
| Episode | Time budget after installation | Question and exercise | Check |
|---|---|---|---|
| Understand inputs | 3 min | Open data/demo/1.0.0/samples.csv, dataset.json, and the calibration reference. Which bytes are versioned? |
Locate a sample SHA-256, the manifest hash, and 0.5 micrometers/pixel calibration. |
| First analysis | 7 min | Run the commands above; predict whether the blank image has objects. | Twelve sample folders; blank image has count zero, null mean area, and NO_OBJECTS. |
| Interpret results | 4 min | Read the report, summary/images.csv, and samples/square/objects.csv. |
Explain pixel versus square-micrometer area and a border QC flag. |
| Change science | 4 min | Repeat with --threshold 200 and a new output directory. Predict changes before comparing. |
Resolved parameters contain 200; a changed scientific result is expected and must not pass the default golden comparison. |
| Recover work | 5 min | Use the exact printed resume command, retaining the launch and work directories. | Intact analysis tasks say CACHED in provenance/trace.tsv. See the interrupted-run exercise in the operations guide. |
| Preserve and cite | 3 min | Run export below; inspect the archived source and manifests. | Verification succeeds before export. Citation identifies this unpublished candidate accurately. |
python reprohpc export --run /scratch/tutorial/first --output /scratch/tutorial/first.tar.gzFor Slurm, first start its services as described in installation, then use --profile slurm --account research --partition demo. Scientific parameters and the SIF stay the same. Inspect squeue -u researcher while it runs and provenance/sacct.tsv afterwards.
The eventual published release will support python reprohpc reproduce --release-lock release.lock.json --profile local --outdir results/reproduction. This requires a real published lock; the repository deliberately contains no invented DOI or placeholder lock.
- Architecture and DAG, algorithm, real microscopy data, FSL neuroimaging, data dictionary, JSON schemas.
- Installation, CLI/configuration, operations and recovery, testing.
- Benchmark methodology, release procedure, acceptance evidence, contributing.
- FAIR/NIH DMS alignment, citation, software license, third-party notices.