Repository navigation
milo: speed up the miloDE section to run in well under 15 minutes - #65
Merged
Merged
Conversation
The miloDE example (added in #61) ran for ~90 minutes because de_nhoods fits a full DESeq2 model per neighbourhood, serially, over all genes and all ~900 neighbourhoods of the kang_2018 dataset. Three changes cut this to a few minutes while keeping the demo intact: - Subsample the kang_2018 object and restrict to highly variable genes, so each per-neighbourhood DESeq2 fit is much cheaper. - Test a random subset of neighbourhoods (`subset_nhoods`, exposed as a tunable `n_test`); the IFN-beta ISG15 signal is still clear (median logFC ~3 across the tested neighbourhoods). - Add the 10k-cell subsample the DA dataset markdown already promised but that was missing from the code. The notebook should be re-executed to refresh its stored outputs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Zethson
added a commit
to scverse/pertpy
that referenced
this pull request
Jun 10, 2026
Points the tutorials submodule at scverse/pertpy-tutorials#65, which speeds up the miloDE section of the Milo notebook to run in well under 15 minutes (it had regressed to ~90 minutes). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Zethson
added a commit
to scverse/pertpy
that referenced
this pull request
Jun 11, 2026
scverse/pertpy-tutorials#65 is merged; point the submodule at the squashed commit on tutorials main. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Zethson
added a commit
to scverse/pertpy
that referenced
this pull request
Jun 11, 2026
* fix: resolve test-suite warnings Clean up the warnings emitted while running the test suite, without changing any behavior. Fixed at the source: - Augur logistic regression no longer passes the `penalty` argument that scikit-learn 1.8 deprecated; the L1/L2 mix is now expressed via `l1_ratio` (l2 -> 0.0, identical coefficients to the old default) and the unregularized case via `C=np.inf`, with `saga` for the L1 paths. - The statsmodels DGE test sets the negative-binomial `alpha=1.0` explicitly (the previous implicit default), removing the dispersion warning raised at collection time. - The perturbation-space test wraps the intentional `ensure_consistency=False` calls in `pytest.warns`, asserting the expected UserWarning instead of letting it leak into the summary. Filtered as unavoidable third-party noise (consistent with the existing filterwarnings entries): - pydeseq2 dispersion-trend / residual-degrees-of-freedom warnings on the small synthetic data used by the Milo and compare_groups tests. - jaxopt maintenance notice, joblib loky worker notice, and the formulaic DataMismatchWarning from no-intercept contrast handling. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * chore: bump tutorials submodule to the milo runtime fix Points the tutorials submodule at scverse/pertpy-tutorials#65, which speeds up the miloDE section of the Milo notebook to run in well under 15 minutes (it had regressed to ~90 minutes). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: bump tutorials submodule to merged miloDE speedup scverse/pertpy-tutorials#65 is merged; point the submodule at the squashed commit on tutorials main. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The miloDE section (added in #61) made
milo.ipynbtake ~90 minutes to run.de_nhoodsfits a full DESeq2 model per neighbourhood, serially, over all genes × all ~900 neighbourhoods ofkang_2018— i.e. ~270 DESeq2 fits. This brings the whole notebook to a few minutes while keeping the demo intact.Changes
kang_2018to 10k cells and restrict to 2,000 highly variable genes, so each per-neighbourhood DESeq2 fit is far cheaper.de_nhoodscell: test a random subset of neighbourhoods viasubset_nhoods, exposed as a tunablen_test(default 60 → ~25 neighbourhoods tested). The IFN-β ISG15 signal stays clear (median logFC ≈ 3.1 across tested neighbourhoods), soplot_de_nhood_graph(gene="ISG15")still demonstrates the method. Bumpn_testfor a denser graph.sc.pp.subsample(..., n_obs=10000)the markdown already promised but that was missing from the code.Verified
The DE section (setup +
de_nhoods+ ISG15 plot) runs in ~65 s on a fast machine; the per-fit DESeq2 cost was the original bottleneck, so on a constrained machine the section now completes in a few minutes rather than ~90.Important
The stored outputs are now stale (datasets/params changed). Please re-execute the notebook to refresh the rendered outputs before merge — this couldn't be done in the authoring sandbox because the DA dataset download is network-blocked there.
🤖 Generated with Claude Code