Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run cuvs-bench pytests and end-to-end tests in CI #574

Merged
merged 30 commits into from
Feb 7, 2025
Merged

Run cuvs-bench pytests and end-to-end tests in CI #574

merged 30 commits into from
Feb 7, 2025
+869 −11,428

Conversation

dantegd
Copy link
Member

@dantegd dantegd commented Jan 15, 2025 •
edited
Loading

PR does the following:

  • Modifies CI to run pytest and e2e test of cuvs-bench
    • We need to test the additional time needed to run the tests. They should be fast, but if they are not, then we can add an additional job to run them in parallel.
  • Adds synthetic test-data generation so the CI jobs don't depend on downloading datasets, and users can have easy testing locally.
    • Few improvements to be done to docs, yaml and other things to make it easy for users.
  • Check in some additional pytests that hadn't been checked in before.

@copy-pr-bot copy-pr-bot
Copy link

copy-pr-bot bot commented Jan 15, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@cjnolet cjnolet added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Jan 16, 2025
@dantegd
Copy link
Member Author

dantegd commented Feb 3, 2025

/ok to test

@dantegd
Copy link
Member Author

dantegd commented Feb 3, 2025

/ok to test

dantegd
dantegd commented Feb 4, 2025
View reviewed changes
ci/test_python.sh Outdated Show resolved Hide resolved
@dantegd
Copy link
Member Author

dantegd commented Feb 4, 2025

/ok to test

@dantegd dantegd marked this pull request as ready for review February 4, 2025 12:39
@dantegd dantegd requested review from a team as code owners February 4, 2025 12:39
@dantegd dantegd requested a review from msarahan February 4, 2025 12:39
cjnolet
cjnolet reviewed Feb 4, 2025
View reviewed changes
python/cuvs_bench/cuvs_bench/config/algos/cuvs_cagra.yaml
@@ -11,3 +11,11 @@ groups:
search:
itopk: [32, 64, 128, 256, 512]
search_width: [1, 2, 4, 8, 16, 32, 64]
test:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this idea!

cjnolet
cjnolet reviewed Feb 4, 2025
View reviewed changes
conda/environments/bench_ann_cuda-118_arch-aarch64.yaml
@@ -37,7 +37,7 @@ dependencies:
- libcusparse=11.7.5.86
- libcuvs==25.2.*,>=0.0.0a0
- librmm==25.2.*,>=0.0.0a0
- matplotlib
- matplotlib-base
- nccl>=2.19
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Scikit-learn also need to be added here?

cjnolet
cjnolet reviewed Feb 5, 2025
View reviewed changes
python/cuvs_bench/cuvs_bench/get_dataset/__main__.py
download(args.dataset, args.normalize, args.dataset_path)
f.attrs["distance"] = metric

convert_hdf5_to_fbin(full_path, normalize=True)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder- should we add debug prints before download and convert_hdf5_to_fbin as well, just in case they fail?

jameslamb
jameslamb approved these changes Feb 5, 2025
View reviewed changes
Copy link
Member

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some minor, but they're minor so giving you a ci-codeowners / packaging-codeowners approval so you have what you need to merge. Please do consider them.

python/cuvs_bench/cuvs_bench/get_dataset/__main__.py Outdated Show resolved Hide resolved
dependencies.yaml Show resolved Hide resolved
@jameslamb jameslamb removed the request for review from msarahan February 5, 2025 23:55
@cjnolet
Copy link
Member

cjnolet commented Feb 7, 2025

/merge

@rapids-bot rapids-bot bot merged commit d760d85 into rapidsai:branch-25.02 Feb 7, 2025
64 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cjnolet cjnolet cjnolet approved these changes

@jameslamb jameslamb jameslamb approved these changes

Assignees

@dantegd dantegd

Labels
ci improvement Improves an existing functionality non-breaking Introduces a non-breaking change Python
Projects
VS/ML/DM Primitives Release Board
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dantegd @cjnolet @jameslamb @lowener