Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Find a formula to convert HNSWLIB build params to CAGRA build params (for HNSW search) #740

Open
cjnolet opened this issue Mar 1, 2025 · 0 comments
Labels
feature request New feature or request

Comments

@cjnolet
Copy link
Member

cjnolet commented Mar 1, 2025

We need to do more investigation of this compatibility in params. Thus far, I just don't think we have analyzed the Paretos for enough datasets to be able to say one way or another how whether this conversion even makes sense.

The goal here is to say "I have params M and efConstruction for HNSW, what graph_degree and intermediate_graph_degree can I use to build a CAGRA graph that when converted to HNSW will yield a similar recall/throughput curve".

One thing we can (and should) try is taking a bunch of data for Pareto curves and testing the predictive capabilities by building a simple regression model to predict the CAGRA parameters and expected throughput on the Pareto when given HNSWlib parameters. We have done little exercises like this in past for finding things like optimal code paths for kernel launches given various shapes of data.

@cjnolet cjnolet added the feature request New feature or request label Mar 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant