Add sklearn.cluster.HDBSCAN acceleration to cuml.accel - #8472
Conversation
csadorf
left a comment
There was a problem hiding this comment.
Nice work! Please address the min_samples semantic mismatch and my other comments. Otherwise, this looks good to me.
There was a problem hiding this comment.
That's a very small number of new xfails -- nice!
| X, | ||
| dtype="float32", | ||
| mem_type=mem_type, | ||
| order="C", |
There was a problem hiding this comment.
Is this fixing a pre-existing issue with Fortran-contiguous input? check_inputs otherwise preserves the input order, while HDBSCAN expects C-major input. If so, could we add a focused regression test?
| - Scikit-learn and cuML both count each point itself in ``min_samples``, so | ||
| this parameter is forwarded without an offset. The separate | ||
| ``hdbscan.HDBSCAN`` proxy retains its own parameter handling. |
There was a problem hiding this comment.
From my understanding, forwarding this value unchanged is incorrect. runner.h increments min_samples before native execution to match the contrib convention:
Line 79 in 7424d70
On this branch, sklearn with k=5 versus cuml.accel with k=5 gives an ARI of 0.0, while using k=4 on the accelerated estimator raises it to 0.864. Independently, sklearn k matches contrib k-1 exactly.
Could we translate the semantics in the sklearn adapter, handle the min_samples=1 boundary explicitly, and add a boundary-sensitive regression test? The current parameter-translation test appears to codify the unchanged forwarding.
Closes #7522.
Adds a dedicated
cuml.accelproxy forsklearn.cluster.HDBSCAN, backed by cuML’s GPU implementation while keeping the existinghdbscan.HDBSCANintegration unchanged.The new adapter translates parameters and fitted state, supports
dbscan_clusteringafter GPU fitting, preserves the originating scikit-learn interface during serialization, and falls back to scikit-learn for unsupported parameters or inputs. It also enforces C-contiguous HDBSCAN input to correctly handle pandas DataFrames.Includes integration, CPU/GPU interoperability, serialization, fallback, and upstream compatibility coverage, with documentation of expected numerical differences.