Skip to content

[feat][spark-4.0.2] Spark Connect compatibility (plugin works under Connect mode) on GPU (cuDF) #90

Description

@winningsix

Background

Velox-native (cuDF) counterpart of the NVIDIA/spark-gluten issue for this Spark 4.0.2 feature. Depends on the native harness #66 and the JVM harness NVIDIA/spark-gluten#134.
Spark tickets:

Scope (native cuDF side)

  • Implement / extend the cuDF GPU operators and expressions under velox/experimental/cudf/ backing this feature.
  • Add / extend gtests under velox/experimental/cudf/tests/.
  • Ensure CPU Velox tests are unaffected (VELOX_ENABLE_CUDF-gated).

Acceptance criteria — all pass on GPU (VELOX_ENABLE_CUDF=ON, ctest)

(none)

Cross-references

Notes

Ensure plan offload works when spark.api.mode = connect.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featNew feature enablementspark-4.0.2Spark 4.0.2 feature enablement on GPU (cuDF)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions