Skip to content

feat(cudf): Support Spark string-to-decimal try_cast - #34

Draft
thirtiseven wants to merge 1 commit into
devfrom
agent/cudf-string-to-decimal
Draft

feat(cudf): Support Spark string-to-decimal try_cast#34
thirtiseven wants to merge 1 commit into
devfrom
agent/cudf-string-to-decimal

Conversation

@thirtiseven

Copy link
Copy Markdown
Collaborator

Summary

  • Add a Spark-compatible GPU implementation of try_cast(VARCHAR AS DECIMAL).
  • Preserve HALF_UP rounding, scientific notation, precision and scale checks, whitespace handling, and null-on-invalid semantics for DECIMAL64 and DECIMAL128.
  • Route decimal logical types away from the existing string-to-integral path.

Motivation

Velox short decimals use an integral physical type, so the cuDF expression adapter previously classified a string-to-decimal cast as a string-to-integer cast. Long decimals were rejected outright. libcudf's fixed-point parser is not a semantic substitute because it truncates fractional values where Spark rounds HALF_UP.

The conversion kernel is adapted from the Apache-licensed NVIDIA/spark-rapids-jni implementation, with additional validation for whitespace-only strings and incomplete mantissa or exponent forms.

Scope

This PR supports non-ANSI try_cast only. It does not change generic cast behavior, introduce a runtime dependency on RAPIDS JNI, or affect non-decimal cast paths.

Validation

  • velox_cudf_spark_string_to_decimal_test: 5/5 passed.
  • GPU results are compared directly with the Spark SQL CPU evaluator across short and long decimals, non-zero scales, HALF_UP boundaries, scientific notation, whitespace, invalid input, precision overflow, nulls, and a composed validation predicate.

@github-actions github-actions Bot added the cudf label Jul 31, 2026
@github-actions

Copy link
Copy Markdown

Selective Build Plan

Linux release with adapters is running a full build (changes touch velox/experimental/ or velox/external/). See the CI workflows README for what this means.


Selective build plan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant