Skip to content

build(ci): Enable hosted GPU Spark compile checks - #103

Open
thirtiseven wants to merge 8 commits into
devfrom
codex/hosted-gpu-spark-ci-hpda
Open

build(ci): Enable hosted GPU Spark compile checks#103
thirtiseven wants to merge 8 commits into
devfrom
codex/hosted-gpu-spark-ci-hpda

Conversation

@thirtiseven

@thirtiseven thirtiseven commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a path-selected GPU Spark compile lane on a standard GitHub-hosted Ubuntu runner.
  • Configure the cuDF/UCX graph and compile Velox-owned production objects without requiring a GPU or linking all of libcudf.
  • Enable one targeted macOS debug compile lane for shared Velox changes while excluding cuDF/UCX-only changes.
  • Keep full adapters builds and GPU runtime tests on their existing larger/GPU-backed runners.

Why

The inherited Linux and GPU workflows require larger or GPU runner labels that are not available in this repository. This adds useful premerge coverage using standard GitHub-hosted capacity without claiming runtime GPU coverage. It complements the Breeze/docs-only scope in #46.

The first canary runs also exposed existing non-mono CMake, generated-header, NVTX include, and transport API drift on dev; this PR repairs those issues so the new lanes start from a passing baseline.

Validation

  • Repository pre-commit hooks pass, including clang-format, workflow validation, zizmor, yamllint, and yamlfmt.
  • git diff --check passes, and no stale TransportType or transportType() references remain.
  • The Ninja target selector identifies 28 Velox-owned cuDF/UCX object targets in the adapters build graph.
  • GPU Spark hosted compile cold run passed in 20m38s; the warm-cache run passed in 5m21s.
  • macOS velox_core_test cold build passed in 1h14m8s after a full build canary demonstrated that the inherited all-target scope exceeds the 120-minute limit.

Coverage boundary

The GPU Spark lane checks C++, CUDA, generated-header, and public API compile compatibility. It does not link the resulting libraries or execute GPU tests. The macOS lane compiles and links the core API target and its production dependencies, but does not run tests.

@github-actions github-actions Bot added the cudf label Aug 25, 2026
@thirtiseven
thirtiseven marked this pull request as ready for review August 25, 2026 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant