Skip to content

CI TPC-DS Tests: Auron queries significantly slower than spark #1926

@yew1eb

Description

@yew1eb

Auron queries are significantly slower than Spark in CI TPC-DS tests (slice: spark-3.5 JDK21 Scala-2.13, q70-q79).

Image

Root Cause

CI Rust compilation opt-level was changed from 1 to 0.

Image

Verification

Reverting to opt-level=1 makes Auron faster than Spark:

Image

Compilation vs Performance

  1. opt-level=0: Negligible compile time reduction, TPC-DS total runtime hits 56 mins
Image
  1. opt-level=1: Slight compile time increase, query performance improves drastically, total runtime hits 39 mins
Image

We should revert CI's Rust opt-level from 0 to 1 (pre profile default). Though compilation time increases slightly, the massive TPC-DS query speedup will cut overall CI runtime (query time dominates total duration).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions