Skip to content

perfect hash join #19411

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
UBarney wants to merge 1 commit into
base: main
Choose a base branch
from
+1,936 −485
Open

perfect hash join #19411

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions benchmarks/bench.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,8 +322,7 @@ main() {
echo "NLJ benchmark does not require data generation"
;;
hj)
# hj uses range() function, no data generation needed
echo "HJ benchmark does not require data generation"
data_tpch "10" "parquet"
;;
smj)
# smj uses range() function, no data generation needed
Expand Down Expand Up @@ -1228,10 +1227,11 @@ run_nlj() {

# Runs the hj benchmark
run_hj() {
TPCH_DIR="${DATA_DIR}/tpch_sf10"
RESULTS_FILE="${RESULTS_DIR}/hj.json"
echo "RESULTS_FILE: ${RESULTS_FILE}"
echo "Running hj benchmark..."
debug_run $CARGO_COMMAND --bin dfbench -- hj --iterations 5 -o "${RESULTS_FILE}" ${QUERY_ARG}
debug_run $CARGO_COMMAND --bin dfbench -- hj --iterations 5 --path "${TPCH_DIR}" -o "${RESULTS_FILE}" ${QUERY_ARG}
}

# Runs the smj benchmark
Expand Down
20 changes: 10 additions & 10 deletions benchmarks/compare.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,17 +154,17 @@ def compare(
baseline = BenchmarkRun.load_from_file(baseline_path)
comparison = BenchmarkRun.load_from_file(comparison_path)

console = Console()
console = Console(width=200)
Copy link
Contributor Author

@UBarney UBarney Dec 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've increased the console width to 200. I added more information like 'density' to the queryName, which made it longer and caused it to be cut off in the output before


# use basename as the column names
baseline_header = baseline_path.parent.stem
comparison_header = comparison_path.parent.stem
baseline_header = baseline_path.parent.name
comparison_header = comparison_path.parent.name
Copy link
Contributor Author

@UBarney UBarney Dec 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before, a path like .../density=0.1/... was incorrectly shortened to density=0. Now, by using .parent.name, we correctly get the full directory name, density=0.1


table = Table(show_header=True, header_style="bold magenta")
table.add_column("Query", style="dim", width=12)
table.add_column(baseline_header, justify="right", style="dim")
table.add_column(comparison_header, justify="right", style="dim")
table.add_column("Change", justify="right", style="dim")
table.add_column("Query", style="dim", no_wrap=True)
table.add_column(baseline_header, justify="right", style="dim", no_wrap=True)
table.add_column(comparison_header, justify="right", style="dim", no_wrap=True)
table.add_column("Change", justify="right", style="dim", no_wrap=True)

faster_count = 0
slower_count = 0
Expand All @@ -175,12 +175,12 @@ def compare(

for baseline_result, comparison_result in zip(baseline.queries, comparison.queries):
assert baseline_result.query == comparison_result.query

base_failed = not baseline_result.success
comp_failed = not comparison_result.success
comp_failed = not comparison_result.success
# If a query fails, its execution time is excluded from the performance comparison
if base_failed or comp_failed:
change_text = "incomparable"
change_text = "incomparable"
failure_count += 1
table.add_row(
f"Q{baseline_result.query}",
Expand Down
Loading