Skip to content

validate all benchmark runs#6939

Closed
joseph-isaacs wants to merge 26 commits into
developfrom
ji/validate-sql-bench
Closed

validate all benchmark runs#6939
joseph-isaacs wants to merge 26 commits into
developfrom
ji/validate-sql-bench

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Use slt (sql logic test) to validate benchmark runs.

@joseph-isaacs
validate all
6400ff9 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added action/benchmark Trigger full benchmarks to run on this PR changelog/chore A trivial change labels Mar 13, 2026
@github-actions github-actions Bot removed the action/benchmark Trigger full benchmarks to run on this PR label Mar 13, 2026
@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 6076d52 1 Explore Profiling Data
Previous Runs (5)
Status Commit Job Attempt Link
🟢 Done 6a58b21 1 Explore Profiling Data
🟢 Done 6542c62 1 Explore Profiling Data
🟢 Done ef1dadf 1 Explore Profiling Data
🟢 Done 70c2e75 1 Explore Profiling Data
🟢 Done 6400ff9 1 Explore Profiling Data

Powered by Polar Signals Cloud

@codspeed-hq

codspeed-hq Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown

Merging this PR will improve performance by 20%

⚡ 2 improved benchmarks
✅ 1161 untouched benchmarks
⏩ 1457 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Simulation take_map[(0.1, 0.5)] 1,154.5 µs 980.3 µs +17.77%
⚡ Simulation take_map[(0.1, 1.0)] 2 ms 1.6 ms +20%

Comparing ji/validate-sql-bench (959e6f0) with develop (3cd74b9)

Open in CodSpeed

Footnotes

  1. 1457 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on S3

Verdict: Likely regression (environment too noisy confidence)
Attributed Vortex impact: +94.2%
Vortex (geomean): 1.058x ➖
Parquet (geomean): 0.545x ✅
Shifts: Parquet (control) -45.5% · Median polish +0.9%

datafusion / vortex-file-compressed (0.896x ➖, 9↑ 5↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 🚨 354562245 204229468 1.74
tpch_q02/datafusion:vortex-file-compressed 🚨 797568664 467522880 1.71
tpch_q03/datafusion:vortex-file-compressed 🚨 988505161 341216091 2.90
tpch_q04/datafusion:vortex-file-compressed 🚨 402752797 197367303 2.04
tpch_q05/datafusion:vortex-file-compressed 475255201 379808806 1.25
tpch_q06/datafusion:vortex-file-compressed 🚨 387203100 267739657 1.45
tpch_q07/datafusion:vortex-file-compressed 472814177 405665184 1.17
tpch_q08/datafusion:vortex-file-compressed 🚀 709407088 1184185696 0.60
tpch_q09/datafusion:vortex-file-compressed 🚀 435090043 731505035 0.59
tpch_q10/datafusion:vortex-file-compressed 🚀 550866177 797294305 0.69
tpch_q11/datafusion:vortex-file-compressed 🚀 378461555 817061454 0.46
tpch_q12/datafusion:vortex-file-compressed 🚀 441093184 712826743 0.62
tpch_q13/datafusion:vortex-file-compressed 🚀 242408361 470112826 0.52
tpch_q14/datafusion:vortex-file-compressed 200932550 280022535 0.72
tpch_q15/datafusion:vortex-file-compressed 399925463 492989288 0.81
tpch_q16/datafusion:vortex-file-compressed 🚀 182575920 280573225 0.65
tpch_q17/datafusion:vortex-file-compressed 🚀 288860154 477897986 0.60
tpch_q18/datafusion:vortex-file-compressed 248215228 325181008 0.76
tpch_q19/datafusion:vortex-file-compressed 321086395 429625797 0.75
tpch_q20/datafusion:vortex-file-compressed 386395215 466571930 0.83
tpch_q21/datafusion:vortex-file-compressed 🚀 415448317 625208103 0.66
tpch_q22/datafusion:vortex-file-compressed 136323230 149186342 0.91
datafusion / vortex-compact (1.232x ➖, 1↑ 10↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 246959501 262998855 0.94
tpch_q02/datafusion:vortex-compact 434990684 491582832 0.88
tpch_q03/datafusion:vortex-compact 386821986 381943321 1.01
tpch_q04/datafusion:vortex-compact 265314816 263706085 1.01
tpch_q05/datafusion:vortex-compact 🚨 766414832 365788190 2.10
tpch_q06/datafusion:vortex-compact 🚨 527537236 323637292 1.63
tpch_q07/datafusion:vortex-compact 🚨 828473692 384128094 2.16
tpch_q08/datafusion:vortex-compact 🚨 787286297 535507772 1.47
tpch_q09/datafusion:vortex-compact 🚨 661109995 353805077 1.87
tpch_q10/datafusion:vortex-compact 763694649 664250661 1.15
tpch_q11/datafusion:vortex-compact 326111801 438991060 0.74
tpch_q12/datafusion:vortex-compact 🚨 754785096 473016216 1.60
tpch_q13/datafusion:vortex-compact 🚨 257409759 182126672 1.41
tpch_q14/datafusion:vortex-compact 319322723 248947824 1.28
tpch_q15/datafusion:vortex-compact 🚨 1094893645 405434362 2.70
tpch_q16/datafusion:vortex-compact 🚨 342934413 191504956 1.79
tpch_q17/datafusion:vortex-compact 503589084 512873145 0.98
tpch_q18/datafusion:vortex-compact 436159910 400395960 1.09
tpch_q19/datafusion:vortex-compact 724850493 614309984 1.18
tpch_q20/datafusion:vortex-compact 🚨 798868342 472934296 1.69
tpch_q21/datafusion:vortex-compact 650239588 837318120 0.78
tpch_q22/datafusion:vortex-compact 🚀 133131759 461182217 0.29
datafusion / parquet (0.926x ➖, 1↑ 1↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 222648832 241552359 0.92
tpch_q02/datafusion:parquet 402719385 559538358 0.72
tpch_q03/datafusion:parquet 305128818 316829626 0.96
tpch_q04/datafusion:parquet 161724892 202142006 0.80
tpch_q05/datafusion:parquet 612084601 551893151 1.11
tpch_q06/datafusion:parquet 181465688 160494926 1.13
tpch_q07/datafusion:parquet 542459621 567717568 0.96
tpch_q08/datafusion:parquet 746622497 638057155 1.17
tpch_q09/datafusion:parquet 621606078 569218260 1.09
tpch_q10/datafusion:parquet 593237784 707941444 0.84
tpch_q11/datafusion:parquet 🚀 340693944 766748610 0.44
tpch_q12/datafusion:parquet 322068360 387804756 0.83
tpch_q13/datafusion:parquet 436821122 510611712 0.86
tpch_q14/datafusion:parquet 178470579 239555181 0.75
tpch_q15/datafusion:parquet 310444691 384678886 0.81
tpch_q16/datafusion:parquet 211222100 185161830 1.14
tpch_q17/datafusion:parquet 417758434 558607728 0.75
tpch_q18/datafusion:parquet 467460577 620513894 0.75
tpch_q19/datafusion:parquet 432083459 407990378 1.06
tpch_q20/datafusion:parquet 385798542 323788168 1.19
tpch_q21/datafusion:parquet 675614075 553047904 1.22
tpch_q22/datafusion:parquet 🚨 180706552 122150966 1.48
duckdb / vortex-file-compressed (1.056x ➖, 0↑ 1↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 254203008 251845988 1.01
tpch_q02/duckdb:vortex-file-compressed 893472689 941878755 0.95
tpch_q03/duckdb:vortex-file-compressed 634649502 608685448 1.04
tpch_q04/duckdb:vortex-file-compressed 399640312 384183870 1.04
tpch_q05/duckdb:vortex-file-compressed 800837205 842345600 0.95
tpch_q06/duckdb:vortex-file-compressed 365273166 515784473 0.71
tpch_q07/duckdb:vortex-file-compressed 830126738 727017516 1.14
tpch_q08/duckdb:vortex-file-compressed 934169149 926934953 1.01
tpch_q09/duckdb:vortex-file-compressed 941577548 808596063 1.16
tpch_q10/duckdb:vortex-file-compressed 821674141 752075881 1.09
tpch_q11/duckdb:vortex-file-compressed 595353087 498846917 1.19
tpch_q12/duckdb:vortex-file-compressed 🚨 584979604 433573340 1.35
tpch_q13/duckdb:vortex-file-compressed 469325240 463435264 1.01
tpch_q14/duckdb:vortex-file-compressed 480777952 439812258 1.09
tpch_q15/duckdb:vortex-file-compressed 306650945 282993586 1.08
tpch_q16/duckdb:vortex-file-compressed 359300929 369622768 0.97
tpch_q17/duckdb:vortex-file-compressed 721284190 667990665 1.08
tpch_q18/duckdb:vortex-file-compressed 571571535 594866759 0.96
tpch_q19/duckdb:vortex-file-compressed 524347538 431520045 1.22
tpch_q20/duckdb:vortex-file-compressed 955815787 770049973 1.24
tpch_q21/duckdb:vortex-file-compressed 1211723064 1180075497 1.03
tpch_q22/duckdb:vortex-file-compressed 352024155 326825125 1.08
duckdb / vortex-compact (1.075x ➖, 0↑ 2↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 🚨 398164893 251315330 1.58
tpch_q02/duckdb:vortex-compact 935941889 945473928 0.99
tpch_q03/duckdb:vortex-compact 615120014 595058068 1.03
tpch_q04/duckdb:vortex-compact 368766209 395753253 0.93
tpch_q05/duckdb:vortex-compact 1021156069 912274602 1.12
tpch_q06/duckdb:vortex-compact 472613543 458027340 1.03
tpch_q07/duckdb:vortex-compact 751549905 709969152 1.06
tpch_q08/duckdb:vortex-compact 882116561 944993584 0.93
tpch_q09/duckdb:vortex-compact 1141975261 939214656 1.22
tpch_q10/duckdb:vortex-compact 688254034 757729077 0.91
tpch_q11/duckdb:vortex-compact 466002144 483047024 0.96
tpch_q12/duckdb:vortex-compact 446861800 455316703 0.98
tpch_q13/duckdb:vortex-compact 450130049 429411329 1.05
tpch_q14/duckdb:vortex-compact 460651732 514236290 0.90
tpch_q15/duckdb:vortex-compact 🚨 398807745 275438145 1.45
tpch_q16/duckdb:vortex-compact 361743534 379187321 0.95
tpch_q17/duckdb:vortex-compact 655427187 583856994 1.12
tpch_q18/duckdb:vortex-compact 487782634 449506179 1.09
tpch_q19/duckdb:vortex-compact 479157957 407427645 1.18
tpch_q20/duckdb:vortex-compact 908043664 759480872 1.20
tpch_q21/duckdb:vortex-compact 1237323516 1012931519 1.22
tpch_q22/duckdb:vortex-compact 354139240 349784769 1.01
duckdb / parquet (0.321x ✅, 22↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 🚀 192976336 459193575 0.42
tpch_q02/duckdb:parquet 🚀 523102504 1243937057 0.42
tpch_q03/duckdb:parquet 🚀 215506992 939471652 0.23
tpch_q04/duckdb:parquet 🚀 155130004 600014494 0.26
tpch_q05/duckdb:parquet 🚀 394584000 1341969448 0.29
tpch_q06/duckdb:parquet 🚀 70851777 369111116 0.19
tpch_q07/duckdb:parquet 🚀 419513695 1151168902 0.36
tpch_q08/duckdb:parquet 🚀 655569299 1753683765 0.37
tpch_q09/duckdb:parquet 🚀 495602025 1672964962 0.30
tpch_q10/duckdb:parquet 🚀 368345416 1229390394 0.30
tpch_q11/duckdb:parquet 🚀 443057431 876367849 0.51
tpch_q12/duckdb:parquet 🚀 154873898 706183501 0.22
tpch_q13/duckdb:parquet 🚀 372817305 853452281 0.44
tpch_q14/duckdb:parquet 🚀 165262820 751614053 0.22
tpch_q15/duckdb:parquet 🚀 129085767 558290888 0.23
tpch_q16/duckdb:parquet 🚀 233257228 530543644 0.44
tpch_q17/duckdb:parquet 🚀 227892860 690491213 0.33
tpch_q18/duckdb:parquet 🚀 372919241 854640173 0.44
tpch_q19/duckdb:parquet 🚀 190215262 785574328 0.24
tpch_q20/duckdb:parquet 🚀 339060638 1067272406 0.32
tpch_q21/duckdb:parquet 🚀 441877721 1081260132 0.41
tpch_q22/duckdb:parquet 🚀 212925924 550289014 0.39
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
1 datafusion:vortex-compact -6.1% -37.8% +50.9% +168.6% ➖ noise
1 datafusion:vortex-file-compressed +73.6% -37.8% +178.9% +167.4% 🚨 regression
1 duckdb:vortex-compact +58.4% -37.8% +154.6% +69.1% 🚨 regression
1 duckdb:vortex-file-compressed +0.9% -37.8% +62.2% +70.2% ➖ noise
2 datafusion:vortex-compact -11.5% -45.0% +60.8% +59.0% 🚨 regression
2 datafusion:vortex-file-compressed +70.6% -45.0% +210.1% +65.2% 🚨 regression
2 duckdb:vortex-compact -1.0% -45.0% +79.9% +58.0% 🚨 regression
2 duckdb:vortex-file-compressed -5.1% -45.0% +72.4% +58.0% 🚨 regression
3 datafusion:vortex-compact +1.3% -53.0% +115.5% +111.6% 🚨 regression
3 datafusion:vortex-file-compressed +189.7% -53.0% +516.4% +135.5% 🚨 regression
3 duckdb:vortex-compact +3.4% -53.0% +119.9% +72.1% 🚨 regression
3 duckdb:vortex-file-compressed +4.3% -53.0% +121.8% +61.7% 🚨 regression
4 datafusion:vortex-compact +0.6% -54.5% +121.2% +65.2% 🚨 regression
4 datafusion:vortex-file-compressed +104.1% -54.5% +348.7% +78.3% 🚨 regression
4 duckdb:vortex-compact -6.8% -54.5% +104.9% +58.0% 🚨 regression
4 duckdb:vortex-file-compressed +4.0% -54.5% +128.7% +58.0% 🚨 regression
5 datafusion:vortex-compact +109.5% -42.9% +266.9% +58.0% 🚨 regression
5 datafusion:vortex-file-compressed +25.1% -42.9% +119.1% +58.0% 🚨 regression
5 duckdb:vortex-compact +11.9% -42.9% +96.0% +58.0% 🚨 regression
5 duckdb:vortex-file-compressed -4.9% -42.9% +66.5% +58.0% 🚨 regression
6 datafusion:vortex-compact +63.0% -53.4% +249.9% +58.0% 🚨 regression
6 datafusion:vortex-file-compressed +44.6% -53.4% +210.4% +58.0% 🚨 regression
6 duckdb:vortex-compact +3.2% -53.4% +121.5% +58.0% 🚨 regression
6 duckdb:vortex-file-compressed -29.2% -53.4% +52.0% +58.0% ➖ noise
7 datafusion:vortex-compact +115.7% -41.0% +265.5% +58.0% 🚨 regression
7 datafusion:vortex-file-compressed +16.6% -41.0% +97.5% +58.0% 🚨 regression
7 duckdb:vortex-compact +5.9% -41.0% +79.4% +58.0% 🚨 regression
7 duckdb:vortex-file-compressed +14.2% -41.0% +93.5% +58.0% 🚨 regression
8 datafusion:vortex-compact +47.0% -33.9% +122.3% +58.0% 🚨 regression
8 datafusion:vortex-file-compressed -40.1% -33.9% -9.4% +63.5% ➖ noise
8 duckdb:vortex-compact -6.7% -33.9% +41.1% +58.0% ➖ noise
8 duckdb:vortex-file-compressed +0.8% -33.9% +52.4% +58.0% ➖ noise
9 datafusion:vortex-compact +86.9% -43.1% +228.5% +58.0% 🚨 regression
9 datafusion:vortex-file-compressed -40.5% -43.1% +4.6% +76.2% ➖ noise
9 duckdb:vortex-compact +21.6% -43.1% +113.8% +58.0% 🚨 regression
9 duckdb:vortex-file-compressed +16.4% -43.1% +104.7% +58.0% 🚨 regression
10 datafusion:vortex-compact +15.0% -49.9% +129.5% +83.2% 🚨 regression
10 datafusion:vortex-file-compressed -30.9% -49.9% +37.9% +82.3% ➖ noise
10 duckdb:vortex-compact -9.2% -49.9% +81.3% +64.4% 🚨 regression
10 duckdb:vortex-file-compressed +9.3% -49.9% +118.0% +58.0% 🚨 regression
11 datafusion:vortex-compact -25.7% -52.6% +56.7% +68.3% ➖ noise
11 datafusion:vortex-file-compressed -53.7% -52.6% -2.3% +58.0% ➖ noise
11 duckdb:vortex-compact -3.5% -52.6% +103.5% +58.0% 🚨 regression
11 duckdb:vortex-file-compressed +19.3% -52.6% +151.8% +58.0% 🚨 regression
12 datafusion:vortex-compact +59.6% -57.3% +273.9% +58.0% 🚨 regression
12 datafusion:vortex-file-compressed -38.1% -57.3% +45.0% +58.0% ➖ noise
12 duckdb:vortex-compact -1.9% -57.3% +130.0% +58.0% 🚨 regression
12 duckdb:vortex-file-compressed +34.9% -57.3% +216.1% +58.0% 🚨 regression
13 datafusion:vortex-compact +41.3% -38.9% +131.2% +110.0% 🚨 regression
13 datafusion:vortex-file-compressed -48.4% -38.9% -15.7% +103.6% ➖ noise
13 duckdb:vortex-compact +4.8% -38.9% +71.5% +80.9% ➖ noise
13 duckdb:vortex-file-compressed +1.3% -38.9% +65.7% +79.7% ➖ noise
14 datafusion:vortex-compact +28.3% -59.5% +216.9% +58.0% 🚨 regression
14 datafusion:vortex-file-compressed -28.2% -59.5% +77.3% +58.0% 🚨 regression
14 duckdb:vortex-compact -10.4% -59.5% +121.3% +58.0% 🚨 regression
14 duckdb:vortex-file-compressed +9.3% -59.5% +170.1% +58.0% 🚨 regression
15 datafusion:vortex-compact +170.1% -56.8% +525.2% +58.0% 🚨 regression
15 datafusion:vortex-file-compressed -18.9% -56.8% +87.8% +58.0% 🚨 regression
15 duckdb:vortex-compact +44.8% -56.8% +235.2% +74.9% 🚨 regression
15 duckdb:vortex-file-compressed +8.4% -56.8% +150.9% +58.0% 🚨 regression
16 datafusion:vortex-compact +79.1% -29.2% +152.9% +58.0% 🚨 regression
16 datafusion:vortex-file-compressed -34.9% -29.2% -8.1% +58.0% ➖ noise
16 duckdb:vortex-compact -4.6% -29.2% +34.7% +58.0% ➖ noise
16 duckdb:vortex-file-compressed -2.8% -29.2% +37.3% +58.0% ➖ noise
17 datafusion:vortex-compact -1.8% -50.3% +97.6% +58.0% 🚨 regression
17 datafusion:vortex-file-compressed -39.6% -50.3% +21.7% +58.0% ➖ noise
17 duckdb:vortex-compact +12.3% -50.3% +126.0% +58.0% 🚨 regression
17 duckdb:vortex-file-compressed +8.0% -50.3% +117.3% +58.0% 🚨 regression
18 datafusion:vortex-compact +8.9% -42.7% +90.0% +58.0% 🚨 regression
18 datafusion:vortex-file-compressed -23.7% -42.7% +33.1% +58.0% ➖ noise
18 duckdb:vortex-compact +8.5% -42.7% +89.3% +58.0% 🚨 regression
18 duckdb:vortex-file-compressed -3.9% -42.7% +67.6% +58.0% 🚨 regression
19 datafusion:vortex-compact +18.0% -49.4% +133.0% +61.5% 🚨 regression
19 datafusion:vortex-file-compressed -25.3% -49.4% +47.6% +59.6% ➖ noise
19 duckdb:vortex-compact +17.6% -49.4% +132.2% +58.0% 🚨 regression
19 duckdb:vortex-file-compressed +21.5% -49.4% +140.0% +58.0% 🚨 regression
20 datafusion:vortex-compact +68.9% -38.5% +174.6% +58.0% 🚨 regression
20 datafusion:vortex-file-compressed -17.2% -38.5% +34.6% +58.0% ➖ noise
20 duckdb:vortex-compact +19.6% -38.5% +94.3% +58.0% 🚨 regression
20 duckdb:vortex-file-compressed +24.1% -38.5% +101.7% +58.0% 🚨 regression
21 datafusion:vortex-compact -22.3% -29.3% +9.9% +58.0% ➖ noise
21 datafusion:vortex-file-compressed -33.6% -29.3% -6.0% +58.0% ➖ noise
21 duckdb:vortex-compact +22.2% -29.3% +72.9% +58.0% 🚨 regression
21 duckdb:vortex-file-compressed +2.7% -29.3% +45.3% +58.0% ➖ noise
22 datafusion:vortex-compact -71.1% -24.3% -61.8% +58.0% ✅ faster
22 datafusion:vortex-file-compressed -8.6% -24.3% +20.8% +58.0% ➖ noise
22 duckdb:vortex-compact +1.2% -24.3% +33.8% +58.0% ➖ noise
22 duckdb:vortex-file-compressed +7.7% -24.3% +42.4% +62.4% ➖ noise

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on NVME

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -1.8%
Vortex (geomean): 1.039x ➖
Parquet (geomean): 1.034x ➖
Shifts: Parquet (control) +3.4% · Median polish +2.8%

datafusion / vortex-file-compressed (1.062x ➖, 0↑ 2↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 50380376 49909449 1.01
tpch_q02/datafusion:vortex-file-compressed 34538110 32732124 1.06
tpch_q03/datafusion:vortex-file-compressed 29498466 27907361 1.06
tpch_q04/datafusion:vortex-file-compressed 19026963 18014780 1.06
tpch_q05/datafusion:vortex-file-compressed 56618218 53265204 1.06
tpch_q06/datafusion:vortex-file-compressed 10352285 9988720 1.04
tpch_q07/datafusion:vortex-file-compressed 67275171 63182345 1.06
tpch_q08/datafusion:vortex-file-compressed 49548689 46236064 1.07
tpch_q09/datafusion:vortex-file-compressed 58742073 56852426 1.03
tpch_q10/datafusion:vortex-file-compressed 49105535 44950696 1.09
tpch_q11/datafusion:vortex-file-compressed 19404404 18421534 1.05
tpch_q12/datafusion:vortex-file-compressed 23768698 22187091 1.07
tpch_q13/datafusion:vortex-file-compressed 30398327 27950544 1.09
tpch_q14/datafusion:vortex-file-compressed 17010220 16270861 1.05
tpch_q15/datafusion:vortex-file-compressed 25825053 24130257 1.07
tpch_q16/datafusion:vortex-file-compressed 🚨 26867948 24329881 1.10
tpch_q17/datafusion:vortex-file-compressed 73948903 67325674 1.10
tpch_q18/datafusion:vortex-file-compressed 85997183 79754777 1.08
tpch_q19/datafusion:vortex-file-compressed 🚨 28028390 24402605 1.15
tpch_q20/datafusion:vortex-file-compressed 34811643 32177589 1.08
tpch_q21/datafusion:vortex-file-compressed 78541105 75044931 1.05
tpch_q22/datafusion:vortex-file-compressed 16765106 17649516 0.95
datafusion / vortex-compact (1.067x ➖, 0↑ 2↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 60638657 59318906 1.02
tpch_q02/datafusion:vortex-compact 38558103 37228970 1.04
tpch_q03/datafusion:vortex-compact 32159859 30966723 1.04
tpch_q04/datafusion:vortex-compact 24018884 23147178 1.04
tpch_q05/datafusion:vortex-compact 59201391 55280237 1.07
tpch_q06/datafusion:vortex-compact 🚨 14597742 12985429 1.12
tpch_q07/datafusion:vortex-compact 72432848 69498754 1.04
tpch_q08/datafusion:vortex-compact 53311355 51561802 1.03
tpch_q09/datafusion:vortex-compact 63437522 62720422 1.01
tpch_q10/datafusion:vortex-compact 56109891 51964513 1.08
tpch_q11/datafusion:vortex-compact 20447177 19237423 1.06
tpch_q12/datafusion:vortex-compact 36043709 33681615 1.07
tpch_q13/datafusion:vortex-compact 38069680 36311443 1.05
tpch_q14/datafusion:vortex-compact 22039619 20464149 1.08
tpch_q15/datafusion:vortex-compact 35714952 34463477 1.04
tpch_q16/datafusion:vortex-compact 31915340 29785140 1.07
tpch_q17/datafusion:vortex-compact 72916056 69191524 1.05
tpch_q18/datafusion:vortex-compact 91581028 85090984 1.08
tpch_q19/datafusion:vortex-compact 🚨 43546520 33022018 1.32
tpch_q20/datafusion:vortex-compact 40123523 38754186 1.04
tpch_q21/datafusion:vortex-compact 94079119 86277725 1.09
tpch_q22/datafusion:vortex-compact 18427856 17363971 1.06
datafusion / parquet (1.066x ➖, 0↑ 4↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 133881283 135402466 0.99
tpch_q02/datafusion:parquet 73893041 69803562 1.06
tpch_q03/datafusion:parquet 🚨 83270318 69712154 1.19
tpch_q04/datafusion:parquet 50058184 48383819 1.03
tpch_q05/datafusion:parquet 105673600 102580082 1.03
tpch_q06/datafusion:parquet 45765746 44139576 1.04
tpch_q07/datafusion:parquet 119471458 112627360 1.06
tpch_q08/datafusion:parquet 🚨 110880187 99029228 1.12
tpch_q09/datafusion:parquet 140672611 129114475 1.09
tpch_q10/datafusion:parquet 123311658 117772811 1.05
tpch_q11/datafusion:parquet 50679153 49498518 1.02
tpch_q12/datafusion:parquet 🚨 94161495 82262998 1.14
tpch_q13/datafusion:parquet 220147081 207876879 1.06
tpch_q14/datafusion:parquet 50223176 51063254 0.98
tpch_q15/datafusion:parquet 67678880 64815234 1.04
tpch_q16/datafusion:parquet 53055133 50726399 1.05
tpch_q17/datafusion:parquet 152446604 143922274 1.06
tpch_q18/datafusion:parquet 175890970 169652653 1.04
tpch_q19/datafusion:parquet 🚨 93094605 70151016 1.33
tpch_q20/datafusion:parquet 80451153 77737294 1.03
tpch_q21/datafusion:parquet 157782111 146942224 1.07
tpch_q22/datafusion:parquet 35530324 35253055 1.01
datafusion / arrow (1.054x ➖, 0↑ 5↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/datafusion:arrow 57067657 57975853 0.98
tpch_q02/datafusion:arrow 22651131 21949904 1.03
tpch_q03/datafusion:arrow 36497614 33591944 1.09
tpch_q04/datafusion:arrow 29029646 28661598 1.01
tpch_q05/datafusion:arrow 81621215 82105634 0.99
tpch_q06/datafusion:arrow 21388929 21694423 0.99
tpch_q07/datafusion:arrow 119222817 112762506 1.06
tpch_q08/datafusion:arrow 50078545 47925216 1.04
tpch_q09/datafusion:arrow 🚨 80980289 71220488 1.14
tpch_q10/datafusion:arrow 🚨 62023142 51782719 1.20
tpch_q11/datafusion:arrow 12374445 11422664 1.08
tpch_q12/datafusion:arrow 58192338 55904732 1.04
tpch_q13/datafusion:arrow 52473390 49186669 1.07
tpch_q14/datafusion:arrow 23951143 23687289 1.01
tpch_q15/datafusion:arrow 48333352 46086103 1.05
tpch_q16/datafusion:arrow 22538035 20610068 1.09
tpch_q17/datafusion:arrow 🚨 78109127 70714577 1.10
tpch_q18/datafusion:arrow 156712680 149549907 1.05
tpch_q19/datafusion:arrow 41361766 43568119 0.95
tpch_q20/datafusion:arrow 🚨 43041436 38638201 1.11
tpch_q21/datafusion:arrow 🚨 170401310 152975664 1.11
tpch_q22/datafusion:arrow 20699253 20302634 1.02
duckdb / vortex-file-compressed (1.020x ➖, 0↑ 1↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 33028912 32876174 1.00
tpch_q02/duckdb:vortex-file-compressed 28187135 28284231 1.00
tpch_q03/duckdb:vortex-file-compressed 30633470 29486092 1.04
tpch_q04/duckdb:vortex-file-compressed 31364076 30295961 1.04
tpch_q05/duckdb:vortex-file-compressed 32380264 33254552 0.97
tpch_q06/duckdb:vortex-file-compressed 11211371 11089359 1.01
tpch_q07/duckdb:vortex-file-compressed 38299736 37896404 1.01
tpch_q08/duckdb:vortex-file-compressed 40062670 41296971 0.97
tpch_q09/duckdb:vortex-file-compressed 83606255 77556317 1.08
tpch_q10/duckdb:vortex-file-compressed 36623796 36044340 1.02
tpch_q11/duckdb:vortex-file-compressed 16687095 16470369 1.01
tpch_q12/duckdb:vortex-file-compressed 23255390 22602302 1.03
tpch_q13/duckdb:vortex-file-compressed 37287887 37946190 0.98
tpch_q14/duckdb:vortex-file-compressed 22172575 21626515 1.03
tpch_q15/duckdb:vortex-file-compressed 🚨 17747173 16004261 1.11
tpch_q16/duckdb:vortex-file-compressed 28202606 30258874 0.93
tpch_q17/duckdb:vortex-file-compressed 27619593 25227473 1.09
tpch_q18/duckdb:vortex-file-compressed 46858243 47002801 1.00
tpch_q19/duckdb:vortex-file-compressed 32014663 31776329 1.01
tpch_q20/duckdb:vortex-file-compressed 36338737 35158758 1.03
tpch_q21/duckdb:vortex-file-compressed 103716444 103475401 1.00
tpch_q22/duckdb:vortex-file-compressed 20415706 18570727 1.10
duckdb / vortex-compact (1.007x ➖, 0↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 40310929 39221439 1.03
tpch_q02/duckdb:vortex-compact 37359478 38363226 0.97
tpch_q03/duckdb:vortex-compact 32626438 31766080 1.03
tpch_q04/duckdb:vortex-compact 34847105 34057085 1.02
tpch_q05/duckdb:vortex-compact 38410983 37133386 1.03
tpch_q06/duckdb:vortex-compact 16068427 14957480 1.07
tpch_q07/duckdb:vortex-compact 44010016 42603793 1.03
tpch_q08/duckdb:vortex-compact 48076897 46246453 1.04
tpch_q09/duckdb:vortex-compact 87295168 89567617 0.97
tpch_q10/duckdb:vortex-compact 40933357 40102252 1.02
tpch_q11/duckdb:vortex-compact 19485098 21080968 0.92
tpch_q12/duckdb:vortex-compact 37335097 38462962 0.97
tpch_q13/duckdb:vortex-compact 44620836 46009115 0.97
tpch_q14/duckdb:vortex-compact 28482242 30154974 0.94
tpch_q15/duckdb:vortex-compact 20351754 19683517 1.03
tpch_q16/duckdb:vortex-compact 35187054 36918376 0.95
tpch_q17/duckdb:vortex-compact 31829746 31482273 1.01
tpch_q18/duckdb:vortex-compact 48061317 50911658 0.94
tpch_q19/duckdb:vortex-compact 38272301 37708092 1.01
tpch_q20/duckdb:vortex-compact 45633001 44149731 1.03
tpch_q21/duckdb:vortex-compact 116519388 108437480 1.07
tpch_q22/duckdb:vortex-compact 21854977 20200342 1.08
duckdb / parquet (1.003x ➖, 3↑ 5↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 🚀 73782967 114936174 0.64
tpch_q02/duckdb:parquet 38028271 39300604 0.97
tpch_q03/duckdb:parquet 🚨 77362784 69322201 1.12
tpch_q04/duckdb:parquet 53135530 56336949 0.94
tpch_q05/duckdb:parquet 76644435 70716332 1.08
tpch_q06/duckdb:parquet 🚨 25999401 22320112 1.16
tpch_q07/duckdb:parquet 64849131 69137379 0.94
tpch_q08/duckdb:parquet 83110269 80955863 1.03
tpch_q09/duckdb:parquet 156050342 159175712 0.98
tpch_q10/duckdb:parquet 125061559 127026651 0.98
tpch_q11/duckdb:parquet 🚀 20617913 22936548 0.90
tpch_q12/duckdb:parquet 🚨 55575828 45304274 1.23
tpch_q13/duckdb:parquet 276963490 260341900 1.06
tpch_q14/duckdb:parquet 🚀 46465806 59355676 0.78
tpch_q15/duckdb:parquet 32821277 29974984 1.09
tpch_q16/duckdb:parquet 52387290 54216129 0.97
tpch_q17/duckdb:parquet 54230762 53883661 1.01
tpch_q18/duckdb:parquet 122389055 125354843 0.98
tpch_q19/duckdb:parquet 70625386 68746225 1.03
tpch_q20/duckdb:parquet 🚨 81415646 62592674 1.30
tpch_q21/duckdb:parquet 🚨 183472841 163953560 1.12
tpch_q22/duckdb:parquet 50222915 51259945 0.98
duckdb / duckdb (0.896x ✅, 12↑ 1↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpch_q01/duckdb:duckdb 🚀 13776533 16481007 0.84
tpch_q02/duckdb:duckdb 11330291 12488087 0.91
tpch_q03/duckdb:duckdb 🚀 16872355 18862459 0.89
tpch_q04/duckdb:duckdb 18889207 19266211 0.98
tpch_q05/duckdb:duckdb 🚀 17088817 20204895 0.85
tpch_q06/duckdb:duckdb 🚀 3691610 5327036 0.69
tpch_q07/duckdb:duckdb 🚀 19156717 23188463 0.83
tpch_q08/duckdb:duckdb 🚀 17741512 20460768 0.87
tpch_q09/duckdb:duckdb 53247341 51694215 1.03
tpch_q10/duckdb:duckdb 39015152 40957113 0.95
tpch_q11/duckdb:duckdb 🚀 4739301 5686093 0.83
tpch_q12/duckdb:duckdb 🚀 11953135 13602591 0.88
tpch_q13/duckdb:duckdb 🚨 44889139 37283245 1.20
tpch_q14/duckdb:duckdb 🚀 13595341 16933171 0.80
tpch_q15/duckdb:duckdb 🚀 9867145 11632884 0.85
tpch_q16/duckdb:duckdb 🚀 21911640 24859068 0.88
tpch_q17/duckdb:duckdb 🚀 11873432 13316385 0.89
tpch_q18/duckdb:duckdb 36095363 39528606 0.91
tpch_q19/duckdb:duckdb 23921748 26230567 0.91
tpch_q20/duckdb:duckdb 20436938 22183289 0.92
tpch_q21/duckdb:duckdb 51609339 55446237 0.93
tpch_q22/duckdb:duckdb 22912947 23975534 0.96
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
1 datafusion:arrow -1.6% -20.3% +23.6% +26.8% ➖ noise
1 datafusion:vortex-compact +2.2% -20.3% +28.3% +26.8% 🚨 regression
1 datafusion:vortex-file-compressed +0.9% -20.3% +26.7% +28.5% ➖ noise
1 duckdb:duckdb -16.4% -20.3% +4.9% +27.1% ➖ noise
1 duckdb:vortex-compact +2.8% -20.3% +29.0% +26.7% 🚨 regression
1 duckdb:vortex-file-compressed +0.5% -20.3% +26.1% +26.7% ➖ noise
2 datafusion:arrow +3.2% +1.2% +2.0% +26.7% ➖ noise
2 datafusion:vortex-compact +3.6% +1.2% +2.3% +26.7% ➖ noise
2 datafusion:vortex-file-compressed +5.5% +1.2% +4.3% +26.7% ➖ noise
2 duckdb:duckdb -9.3% +1.2% -10.4% +26.7% ➖ noise
2 duckdb:vortex-compact -2.6% +1.2% -3.8% +26.7% ➖ noise
2 duckdb:vortex-file-compressed -0.3% +1.2% -1.5% +26.7% ➖ noise
3 datafusion:arrow +8.6% +15.5% -5.9% +26.7% ➖ noise
3 datafusion:vortex-compact +3.9% +15.5% -10.1% +26.7% ➖ noise
3 datafusion:vortex-file-compressed +5.7% +15.5% -8.4% +26.7% ➖ noise
3 duckdb:duckdb -10.6% +15.5% -22.5% +26.7% ✅ faster
3 duckdb:vortex-compact +2.7% +15.5% -11.0% +26.7% ➖ noise
3 duckdb:vortex-file-compressed +3.9% +15.5% -10.0% +26.7% ➖ noise
4 datafusion:arrow +1.3% -1.2% +2.5% +26.7% ➖ noise
4 datafusion:vortex-compact +3.8% -1.2% +5.0% +26.7% ➖ noise
4 datafusion:vortex-file-compressed +5.6% -1.2% +6.9% +26.7% ➖ noise
4 duckdb:duckdb -2.0% -1.2% -0.7% +26.7% ➖ noise
4 duckdb:vortex-compact +2.3% -1.2% +3.6% +26.7% ➖ noise
4 duckdb:vortex-file-compressed +3.5% -1.2% +4.8% +26.7% ➖ noise
5 datafusion:arrow -0.6% +5.7% -5.9% +26.7% ➖ noise
5 datafusion:vortex-compact +7.1% +5.7% +1.4% +26.7% ➖ noise
5 datafusion:vortex-file-compressed +6.3% +5.7% +0.6% +26.7% ➖ noise
5 duckdb:duckdb -15.4% +5.7% -20.0% +26.7% ➖ noise
5 duckdb:vortex-compact +3.4% +5.7% -2.1% +26.7% ➖ noise
5 duckdb:vortex-file-compressed -2.6% +5.7% -7.8% +26.7% ➖ noise
6 datafusion:arrow -1.4% +9.9% -10.3% +26.7% ➖ noise
6 datafusion:vortex-compact +12.4% +9.9% +2.3% +26.7% ➖ noise
6 datafusion:vortex-file-compressed +3.6% +9.9% -5.7% +26.7% ➖ noise
6 duckdb:duckdb -30.7% +9.9% -36.9% +26.7% ✅ faster
6 duckdb:vortex-compact +7.4% +9.9% -2.2% +26.7% ➖ noise
6 duckdb:vortex-file-compressed +1.1% +9.9% -8.0% +26.7% ➖ noise
7 datafusion:arrow +5.7% -0.3% +6.0% +26.7% ➖ noise
7 datafusion:vortex-compact +4.2% -0.3% +4.5% +26.7% ➖ noise
7 datafusion:vortex-file-compressed +6.5% -0.3% +6.7% +26.7% ➖ noise
7 duckdb:duckdb -17.4% -0.3% -17.2% +26.7% ➖ noise
7 duckdb:vortex-compact +3.3% -0.3% +3.6% +26.7% ➖ noise
7 duckdb:vortex-file-compressed +1.1% -0.3% +1.3% +26.7% ➖ noise
8 datafusion:arrow +4.5% +7.2% -2.5% +26.7% ➖ noise
8 datafusion:vortex-compact +3.4% +7.2% -3.6% +26.7% ➖ noise
8 datafusion:vortex-file-compressed +7.2% +7.2% -0.0% +26.7% ➖ noise
8 duckdb:duckdb -13.3% +7.2% -19.1% +26.7% ➖ noise
8 duckdb:vortex-compact +4.0% +7.2% -3.0% +26.7% ➖ noise
8 duckdb:vortex-file-compressed -3.0% +7.2% -9.5% +26.7% ➖ noise
9 datafusion:arrow +13.7% +3.4% +10.0% +26.7% ➖ noise
9 datafusion:vortex-compact +1.1% +3.4% -2.1% +26.7% ➖ noise
9 datafusion:vortex-file-compressed +3.3% +3.4% -0.0% +26.7% ➖ noise
9 duckdb:duckdb +3.0% +3.4% -0.3% +26.7% ➖ noise
9 duckdb:vortex-compact -2.5% +3.4% -5.7% +26.7% ➖ noise
9 duckdb:vortex-file-compressed +7.8% +3.4% +4.3% +26.7% ➖ noise
10 datafusion:arrow +19.8% +1.5% +18.0% +26.7% ➖ noise
10 datafusion:vortex-compact +8.0% +1.5% +6.4% +26.7% ➖ noise
10 datafusion:vortex-file-compressed +9.2% +1.5% +7.6% +26.7% ➖ noise
10 duckdb:duckdb -4.7% +1.5% -6.2% +26.7% ➖ noise
10 duckdb:vortex-compact +2.1% +1.5% +0.5% +26.7% ➖ noise
10 duckdb:vortex-file-compressed +1.6% +1.5% +0.1% +26.7% ➖ noise
11 datafusion:arrow +8.3% -4.1% +12.9% +26.7% ➖ noise
11 datafusion:vortex-compact +6.3% -4.1% +10.8% +26.7% ➖ noise
11 datafusion:vortex-file-compressed +5.3% -4.1% +9.8% +26.7% ➖ noise
11 duckdb:duckdb -16.7% -4.1% -13.1% +26.7% ➖ noise
11 duckdb:vortex-compact -7.6% -4.1% -3.7% +26.7% ➖ noise
11 duckdb:vortex-file-compressed +1.3% -4.1% +5.6% +26.7% ➖ noise
12 datafusion:arrow +4.1% +18.5% -12.2% +26.7% ➖ noise
12 datafusion:vortex-compact +7.0% +18.5% -9.7% +26.7% ➖ noise
12 datafusion:vortex-file-compressed +7.1% +18.5% -9.6% +26.7% ➖ noise
12 duckdb:duckdb -12.1% +18.5% -25.8% +26.7% ✅ faster
12 duckdb:vortex-compact -2.9% +18.5% -18.1% +26.7% ➖ noise
12 duckdb:vortex-file-compressed +2.9% +18.5% -13.2% +26.7% ➖ noise
13 datafusion:arrow +6.7% +6.1% +0.5% +26.7% ➖ noise
13 datafusion:vortex-compact +4.8% +6.1% -1.2% +26.7% ➖ noise
13 datafusion:vortex-file-compressed +8.8% +6.1% +2.5% +26.7% ➖ noise
13 duckdb:duckdb +20.4% +6.1% +13.4% +26.7% ➖ noise
13 duckdb:vortex-compact -3.0% +6.1% -8.6% +26.7% ➖ noise
13 duckdb:vortex-file-compressed -1.7% +6.1% -7.4% +26.7% ➖ noise
14 datafusion:arrow +1.1% -12.3% +15.2% +26.7% ➖ noise
14 datafusion:vortex-compact +7.7% -12.3% +22.7% +26.7% ➖ noise
14 datafusion:vortex-file-compressed +4.5% -12.3% +19.1% +26.7% ➖ noise
14 duckdb:duckdb -19.7% -12.3% -8.5% +26.7% ➖ noise
14 duckdb:vortex-compact -5.5% -12.3% +7.6% +26.7% ➖ noise
14 duckdb:vortex-file-compressed +2.5% -12.3% +16.8% +26.7% ➖ noise
15 datafusion:arrow +4.9% +6.9% -1.9% +26.7% ➖ noise
15 datafusion:vortex-compact +3.6% +6.9% -3.1% +26.7% ➖ noise
15 datafusion:vortex-file-compressed +7.0% +6.9% +0.1% +26.7% ➖ noise
15 duckdb:duckdb -15.2% +6.9% -20.7% +26.7% ➖ noise
15 duckdb:vortex-compact +3.4% +6.9% -3.3% +26.7% ➖ noise
15 duckdb:vortex-file-compressed +10.9% +6.9% +3.7% +26.7% ➖ noise
16 datafusion:arrow +9.4% +0.5% +8.8% +26.7% ➖ noise
16 datafusion:vortex-compact +7.2% +0.5% +6.6% +26.7% ➖ noise
16 datafusion:vortex-file-compressed +10.4% +0.5% +9.8% +26.7% ➖ noise
16 duckdb:duckdb -11.9% +0.5% -12.3% +26.7% ➖ noise
16 duckdb:vortex-compact -4.7% +0.5% -5.2% +26.7% ➖ noise
16 duckdb:vortex-file-compressed -6.8% +0.5% -7.3% +26.7% ➖ noise
17 datafusion:arrow +10.5% +3.2% +7.0% +26.7% ➖ noise
17 datafusion:vortex-compact +5.4% +3.2% +2.1% +26.7% ➖ noise
17 datafusion:vortex-file-compressed +9.8% +3.2% +6.4% +26.7% ➖ noise
17 duckdb:duckdb -10.8% +3.2% -13.6% +26.7% ➖ noise
17 duckdb:vortex-compact +1.1% +3.2% -2.1% +26.7% ➖ noise
17 duckdb:vortex-file-compressed +9.5% +3.2% +6.0% +26.7% ➖ noise
18 datafusion:arrow +4.8% +0.6% +4.2% +26.7% ➖ noise
18 datafusion:vortex-compact +7.6% +0.6% +7.0% +26.7% ➖ noise
18 datafusion:vortex-file-compressed +7.8% +0.6% +7.2% +26.7% ➖ noise
18 duckdb:duckdb -8.7% +0.6% -9.2% +26.7% ➖ noise
18 duckdb:vortex-compact -5.6% +0.6% -6.2% +26.7% ➖ noise
18 duckdb:vortex-file-compressed -0.3% +0.6% -0.9% +26.7% ➖ noise
19 datafusion:arrow -5.1% +16.8% -18.7% +26.7% ➖ noise
19 datafusion:vortex-compact +31.9% +16.8% +12.9% +26.7% ➖ noise
19 datafusion:vortex-file-compressed +14.9% +16.8% -1.6% +26.7% ➖ noise
19 duckdb:duckdb -8.8% +16.8% -21.9% +26.7% ✅ faster
19 duckdb:vortex-compact +1.5% +16.8% -13.1% +26.7% ➖ noise
19 duckdb:vortex-file-compressed +0.8% +16.8% -13.7% +26.7% ➖ noise
20 datafusion:arrow +11.4% +16.0% -4.0% +26.7% ➖ noise
20 datafusion:vortex-compact +3.5% +16.0% -10.8% +26.7% ➖ noise
20 datafusion:vortex-file-compressed +8.2% +16.0% -6.8% +26.7% ➖ noise
20 duckdb:duckdb -7.9% +16.0% -20.6% +26.7% ➖ noise
20 duckdb:vortex-compact +3.4% +16.0% -10.9% +26.7% ➖ noise
20 duckdb:vortex-file-compressed +3.4% +16.0% -10.9% +26.7% ➖ noise
21 datafusion:arrow +11.4% +9.6% +1.6% +26.7% ➖ noise
21 datafusion:vortex-compact +9.0% +9.6% -0.5% +26.7% ➖ noise
21 datafusion:vortex-file-compressed +4.7% +9.6% -4.5% +26.7% ➖ noise
21 duckdb:duckdb -6.9% +9.6% -15.1% +26.7% ➖ noise
21 duckdb:vortex-compact +7.5% +9.6% -2.0% +26.7% ➖ noise
21 duckdb:vortex-file-compressed +0.2% +9.6% -8.6% +26.7% ➖ noise
22 datafusion:arrow +2.0% -0.6% +2.6% +26.7% ➖ noise
22 datafusion:vortex-compact +6.1% -0.6% +6.8% +26.7% ➖ noise
22 datafusion:vortex-file-compressed -5.0% -0.6% -4.4% +26.7% ➖ noise
22 duckdb:duckdb -4.4% -0.6% -3.8% +26.7% ➖ noise
22 duckdb:vortex-compact +8.2% -0.6% +8.9% +26.7% ➖ noise
22 duckdb:vortex-file-compressed +9.9% -0.6% +10.6% +26.7% ➖ noise

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: FineWeb NVMe

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +3.1%
Vortex (geomean): 0.998x ➖
Parquet (geomean): 0.968x ➖
Shifts: Parquet (control) -3.2% · Median polish -1.1%

datafusion / vortex-file-compressed (1.016x ➖, 0↑ 1↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-file-compressed 5476142 5292249 1.03
fineweb_q01/datafusion:vortex-file-compressed 30535693 28521298 1.07
fineweb_q02/datafusion:vortex-file-compressed 🚨 34585123 29890895 1.16
fineweb_q03/datafusion:vortex-file-compressed 97985993 90461208 1.08
fineweb_q04/datafusion:vortex-file-compressed 259944126 264629213 0.98
fineweb_q05/datafusion:vortex-file-compressed 227233638 248335365 0.92
fineweb_q06/datafusion:vortex-file-compressed 70881368 73415976 0.97
fineweb_q07/datafusion:vortex-file-compressed 70704366 75082344 0.94
fineweb_q08/datafusion:vortex-file-compressed 28830199 28407686 1.01
datafusion / vortex-compact (0.990x ➖, 1↑ 1↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-compact 🚀 5235408 6171280 0.85
fineweb_q01/datafusion:vortex-compact 172958148 169691227 1.02
fineweb_q02/datafusion:vortex-compact 169343662 169046553 1.00
fineweb_q03/datafusion:vortex-compact 1456024333 1480878677 0.98
fineweb_q04/datafusion:vortex-compact 1584792051 1607438199 0.99
fineweb_q05/datafusion:vortex-compact 1262722654 1268455500 1.00
fineweb_q06/datafusion:vortex-compact 707318606 709381315 1.00
fineweb_q07/datafusion:vortex-compact 742440918 750034450 0.99
fineweb_q08/datafusion:vortex-compact 🚨 24633311 22187611 1.11
datafusion / parquet (0.969x ➖, 1↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/datafusion:parquet 🚀 6268564 6966520 0.90
fineweb_q01/datafusion:parquet 269410117 285424348 0.94
fineweb_q02/datafusion:parquet 273382100 282514399 0.97
fineweb_q03/datafusion:parquet 270327127 279519472 0.97
fineweb_q04/datafusion:parquet 291822789 291879205 1.00
fineweb_q05/datafusion:parquet 285136092 289334396 0.99
fineweb_q06/datafusion:parquet 275997561 284929772 0.97
fineweb_q07/datafusion:parquet 274754382 276252229 0.99
fineweb_q08/datafusion:parquet 272080209 273500045 0.99
duckdb / vortex-file-compressed (1.023x ➖, 1↑ 2↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-file-compressed 3554017 3626356 0.98
fineweb_q01/duckdb:vortex-file-compressed 🚨 34326648 28431434 1.21
fineweb_q02/duckdb:vortex-file-compressed 🚨 35816053 29884347 1.20
fineweb_q03/duckdb:vortex-file-compressed 153829462 163750441 0.94
fineweb_q04/duckdb:vortex-file-compressed 234587757 234027926 1.00
fineweb_q05/duckdb:vortex-file-compressed 213965060 214024369 1.00
fineweb_q06/duckdb:vortex-file-compressed 72274085 69168204 1.04
fineweb_q07/duckdb:vortex-file-compressed 69664951 70867482 0.98
fineweb_q08/duckdb:vortex-file-compressed 🚀 24274728 27179127 0.89
duckdb / vortex-compact (0.966x ➖, 1↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-compact 4210172 4388577 0.96
fineweb_q01/duckdb:vortex-compact 🚀 98665284 115170466 0.86
fineweb_q02/duckdb:vortex-compact 122156486 120358307 1.01
fineweb_q03/duckdb:vortex-compact 870980594 900581122 0.97
fineweb_q04/duckdb:vortex-compact 911706905 930689803 0.98
fineweb_q05/duckdb:vortex-compact 815827162 834784713 0.98
fineweb_q06/duckdb:vortex-compact 461670858 478192204 0.97
fineweb_q07/duckdb:vortex-compact 476522175 491382613 0.97
fineweb_q08/duckdb:vortex-compact 22890453 22637650 1.01
duckdb / parquet (0.968x ➖, 0↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/duckdb:parquet 35924457 38393811 0.94
fineweb_q01/duckdb:parquet 92773059 96689884 0.96
fineweb_q02/duckdb:parquet 97684228 100101431 0.98
fineweb_q03/duckdb:parquet 309719594 316374432 0.98
fineweb_q04/duckdb:parquet 437861883 444325177 0.99
fineweb_q05/duckdb:parquet 410261934 416317563 0.99
fineweb_q06/duckdb:parquet 199164397 202961914 0.98
fineweb_q07/duckdb:parquet 210719199 215269892 0.98
fineweb_q08/duckdb:parquet 57586884 61904104 0.93
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
0 datafusion:vortex-compact -15.2% -8.2% -7.5% +125.2% ➖ noise
0 datafusion:vortex-file-compressed +3.5% -8.2% +12.8% +60.4% ➖ noise
0 duckdb:vortex-compact -4.1% -8.2% +4.6% +38.0% ➖ noise
0 duckdb:vortex-file-compressed -2.0% -8.2% +6.8% +48.0% ➖ noise
1 datafusion:vortex-compact +1.9% -4.8% +7.1% +10.0% ➖ noise
1 datafusion:vortex-file-compressed +7.1% -4.8% +12.5% +20.3% ➖ noise
1 duckdb:vortex-compact -14.3% -4.8% -10.0% +26.5% ➖ noise
1 duckdb:vortex-file-compressed +20.7% -4.8% +26.9% +33.2% ➖ noise
2 datafusion:vortex-compact +0.2% -2.8% +3.1% +10.0% ➖ noise
2 datafusion:vortex-file-compressed +15.7% -2.8% +19.1% +15.7% 🚨 regression
2 duckdb:vortex-compact +1.5% -2.8% +4.4% +12.7% ➖ noise
2 duckdb:vortex-file-compressed +19.8% -2.8% +23.3% +11.2% 🚨 regression
3 datafusion:vortex-compact -1.7% -2.7% +1.0% +10.0% ➖ noise
3 datafusion:vortex-file-compressed +8.3% -2.7% +11.3% +30.5% ➖ noise
3 duckdb:vortex-compact -3.3% -2.7% -0.6% +10.0% ➖ noise
3 duckdb:vortex-file-compressed -6.1% -2.7% -3.5% +35.9% ➖ noise
4 datafusion:vortex-compact -1.4% -0.7% -0.7% +10.0% ➖ noise
4 datafusion:vortex-file-compressed -1.8% -0.7% -1.0% +11.1% ➖ noise
4 duckdb:vortex-compact -2.0% -0.7% -1.3% +10.0% ➖ noise
4 duckdb:vortex-file-compressed +0.2% -0.7% +1.0% +10.0% ➖ noise
5 datafusion:vortex-compact -0.5% -1.5% +1.0% +10.0% ➖ noise
5 datafusion:vortex-file-compressed -8.5% -1.5% -7.1% +10.0% ➖ noise
5 duckdb:vortex-compact -2.3% -1.5% -0.8% +10.0% ➖ noise
5 duckdb:vortex-file-compressed -0.0% -1.5% +1.4% +10.0% ➖ noise
6 datafusion:vortex-compact -0.3% -2.5% +2.3% +10.0% ➖ noise
6 datafusion:vortex-file-compressed -3.5% -2.5% -1.0% +17.7% ➖ noise
6 duckdb:vortex-compact -3.5% -2.5% -1.0% +10.0% ➖ noise
6 duckdb:vortex-file-compressed +4.5% -2.5% +7.2% +10.0% ➖ noise
7 datafusion:vortex-compact -1.0% -1.3% +0.3% +10.0% ➖ noise
7 datafusion:vortex-file-compressed -5.8% -1.3% -4.6% +10.3% ➖ noise
7 duckdb:vortex-compact -3.0% -1.3% -1.7% +10.0% ➖ noise
7 duckdb:vortex-file-compressed -1.7% -1.3% -0.4% +10.0% ➖ noise
8 datafusion:vortex-compact +11.0% -3.8% +15.4% +24.3% ➖ noise
8 datafusion:vortex-file-compressed +1.5% -3.8% +5.5% +44.5% ➖ noise
8 duckdb:vortex-compact +1.1% -3.8% +5.1% +20.2% ➖ noise
8 duckdb:vortex-file-compressed -10.7% -3.8% -7.2% +20.0% ➖ noise

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-DS SF=1 on NVME

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.3%
Vortex (geomean): 0.963x ➖
Parquet (geomean): 0.962x ➖
Shifts: Parquet (control) -3.8% · Median polish -3.5%

datafusion / vortex-file-compressed (0.935x ➖, 14↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpcds_q01/datafusion:vortex-file-compressed 🚀 27268733 30837802 0.88
tpcds_q02/datafusion:vortex-file-compressed 52000205 57579125 0.90
tpcds_q03/datafusion:vortex-file-compressed 21044734 22770327 0.92
tpcds_q04/datafusion:vortex-file-compressed 289388199 308763448 0.94
tpcds_q05/datafusion:vortex-file-compressed 🚀 50798646 57181863 0.89
tpcds_q06/datafusion:vortex-file-compressed 50340345 52885892 0.95
tpcds_q07/datafusion:vortex-file-compressed 50817603 53020880 0.96
tpcds_q08/datafusion:vortex-file-compressed 36575509 40042216 0.91
tpcds_q09/datafusion:vortex-file-compressed 54380384 57252694 0.95
tpcds_q10/datafusion:vortex-file-compressed 45736911 48863847 0.94
tpcds_q11/datafusion:vortex-file-compressed 164665971 173948454 0.95
tpcds_q12/datafusion:vortex-file-compressed 23182747 23336280 0.99
tpcds_q13/datafusion:vortex-file-compressed 62618797 63881229 0.98
tpcds_q14/datafusion:vortex-file-compressed 201822473 213551094 0.95
tpcds_q15/datafusion:vortex-file-compressed 33674389 37023868 0.91
tpcds_q16/datafusion:vortex-file-compressed 🚀 27759896 31049830 0.89
tpcds_q17/datafusion:vortex-file-compressed 66295808 72328105 0.92
tpcds_q18/datafusion:vortex-file-compressed 78455482 81638880 0.96
tpcds_q19/datafusion:vortex-file-compressed 31495683 33269385 0.95
tpcds_q20/datafusion:vortex-file-compressed 25243061 26931989 0.94
tpcds_q21/datafusion:vortex-file-compressed 137972486 140469245 0.98
tpcds_q22/datafusion:vortex-file-compressed 57190051 62364183 0.92
tpcds_q23/datafusion:vortex-file-compressed 148439857 155599019 0.95
tpcds_q24/datafusion:vortex-file-compressed 111921015 118527058 0.94
tpcds_q25/datafusion:vortex-file-compressed 66740705 71551758 0.93
tpcds_q26/datafusion:vortex-file-compressed 36816523 38671330 0.95
tpcds_q27/datafusion:vortex-file-compressed 119862623 125669004 0.95
tpcds_q28/datafusion:vortex-file-compressed 49486390 51178213 0.97
tpcds_q29/datafusion:vortex-file-compressed 66964082 71594505 0.94
tpcds_q30/datafusion:vortex-file-compressed 30265020 32184778 0.94
tpcds_q31/datafusion:vortex-file-compressed 95182187 101796017 0.94
tpcds_q32/datafusion:vortex-file-compressed 20454206 22150206 0.92
tpcds_q33/datafusion:vortex-file-compressed 38343839 41420378 0.93
tpcds_q34/datafusion:vortex-file-compressed 🚀 31792741 36172021 0.88
tpcds_q35/datafusion:vortex-file-compressed 55330338 58596265 0.94
tpcds_q36/datafusion:vortex-file-compressed 76471657 80298059 0.95
tpcds_q37/datafusion:vortex-file-compressed 34998368 35980411 0.97
tpcds_q38/datafusion:vortex-file-compressed 52415989 56788096 0.92
tpcds_q39/datafusion:vortex-file-compressed 137046036 148000246 0.93
tpcds_q40/datafusion:vortex-file-compressed 52526117 55321467 0.95
tpcds_q41/datafusion:vortex-file-compressed 18211044 19734451 0.92
tpcds_q42/datafusion:vortex-file-compressed 🚀 17407342 20591457 0.85
tpcds_q43/datafusion:vortex-file-compressed 32885850 34676676 0.95
tpcds_q44/datafusion:vortex-file-compressed 🚀 43121502 48785683 0.88
tpcds_q45/datafusion:vortex-file-compressed 34978967 35634011 0.98
tpcds_q46/datafusion:vortex-file-compressed 47787316 53064670 0.90
tpcds_q47/datafusion:vortex-file-compressed 202267901 215827140 0.94
tpcds_q48/datafusion:vortex-file-compressed 48570221 49691496 0.98
tpcds_q49/datafusion:vortex-file-compressed 70168338 73775600 0.95
tpcds_q50/datafusion:vortex-file-compressed 46408883 49240336 0.94
tpcds_q51/datafusion:vortex-file-compressed 90924304 92773776 0.98
tpcds_q52/datafusion:vortex-file-compressed 🚀 18457005 20700090 0.89
tpcds_q53/datafusion:vortex-file-compressed 26157910 28678140 0.91
tpcds_q54/datafusion:vortex-file-compressed 45015999 46995992 0.96
tpcds_q55/datafusion:vortex-file-compressed 18144558 19499768 0.93
tpcds_q56/datafusion:vortex-file-compressed 40540173 42912277 0.94
tpcds_q57/datafusion:vortex-file-compressed 125248251 134157093 0.93
tpcds_q58/datafusion:vortex-file-compressed 69684052 71278464 0.98
tpcds_q59/datafusion:vortex-file-compressed 71650883 75046606 0.95
tpcds_q60/datafusion:vortex-file-compressed 40445333 43588628 0.93
tpcds_q61/datafusion:vortex-file-compressed 50538478 54406366 0.93
tpcds_q62/datafusion:vortex-file-compressed 31125019 34148221 0.91
tpcds_q63/datafusion:vortex-file-compressed 🚀 25277597 28234971 0.90
tpcds_q64/datafusion:vortex-file-compressed 475111480 502496990 0.95
tpcds_q65/datafusion:vortex-file-compressed 48076273 53368304 0.90
tpcds_q66/datafusion:vortex-file-compressed 91711826 94369877 0.97
tpcds_q67/datafusion:vortex-file-compressed 114264441 120432346 0.95
tpcds_q68/datafusion:vortex-file-compressed 47161400 49913728 0.94
tpcds_q69/datafusion:vortex-file-compressed 43302377 46614085 0.93
tpcds_q70/datafusion:vortex-file-compressed 112470472 115472589 0.97
tpcds_q71/datafusion:vortex-file-compressed 31072101 33591583 0.92
tpcds_q72/datafusion:vortex-file-compressed 2612985692 2691585220 0.97
tpcds_q73/datafusion:vortex-file-compressed 34837869 37478993 0.93
tpcds_q74/datafusion:vortex-file-compressed 115726939 122609047 0.94
tpcds_q75/datafusion:vortex-file-compressed 119009612 124687147 0.95
tpcds_q76/datafusion:vortex-file-compressed 34323259 38011618 0.90
tpcds_q77/datafusion:vortex-file-compressed 49660683 51993796 0.96
tpcds_q78/datafusion:vortex-file-compressed 127448113 128462363 0.99
tpcds_q79/datafusion:vortex-file-compressed 39109238 41214070 0.95
tpcds_q80/datafusion:vortex-file-compressed 114230225 118225993 0.97
tpcds_q81/datafusion:vortex-file-compressed 37536298 40405786 0.93
tpcds_q82/datafusion:vortex-file-compressed 36179931 37930878 0.95
tpcds_q83/datafusion:vortex-file-compressed 49599355 54499519 0.91
tpcds_q84/datafusion:vortex-file-compressed 15595743 16235544 0.96
tpcds_q85/datafusion:vortex-file-compressed 101554670 105566143 0.96
tpcds_q86/datafusion:vortex-file-compressed 🚀 18202353 20317447 0.90
tpcds_q87/datafusion:vortex-file-compressed 🚀 53100639 59381555 0.89
tpcds_q88/datafusion:vortex-file-compressed 🚀 77764545 86446557 0.90
tpcds_q89/datafusion:vortex-file-compressed 37229590 38835476 0.96
tpcds_q90/datafusion:vortex-file-compressed 🚀 17511421 19710537 0.89
tpcds_q91/datafusion:vortex-file-compressed 24834706 27294791 0.91
tpcds_q92/datafusion:vortex-file-compressed 🚀 17615010 20481772 0.86
tpcds_q93/datafusion:vortex-file-compressed 34060196 36368817 0.94
tpcds_q94/datafusion:vortex-file-compressed 25399747 26621604 0.95
tpcds_q95/datafusion:vortex-file-compressed 59249513 62774807 0.94
tpcds_q96/datafusion:vortex-file-compressed 🚀 17403380 19645175 0.89
tpcds_q97/datafusion:vortex-file-compressed 30834277 33182356 0.93
tpcds_q98/datafusion:vortex-file-compressed 32460267 33283781 0.98
tpcds_q99/datafusion:vortex-file-compressed 46612604 48445731 0.96
datafusion / vortex-compact (0.968x ➖, 1↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpcds_q01/datafusion:vortex-compact 34858854 38395427 0.91
tpcds_q02/datafusion:vortex-compact 56865243 59130329 0.96
tpcds_q03/datafusion:vortex-compact 25996674 26126390 1.00
tpcds_q04/datafusion:vortex-compact 328473402 336084007 0.98
tpcds_q05/datafusion:vortex-compact 64714231 66808721 0.97
tpcds_q06/datafusion:vortex-compact 56999929 59308208 0.96
tpcds_q07/datafusion:vortex-compact 65478298 68704073 0.95
tpcds_q08/datafusion:vortex-compact 46178651 47461783 0.97
tpcds_q09/datafusion:vortex-compact 100498335 106825416 0.94
tpcds_q10/datafusion:vortex-compact 62930263 65833797 0.96
tpcds_q11/datafusion:vortex-compact 186612900 191529215 0.97
tpcds_q12/datafusion:vortex-compact 26279301 27326235 0.96
tpcds_q13/datafusion:vortex-compact 126527951 131353458 0.96
tpcds_q14/datafusion:vortex-compact 253636865 257511010 0.98
tpcds_q15/datafusion:vortex-compact 37940055 39802925 0.95
tpcds_q16/datafusion:vortex-compact 40718104 39097922 1.04
tpcds_q17/datafusion:vortex-compact 94262484 92311169 1.02
tpcds_q18/datafusion:vortex-compact 94740223 93438170 1.01
tpcds_q19/datafusion:vortex-compact 43647498 43394912 1.01
tpcds_q20/datafusion:vortex-compact 30162137 31274806 0.96
tpcds_q21/datafusion:vortex-compact 138371639 144227570 0.96
tpcds_q22/datafusion:vortex-compact 66446656 63991850 1.04
tpcds_q23/datafusion:vortex-compact 185443287 187773051 0.99
tpcds_q24/datafusion:vortex-compact 142236587 144175776 0.99
tpcds_q25/datafusion:vortex-compact 92144027 96117434 0.96
tpcds_q26/datafusion:vortex-compact 46723353 48208566 0.97
tpcds_q27/datafusion:vortex-compact 155574025 161748794 0.96
tpcds_q28/datafusion:vortex-compact 99433950 100767351 0.99
tpcds_q29/datafusion:vortex-compact 94914002 95768900 0.99
tpcds_q30/datafusion:vortex-compact 🚀 35474509 39711430 0.89
tpcds_q31/datafusion:vortex-compact 123875626 129639778 0.96
tpcds_q32/datafusion:vortex-compact 26757122 27993778 0.96
tpcds_q33/datafusion:vortex-compact 52212495 52878569 0.99
tpcds_q34/datafusion:vortex-compact 49480345 50332610 0.98
tpcds_q35/datafusion:vortex-compact 67555529 70530886 0.96
tpcds_q36/datafusion:vortex-compact 99203208 100010206 0.99
tpcds_q37/datafusion:vortex-compact 46665154 49277573 0.95
tpcds_q38/datafusion:vortex-compact 65408547 66673005 0.98
tpcds_q39/datafusion:vortex-compact 153310197 164460918 0.93
tpcds_q40/datafusion:vortex-compact 58361913 60568527 0.96
tpcds_q41/datafusion:vortex-compact 21397452 21819264 0.98
tpcds_q42/datafusion:vortex-compact 24429399 25525413 0.96
tpcds_q43/datafusion:vortex-compact 38404709 38862737 0.99
tpcds_q44/datafusion:vortex-compact 74537277 76077348 0.98
tpcds_q45/datafusion:vortex-compact 37398494 38438683 0.97
tpcds_q46/datafusion:vortex-compact 69678996 68849755 1.01
tpcds_q47/datafusion:vortex-compact 230635653 242936630 0.95
tpcds_q48/datafusion:vortex-compact 94476560 99941025 0.95
tpcds_q49/datafusion:vortex-compact 95252635 100581996 0.95
tpcds_q50/datafusion:vortex-compact 64475721 64958120 0.99
tpcds_q51/datafusion:vortex-compact 97332292 99119908 0.98
tpcds_q52/datafusion:vortex-compact 24130364 26510411 0.91
tpcds_q53/datafusion:vortex-compact 35089301 35098920 1.00
tpcds_q54/datafusion:vortex-compact 59103895 59987156 0.99
tpcds_q55/datafusion:vortex-compact 24526885 25656638 0.96
tpcds_q56/datafusion:vortex-compact 54802479 55912396 0.98
tpcds_q57/datafusion:vortex-compact 137010158 145757977 0.94
tpcds_q58/datafusion:vortex-compact 77437515 81685052 0.95
tpcds_q59/datafusion:vortex-compact 87497637 88079826 0.99
tpcds_q60/datafusion:vortex-compact 54009655 56255340 0.96
tpcds_q61/datafusion:vortex-compact 73465417 72469990 1.01
tpcds_q62/datafusion:vortex-compact 32521321 34545420 0.94
tpcds_q63/datafusion:vortex-compact 35661196 35927854 0.99
tpcds_q64/datafusion:vortex-compact 539731253 573034379 0.94
tpcds_q65/datafusion:vortex-compact 69505079 68896636 1.01
tpcds_q66/datafusion:vortex-compact 97545778 105814864 0.92
tpcds_q67/datafusion:vortex-compact 127581260 131480354 0.97
tpcds_q68/datafusion:vortex-compact 69947448 72960390 0.96
tpcds_q69/datafusion:vortex-compact 60819842 63889321 0.95
tpcds_q70/datafusion:vortex-compact 126931600 133994844 0.95
tpcds_q71/datafusion:vortex-compact 45635570 47230941 0.97
tpcds_q72/datafusion:vortex-compact 2632685224 2741101491 0.96
tpcds_q73/datafusion:vortex-compact 49902660 52114219 0.96
tpcds_q74/datafusion:vortex-compact 131105814 136199215 0.96
tpcds_q75/datafusion:vortex-compact 166219589 172363588 0.96
tpcds_q76/datafusion:vortex-compact 52416116 52767039 0.99
tpcds_q77/datafusion:vortex-compact 69573862 68038604 1.02
tpcds_q78/datafusion:vortex-compact 147297936 151724345 0.97
tpcds_q79/datafusion:vortex-compact 59040488 63257409 0.93
tpcds_q80/datafusion:vortex-compact 131355921 136234109 0.96
tpcds_q81/datafusion:vortex-compact 42601773 44108365 0.97
tpcds_q82/datafusion:vortex-compact 48221087 50918687 0.95
tpcds_q83/datafusion:vortex-compact 59958303 62431930 0.96
tpcds_q84/datafusion:vortex-compact 18931329 20346554 0.93
tpcds_q85/datafusion:vortex-compact 125641637 129950720 0.97
tpcds_q86/datafusion:vortex-compact 19903565 21610269 0.92
tpcds_q87/datafusion:vortex-compact 65442685 67856086 0.96
tpcds_q88/datafusion:vortex-compact 138275641 131754484 1.05
tpcds_q89/datafusion:vortex-compact 45718596 46186299 0.99
tpcds_q90/datafusion:vortex-compact 23737103 25038292 0.95
tpcds_q91/datafusion:vortex-compact 40797521 42818452 0.95
tpcds_q92/datafusion:vortex-compact 20756261 22307422 0.93
tpcds_q93/datafusion:vortex-compact 47703764 47858458 1.00
tpcds_q94/datafusion:vortex-compact 31249138 32788520 0.95
tpcds_q95/datafusion:vortex-compact 70522534 75496977 0.93
tpcds_q96/datafusion:vortex-compact 27455893 27244199 1.01
tpcds_q97/datafusion:vortex-compact 41432731 43063069 0.96
tpcds_q98/datafusion:vortex-compact 38027403 40234098 0.95
tpcds_q99/datafusion:vortex-compact 50644603 52585527 0.96
datafusion / parquet (0.952x ➖, 4↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpcds_q01/datafusion:parquet 38609321 37814130 1.02
tpcds_q02/datafusion:parquet 45453896 48021527 0.95
tpcds_q03/datafusion:parquet 🚀 14386810 16339273 0.88
tpcds_q04/datafusion:parquet 327639047 342073122 0.96
tpcds_q05/datafusion:parquet 53468999 54988018 0.97
tpcds_q06/datafusion:parquet 65721700 66725240 0.98
tpcds_q07/datafusion:parquet 89120733 93792012 0.95
tpcds_q08/datafusion:parquet 33226624 34015507 0.98
tpcds_q09/datafusion:parquet 60029450 59799833 1.00
tpcds_q10/datafusion:parquet 83767264 87616343 0.96
tpcds_q11/datafusion:parquet 182374749 197403705 0.92
tpcds_q12/datafusion:parquet 24639949 24054426 1.02
tpcds_q13/datafusion:parquet 85434307 89890752 0.95
tpcds_q14/datafusion:parquet 194131038 202286694 0.96
tpcds_q15/datafusion:parquet 28289102 27736922 1.02
tpcds_q16/datafusion:parquet 27688403 30267812 0.91
tpcds_q17/datafusion:parquet 71946013 75514891 0.95
tpcds_q18/datafusion:parquet 130474800 136599847 0.96
tpcds_q19/datafusion:parquet 27432120 28630178 0.96
tpcds_q20/datafusion:parquet 21236404 22777265 0.93
tpcds_q21/datafusion:parquet 21662880 23038860 0.94
tpcds_q22/datafusion:parquet 🚀 132340267 152779119 0.87
tpcds_q23/datafusion:parquet 164020466 167459293 0.98
tpcds_q24/datafusion:parquet 107009265 107780946 0.99
tpcds_q25/datafusion:parquet 70501459 72674381 0.97
tpcds_q26/datafusion:parquet 75037955 77975861 0.96
tpcds_q27/datafusion:parquet 164569419 172350472 0.95
tpcds_q28/datafusion:parquet 48397086 50321081 0.96
tpcds_q29/datafusion:parquet 74425699 76224563 0.98
tpcds_q30/datafusion:parquet 46360798 46940398 0.99
tpcds_q31/datafusion:parquet 75000595 80250664 0.93
tpcds_q32/datafusion:parquet 22702921 24479260 0.93
tpcds_q33/datafusion:parquet 36172258 36631173 0.99
tpcds_q34/datafusion:parquet 28372967 27695166 1.02
tpcds_q35/datafusion:parquet 86119276 91177941 0.94
tpcds_q36/datafusion:parquet 64727032 66566170 0.97
tpcds_q37/datafusion:parquet 23442439 25504985 0.92
tpcds_q38/datafusion:parquet 49211440 54461508 0.90
tpcds_q39/datafusion:parquet 83439180 89955926 0.93
tpcds_q40/datafusion:parquet 27942636 29944336 0.93
tpcds_q41/datafusion:parquet 16405438 17684867 0.93
tpcds_q42/datafusion:parquet 13933131 14616738 0.95
tpcds_q43/datafusion:parquet 19435912 20562496 0.95
tpcds_q44/datafusion:parquet 42282390 42928928 0.98
tpcds_q45/datafusion:parquet 33238285 33663857 0.99
tpcds_q46/datafusion:parquet 38343295 39805991 0.96
tpcds_q47/datafusion:parquet 136583624 140760415 0.97
tpcds_q48/datafusion:parquet 80430811 86131678 0.93
tpcds_q49/datafusion:parquet 67182797 68839607 0.98
tpcds_q50/datafusion:parquet 50174421 51431583 0.98
tpcds_q51/datafusion:parquet 92196164 92369098 1.00
tpcds_q52/datafusion:parquet 14104323 15304462 0.92
tpcds_q53/datafusion:parquet 21378900 23380231 0.91
tpcds_q54/datafusion:parquet 42685247 44603124 0.96
tpcds_q55/datafusion:parquet 14221696 15128068 0.94
tpcds_q56/datafusion:parquet 35866025 37643003 0.95
tpcds_q57/datafusion:parquet 114902825 119801361 0.96
tpcds_q58/datafusion:parquet 58776224 59648171 0.99
tpcds_q59/datafusion:parquet 59280238 59809198 0.99
tpcds_q60/datafusion:parquet 35860668 38929885 0.92
tpcds_q61/datafusion:parquet 50926855 53861002 0.95
tpcds_q62/datafusion:parquet 🚀 23930323 31464912 0.76
tpcds_q63/datafusion:parquet 21475822 22847769 0.94
tpcds_q64/datafusion:parquet 548642352 577304657 0.95
tpcds_q65/datafusion:parquet 44030657 45140599 0.98
tpcds_q66/datafusion:parquet 76282189 82744436 0.92
tpcds_q67/datafusion:parquet 153810644 154108958 1.00
tpcds_q68/datafusion:parquet 39075984 41372714 0.94
tpcds_q69/datafusion:parquet 79535410 80819247 0.98
tpcds_q70/datafusion:parquet 94080603 103767683 0.91
tpcds_q71/datafusion:parquet 25858491 26685803 0.97
tpcds_q72/datafusion:parquet 642479572 644839109 1.00
tpcds_q73/datafusion:parquet 25407056 27311574 0.93
tpcds_q74/datafusion:parquet 106344157 109651852 0.97
tpcds_q75/datafusion:parquet 117217172 123343855 0.95
tpcds_q76/datafusion:parquet 36558132 37487906 0.98
tpcds_q77/datafusion:parquet 48985354 52833106 0.93
tpcds_q78/datafusion:parquet 121375145 126200573 0.96
tpcds_q79/datafusion:parquet 31474882 31819017 0.99
tpcds_q80/datafusion:parquet 92707131 95785753 0.97
tpcds_q81/datafusion:parquet 41538073 42314772 0.98
tpcds_q82/datafusion:parquet 23988855 24611664 0.97
tpcds_q83/datafusion:parquet 46093449 49879051 0.92
tpcds_q84/datafusion:parquet 47779395 50725234 0.94
tpcds_q85/datafusion:parquet 169297779 181909230 0.93
tpcds_q86/datafusion:parquet 17979239 18525817 0.97
tpcds_q87/datafusion:parquet 48914174 51910256 0.94
tpcds_q88/datafusion:parquet 69375619 72136568 0.96
tpcds_q89/datafusion:parquet 24402131 25337097 0.96
tpcds_q90/datafusion:parquet 18582941 20132354 0.92
tpcds_q91/datafusion:parquet 69298534 72874354 0.95
tpcds_q92/datafusion:parquet 21938219 23641076 0.93
tpcds_q93/datafusion:parquet 35800185 39292396 0.91
tpcds_q94/datafusion:parquet 25261280 27348911 0.92
tpcds_q95/datafusion:parquet 67961092 71779790 0.95
tpcds_q96/datafusion:parquet 14168072 15458938 0.92
tpcds_q97/datafusion:parquet 34548886 35482268 0.97
tpcds_q98/datafusion:parquet 25115738 27523253 0.91
tpcds_q99/datafusion:parquet 🚀 28109666 31641117 0.89
duckdb / vortex-file-compressed (0.959x ➖, 6↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpcds_q01/duckdb:vortex-file-compressed 26302311 27340126 0.96
tpcds_q02/duckdb:vortex-file-compressed 89410810 84854761 1.05
tpcds_q03/duckdb:vortex-file-compressed 23623226 24746292 0.95
tpcds_q04/duckdb:vortex-file-compressed 105730587 112090503 0.94
tpcds_q05/duckdb:vortex-file-compressed 100362884 104507929 0.96
tpcds_q06/duckdb:vortex-file-compressed 36482717 38292540 0.95
tpcds_q07/duckdb:vortex-file-compressed 24469619 25361780 0.96
tpcds_q08/duckdb:vortex-file-compressed 23359442 25195883 0.93
tpcds_q09/duckdb:vortex-file-compressed 39429563 41387225 0.95
tpcds_q10/duckdb:vortex-file-compressed 36354952 38195185 0.95
tpcds_q11/duckdb:vortex-file-compressed 62804982 64753393 0.97
tpcds_q12/duckdb:vortex-file-compressed 14141125 15077821 0.94
tpcds_q13/duckdb:vortex-file-compressed 34211535 34977651 0.98
tpcds_q14/duckdb:vortex-file-compressed 200112907 210022364 0.95
tpcds_q15/duckdb:vortex-file-compressed 41243258 43442215 0.95
tpcds_q16/duckdb:vortex-file-compressed 30672922 31410508 0.98
tpcds_q17/duckdb:vortex-file-compressed 39979629 39629788 1.01
tpcds_q18/duckdb:vortex-file-compressed 46249447 49213126 0.94
tpcds_q19/duckdb:vortex-file-compressed 32872551 35459356 0.93
tpcds_q20/duckdb:vortex-file-compressed 16009698 15771790 1.02
tpcds_q21/duckdb:vortex-file-compressed 21851081 20182867 1.08
tpcds_q22/duckdb:vortex-file-compressed 110363865 115767119 0.95
tpcds_q23/duckdb:vortex-file-compressed 131513068 129263011 1.02
tpcds_q24/duckdb:vortex-file-compressed 56158958 57832541 0.97
tpcds_q25/duckdb:vortex-file-compressed 37809108 40584996 0.93
tpcds_q26/duckdb:vortex-file-compressed 42046192 40407962 1.04
tpcds_q27/duckdb:vortex-file-compressed 89382413 91292922 0.98
tpcds_q28/duckdb:vortex-file-compressed 36321991 36470432 1.00
tpcds_q29/duckdb:vortex-file-compressed 38397922 38902311 0.99
tpcds_q30/duckdb:vortex-file-compressed 28733384 27940221 1.03
tpcds_q31/duckdb:vortex-file-compressed 26039530 28851200 0.90
tpcds_q32/duckdb:vortex-file-compressed 16688451 17940804 0.93
tpcds_q33/duckdb:vortex-file-compressed 68062114 72259964 0.94
tpcds_q34/duckdb:vortex-file-compressed 20120602 20483655 0.98
tpcds_q35/duckdb:vortex-file-compressed 65980162 69575743 0.95
tpcds_q36/duckdb:vortex-file-compressed 27888590 29910703 0.93
tpcds_q37/duckdb:vortex-file-compressed 21887611 22641327 0.97
tpcds_q38/duckdb:vortex-file-compressed 36677789 39831795 0.92
tpcds_q39/duckdb:vortex-file-compressed 47689470 47662266 1.00
tpcds_q40/duckdb:vortex-file-compressed 24102681 26024940 0.93
tpcds_q41/duckdb:vortex-file-compressed 13214384 14194310 0.93
tpcds_q42/duckdb:vortex-file-compressed 11628927 12834039 0.91
tpcds_q43/duckdb:vortex-file-compressed 20036838 20641825 0.97
tpcds_q44/duckdb:vortex-file-compressed 23305241 24157759 0.96
tpcds_q45/duckdb:vortex-file-compressed 🚀 26491506 29514127 0.90
tpcds_q46/duckdb:vortex-file-compressed 48599347 51974910 0.94
tpcds_q47/duckdb:vortex-file-compressed 50158563 51198819 0.98
tpcds_q48/duckdb:vortex-file-compressed 30303741 32667256 0.93
tpcds_q49/duckdb:vortex-file-compressed 30389977 29800075 1.02
tpcds_q50/duckdb:vortex-file-compressed 37226761 37370682 1.00
tpcds_q51/duckdb:vortex-file-compressed 102860602 103440064 0.99
tpcds_q52/duckdb:vortex-file-compressed 12399551 13459127 0.92
tpcds_q53/duckdb:vortex-file-compressed 17287291 17848249 0.97
tpcds_q54/duckdb:vortex-file-compressed 48178063 52402790 0.92
tpcds_q55/duckdb:vortex-file-compressed 12070198 12186793 0.99
tpcds_q56/duckdb:vortex-file-compressed 67460741 72343881 0.93
tpcds_q57/duckdb:vortex-file-compressed 46842803 48854636 0.96
tpcds_q58/duckdb:vortex-file-compressed 33069080 36330634 0.91
tpcds_q59/duckdb:vortex-file-compressed 🚀 55811148 62692560 0.89
tpcds_q60/duckdb:vortex-file-compressed 71281342 76162346 0.94
tpcds_q61/duckdb:vortex-file-compressed 40233993 40414328 1.00
tpcds_q62/duckdb:vortex-file-compressed 17783818 18957405 0.94
tpcds_q63/duckdb:vortex-file-compressed 18073171 18662928 0.97
tpcds_q64/duckdb:vortex-file-compressed 80931591 82563941 0.98
tpcds_q65/duckdb:vortex-file-compressed 23425030 24233924 0.97
tpcds_q66/duckdb:vortex-file-compressed 57615962 58384505 0.99
tpcds_q67/duckdb:vortex-file-compressed 147873758 148857441 0.99
tpcds_q68/duckdb:vortex-file-compressed 42110539 43479819 0.97
tpcds_q69/duckdb:vortex-file-compressed 38023122 40730204 0.93
tpcds_q70/duckdb:vortex-file-compressed 24938239 26061146 0.96
tpcds_q71/duckdb:vortex-file-compressed 22562295 22193574 1.02
tpcds_q72/duckdb:vortex-file-compressed 177873385 180917465 0.98
tpcds_q73/duckdb:vortex-file-compressed 20262192 19252798 1.05
tpcds_q74/duckdb:vortex-file-compressed 🚀 50983960 57847278 0.88
tpcds_q75/duckdb:vortex-file-compressed 62493736 63684639 0.98
tpcds_q76/duckdb:vortex-file-compressed 69191377 71514356 0.97
tpcds_q77/duckdb:vortex-file-compressed 67276483 67666113 0.99
tpcds_q78/duckdb:vortex-file-compressed 79350212 78183478 1.01
tpcds_q79/duckdb:vortex-file-compressed 🚀 36571575 43456307 0.84
tpcds_q80/duckdb:vortex-file-compressed 94873002 95907026 0.99
tpcds_q81/duckdb:vortex-file-compressed 33487310 36750462 0.91
tpcds_q82/duckdb:vortex-file-compressed 21167859 23125000 0.92
tpcds_q83/duckdb:vortex-file-compressed 27203993 29027133 0.94
tpcds_q84/duckdb:vortex-file-compressed 17060879 18300420 0.93
tpcds_q85/duckdb:vortex-file-compressed 44885757 46952437 0.96
tpcds_q86/duckdb:vortex-file-compressed 🚀 15972072 18682650 0.85
tpcds_q87/duckdb:vortex-file-compressed 42620436 43475020 0.98
tpcds_q88/duckdb:vortex-file-compressed 35283045 37284687 0.95
tpcds_q89/duckdb:vortex-file-compressed 18845702 18660409 1.01
tpcds_q90/duckdb:vortex-file-compressed 15281582 16564963 0.92
tpcds_q91/duckdb:vortex-file-compressed 🚀 26718866 29791113 0.90
tpcds_q92/duckdb:vortex-file-compressed 14203149 14859541 0.96
tpcds_q93/duckdb:vortex-file-compressed 29392904 31754503 0.93
tpcds_q94/duckdb:vortex-file-compressed 22225306 23359712 0.95
tpcds_q95/duckdb:vortex-file-compressed 156574322 164852644 0.95
tpcds_q96/duckdb:vortex-file-compressed 12583650 12339372 1.02
tpcds_q97/duckdb:vortex-file-compressed 36930878 40676665 0.91
tpcds_q98/duckdb:vortex-file-compressed 18801872 19334827 0.97
tpcds_q99/duckdb:vortex-file-compressed 33564876 34018262 0.99
duckdb / vortex-compact (0.989x ➖, 6↑ 2↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpcds_q01/duckdb:vortex-compact 33717369 34944246 0.96
tpcds_q02/duckdb:vortex-compact 124852519 129339307 0.97
tpcds_q03/duckdb:vortex-compact 51721423 53158940 0.97
tpcds_q04/duckdb:vortex-compact 123093415 124755351 0.99
tpcds_q05/duckdb:vortex-compact 160436577 159682477 1.00
tpcds_q06/duckdb:vortex-compact 37680799 38747643 0.97
tpcds_q07/duckdb:vortex-compact 36234830 37354806 0.97
tpcds_q08/duckdb:vortex-compact 28459621 30363751 0.94
tpcds_q09/duckdb:vortex-compact 110175970 113169233 0.97
tpcds_q10/duckdb:vortex-compact 45854652 49689610 0.92
tpcds_q11/duckdb:vortex-compact 76621960 78669348 0.97
tpcds_q12/duckdb:vortex-compact 18575513 20309575 0.91
tpcds_q13/duckdb:vortex-compact 48870363 49750184 0.98
tpcds_q14/duckdb:vortex-compact 283194359 279964841 1.01
tpcds_q15/duckdb:vortex-compact 44730393 48476794 0.92
tpcds_q16/duckdb:vortex-compact 36369264 38058639 0.96
tpcds_q17/duckdb:vortex-compact 50688332 49879742 1.02
tpcds_q18/duckdb:vortex-compact 68803192 72090071 0.95
tpcds_q19/duckdb:vortex-compact 38762559 39669967 0.98
tpcds_q20/duckdb:vortex-compact 22336456 22164444 1.01
tpcds_q21/duckdb:vortex-compact 28287362 28495970 0.99
tpcds_q22/duckdb:vortex-compact 118334023 119936635 0.99
tpcds_q23/duckdb:vortex-compact 169969493 166327891 1.02
tpcds_q24/duckdb:vortex-compact 68597925 71539146 0.96
tpcds_q25/duckdb:vortex-compact 49677350 48686893 1.02
tpcds_q26/duckdb:vortex-compact 50581838 50859532 0.99
tpcds_q27/duckdb:vortex-compact 132024228 138863108 0.95
tpcds_q28/duckdb:vortex-compact 106685471 108122657 0.99
tpcds_q29/duckdb:vortex-compact 50565944 48523347 1.04
tpcds_q30/duckdb:vortex-compact 34573236 35809906 0.97
tpcds_q31/duckdb:vortex-compact 🚀 29453767 33982668 0.87
tpcds_q32/duckdb:vortex-compact 23813879 24717127 0.96
tpcds_q33/duckdb:vortex-compact 135709353 132066615 1.03
tpcds_q34/duckdb:vortex-compact 25073682 23708657 1.06
tpcds_q35/duckdb:vortex-compact 67718442 71156604 0.95
tpcds_q36/duckdb:vortex-compact 35880437 35180379 1.02
tpcds_q37/duckdb:vortex-compact 31156980 32481147 0.96
tpcds_q38/duckdb:vortex-compact 40576272 39878043 1.02
tpcds_q39/duckdb:vortex-compact 54240198 57156975 0.95
tpcds_q40/duckdb:vortex-compact 🚀 29513336 33504327 0.88
tpcds_q41/duckdb:vortex-compact 17131836 18706911 0.92
tpcds_q42/duckdb:vortex-compact 🚀 14646232 16285051 0.90
tpcds_q43/duckdb:vortex-compact 25197591 24684949 1.02
tpcds_q44/duckdb:vortex-compact 31274238 29833638 1.05
tpcds_q45/duckdb:vortex-compact 32748007 35290660 0.93
tpcds_q46/duckdb:vortex-compact 57064468 61499793 0.93
tpcds_q47/duckdb:vortex-compact 55561156 56358120 0.99
tpcds_q48/duckdb:vortex-compact 40619791 41998596 0.97
tpcds_q49/duckdb:vortex-compact 48765471 49519745 0.98
tpcds_q50/duckdb:vortex-compact 42421701 38917438 1.09
tpcds_q51/duckdb:vortex-compact 105432305 108924429 0.97
tpcds_q52/duckdb:vortex-compact 15417657 15960421 0.97
tpcds_q53/duckdb:vortex-compact 22681708 23093058 0.98
tpcds_q54/duckdb:vortex-compact 86482253 89916713 0.96
tpcds_q55/duckdb:vortex-compact 15660872 16086610 0.97
tpcds_q56/duckdb:vortex-compact 132113689 138894952 0.95
tpcds_q57/duckdb:vortex-compact 47707731 49679503 0.96
tpcds_q58/duckdb:vortex-compact 🚀 38280536 42719695 0.90
tpcds_q59/duckdb:vortex-compact 🚀 65215915 76507440 0.85
tpcds_q60/duckdb:vortex-compact 134004856 130339304 1.03
tpcds_q61/duckdb:vortex-compact 47495749 46174549 1.03
tpcds_q62/duckdb:vortex-compact 22698374 22761744 1.00
tpcds_q63/duckdb:vortex-compact 23057080 22178431 1.04
tpcds_q64/duckdb:vortex-compact 103218990 103884831 0.99
tpcds_q65/duckdb:vortex-compact 28498643 29303401 0.97
tpcds_q66/duckdb:vortex-compact 97857204 99154711 0.99
tpcds_q67/duckdb:vortex-compact 152334417 152925957 1.00
tpcds_q68/duckdb:vortex-compact 52022287 50540206 1.03
tpcds_q69/duckdb:vortex-compact 46133800 47368969 0.97
tpcds_q70/duckdb:vortex-compact 29118141 28054102 1.04
tpcds_q71/duckdb:vortex-compact 27242101 25843586 1.05
tpcds_q72/duckdb:vortex-compact 196536546 198635886 0.99
tpcds_q73/duckdb:vortex-compact 23590710 22567690 1.05
tpcds_q74/duckdb:vortex-compact 51963789 53883059 0.96
tpcds_q75/duckdb:vortex-compact 72240559 71021439 1.02
tpcds_q76/duckdb:vortex-compact 121066316 117607575 1.03
tpcds_q77/duckdb:vortex-compact 129652363 124508651 1.04
tpcds_q78/duckdb:vortex-compact 85303589 87982318 0.97
tpcds_q79/duckdb:vortex-compact 🚀 57984348 68936608 0.84
tpcds_q80/duckdb:vortex-compact 162872677 158534806 1.03
tpcds_q81/duckdb:vortex-compact 40847413 40713470 1.00
tpcds_q82/duckdb:vortex-compact 33460910 31250969 1.07
tpcds_q83/duckdb:vortex-compact 46127220 45376929 1.02
tpcds_q84/duckdb:vortex-compact 21401871 21026085 1.02
tpcds_q85/duckdb:vortex-compact 59197234 60774561 0.97
tpcds_q86/duckdb:vortex-compact 19430195 19694353 0.99
tpcds_q87/duckdb:vortex-compact 41743554 42847066 0.97
tpcds_q88/duckdb:vortex-compact 🚨 38847121 34162292 1.14
tpcds_q89/duckdb:vortex-compact 23043212 22014821 1.05
tpcds_q90/duckdb:vortex-compact 16128321 16252585 0.99
tpcds_q91/duckdb:vortex-compact 44618234 44155014 1.01
tpcds_q92/duckdb:vortex-compact 20197831 19727421 1.02
tpcds_q93/duckdb:vortex-compact 37577592 35692003 1.05
tpcds_q94/duckdb:vortex-compact 27439139 27540362 1.00
tpcds_q95/duckdb:vortex-compact 156785248 156839449 1.00
tpcds_q96/duckdb:vortex-compact 🚨 14138554 11171646 1.27
tpcds_q97/duckdb:vortex-compact 43269252 39667737 1.09
tpcds_q98/duckdb:vortex-compact 25149362 23873939 1.05
tpcds_q99/duckdb:vortex-compact 38951967 39902449 0.98
duckdb / parquet (0.972x ➖, 3↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpcds_q01/duckdb:parquet 43771503 46918855 0.93
tpcds_q02/duckdb:parquet 60743969 60807516 1.00
tpcds_q03/duckdb:parquet 12024749 12289191 0.98
tpcds_q04/duckdb:parquet 126073256 128685538 0.98
tpcds_q05/duckdb:parquet 74793865 76623022 0.98
tpcds_q06/duckdb:parquet 27272584 29695826 0.92
tpcds_q07/duckdb:parquet 23403362 24298655 0.96
tpcds_q08/duckdb:parquet 28202750 29211716 0.97
tpcds_q09/duckdb:parquet 39725323 40624963 0.98
tpcds_q10/duckdb:parquet 34901738 34478563 1.01
tpcds_q11/duckdb:parquet 88844383 92144854 0.96
tpcds_q12/duckdb:parquet 14139749 15185308 0.93
tpcds_q13/duckdb:parquet 34244654 34866692 0.98
tpcds_q14/duckdb:parquet 157903869 165758317 0.95
tpcds_q15/duckdb:parquet 30062575 31197071 0.96
tpcds_q16/duckdb:parquet 22787457 22926034 0.99
tpcds_q17/duckdb:parquet 37208261 39697475 0.94
tpcds_q18/duckdb:parquet 46550159 46344436 1.00
tpcds_q19/duckdb:parquet 28388350 31107423 0.91
tpcds_q20/duckdb:parquet 15537120 14970140 1.04
tpcds_q21/duckdb:parquet 11402389 12582020 0.91
tpcds_q22/duckdb:parquet 64326928 68510217 0.94
tpcds_q23/duckdb:parquet 114391104 115547190 0.99
tpcds_q24/duckdb:parquet 44749599 45687415 0.98
tpcds_q25/duckdb:parquet 32779223 32441703 1.01
tpcds_q26/duckdb:parquet 34644090 35618574 0.97
tpcds_q27/duckdb:parquet 88183351 90197158 0.98
tpcds_q28/duckdb:parquet 36873043 37510800 0.98
tpcds_q29/duckdb:parquet 35186325 38282550 0.92
tpcds_q30/duckdb:parquet 40136151 41434161 0.97
tpcds_q31/duckdb:parquet 22784720 22900015 0.99
tpcds_q32/duckdb:parquet 11341811 11086465 1.02
tpcds_q33/duckdb:parquet 70531919 71974037 0.98
tpcds_q34/duckdb:parquet 20901861 21759847 0.96
tpcds_q35/duckdb:parquet 60966611 60811972 1.00
tpcds_q36/duckdb:parquet 20164709 20961754 0.96
tpcds_q37/duckdb:parquet 13366538 13846524 0.97
tpcds_q38/duckdb:parquet 33348915 36385484 0.92
tpcds_q39/duckdb:parquet 32463432 33359985 0.97
tpcds_q40/duckdb:parquet 17804260 18541576 0.96
tpcds_q41/duckdb:parquet 7581844 7993668 0.95
tpcds_q42/duckdb:parquet 🚀 10163575 11325147 0.90
tpcds_q43/duckdb:parquet 15986673 16155913 0.99
tpcds_q44/duckdb:parquet 22692078 22521724 1.01
tpcds_q45/duckdb:parquet 26144585 27086166 0.97
tpcds_q46/duckdb:parquet 45552288 46478997 0.98
tpcds_q47/duckdb:parquet 46293250 45144437 1.03
tpcds_q48/duckdb:parquet 30203810 32512061 0.93
tpcds_q49/duckdb:parquet 25673830 25334840 1.01
tpcds_q50/duckdb:parquet 24281646 25606809 0.95
tpcds_q51/duckdb:parquet 94845218 94245525 1.01
tpcds_q52/duckdb:parquet 12133364 11388837 1.07
tpcds_q53/duckdb:parquet 16086829 16570390 0.97
tpcds_q54/duckdb:parquet 41760470 39691167 1.05
tpcds_q55/duckdb:parquet 10698907 10739021 1.00
tpcds_q56/duckdb:parquet 68949154 68052506 1.01
tpcds_q57/duckdb:parquet 35373741 35484275 1.00
tpcds_q58/duckdb:parquet 20849833 21251015 0.98
tpcds_q59/duckdb:parquet 36501210 37247539 0.98
tpcds_q60/duckdb:parquet 70393563 72214960 0.97
tpcds_q61/duckdb:parquet 30797371 31301333 0.98
tpcds_q62/duckdb:parquet 12405499 11602987 1.07
tpcds_q63/duckdb:parquet 15013358 15690110 0.96
tpcds_q64/duckdb:parquet 78198705 74770465 1.05
tpcds_q65/duckdb:parquet 19860543 21238875 0.94
tpcds_q66/duckdb:parquet 🚀 55043124 61902298 0.89
tpcds_q67/duckdb:parquet 132339438 137458354 0.96
tpcds_q68/duckdb:parquet 37381441 39186123 0.95
tpcds_q69/duckdb:parquet 35948404 38638846 0.93
tpcds_q70/duckdb:parquet 19373173 20015314 0.97
tpcds_q71/duckdb:parquet 19577016 18874067 1.04
tpcds_q72/duckdb:parquet 161521154 166825866 0.97
tpcds_q73/duckdb:parquet 18048611 19696477 0.92
tpcds_q74/duckdb:parquet 49855230 51414774 0.97
tpcds_q75/duckdb:parquet 53103020 56841195 0.93
tpcds_q76/duckdb:parquet 69612844 67987400 1.02
tpcds_q77/duckdb:parquet 62244755 63257784 0.98
tpcds_q78/duckdb:parquet 74215227 74910230 0.99
tpcds_q79/duckdb:parquet 27957811 28372486 0.99
tpcds_q80/duckdb:parquet 82868477 86193528 0.96
tpcds_q81/duckdb:parquet 47222370 48719240 0.97
tpcds_q82/duckdb:parquet 15597503 15987017 0.98
tpcds_q83/duckdb:parquet 16045307 16661449 0.96
tpcds_q84/duckdb:parquet 18855349 20101207 0.94
tpcds_q85/duckdb:parquet 40091117 42014590 0.95
tpcds_q86/duckdb:parquet 12035574 12691995 0.95
tpcds_q87/duckdb:parquet 37759340 38402541 0.98
tpcds_q88/duckdb:parquet 28892371 29566850 0.98
tpcds_q89/duckdb:parquet 16994330 17921586 0.95
tpcds_q90/duckdb:parquet 7620724 8024564 0.95
tpcds_q91/duckdb:parquet 22959925 23260222 0.99
tpcds_q92/duckdb:parquet 11974072 12027330 1.00
tpcds_q93/duckdb:parquet 30171066 30928622 0.98
tpcds_q94/duckdb:parquet 17545085 17755570 0.99
tpcds_q95/duckdb:parquet 141227349 147364352 0.96
tpcds_q96/duckdb:parquet 🚀 8743286 9717052 0.90
tpcds_q97/duckdb:parquet 31873358 33888864 0.94
tpcds_q98/duckdb:parquet 20059917 18508532 1.08
tpcds_q99/duckdb:parquet 19273388 20894559 0.92
duckdb / duckdb (0.972x ➖, 4↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
tpcds_q01/duckdb:duckdb 20016158 21101126 0.95
tpcds_q02/duckdb:duckdb 19208925 18886669 1.02
tpcds_q03/duckdb:duckdb 8421402 8178972 1.03
tpcds_q04/duckdb:duckdb 157584304 162419882 0.97
tpcds_q05/duckdb:duckdb 17572026 18837127 0.93
tpcds_q06/duckdb:duckdb 🚀 21633143 24160773 0.90
tpcds_q07/duckdb:duckdb 36717754 39149582 0.94
tpcds_q08/duckdb:duckdb 118783972 119632826 0.99
tpcds_q09/duckdb:duckdb 19219178 19490459 0.99
tpcds_q10/duckdb:duckdb 22692541 23288627 0.97
tpcds_q11/duckdb:duckdb 136670452 140153829 0.98
tpcds_q12/duckdb:duckdb 11042297 11484659 0.96
tpcds_q13/duckdb:duckdb 23072158 23608304 0.98
tpcds_q14/duckdb:duckdb 90733603 95905785 0.95
tpcds_q15/duckdb:duckdb 23725777 25645586 0.93
tpcds_q16/duckdb:duckdb 16491614 18130391 0.91
tpcds_q17/duckdb:duckdb 22031440 21751380 1.01
tpcds_q18/duckdb:duckdb 42578377 44667135 0.95
tpcds_q19/duckdb:duckdb 15567579 15878110 0.98
tpcds_q20/duckdb:duckdb 11708926 12293188 0.95
tpcds_q21/duckdb:duckdb 6782115 6926981 0.98
tpcds_q22/duckdb:duckdb 59635749 61727046 0.97
tpcds_q23/duckdb:duckdb 🚀 72964863 81173383 0.90
tpcds_q24/duckdb:duckdb 22647638 22834837 0.99
tpcds_q25/duckdb:duckdb 17268463 17697735 0.98
tpcds_q26/duckdb:duckdb 26180406 26699923 0.98
tpcds_q27/duckdb:duckdb 38975819 37646049 1.04
tpcds_q28/duckdb:duckdb 20851989 21749057 0.96
tpcds_q29/duckdb:duckdb 19960425 19791888 1.01
tpcds_q30/duckdb:duckdb 28128225 30201118 0.93
tpcds_q31/duckdb:duckdb 21175648 20261722 1.05
tpcds_q32/duckdb:duckdb 7148580 7302933 0.98
tpcds_q33/duckdb:duckdb 🚀 12411772 14070750 0.88
tpcds_q34/duckdb:duckdb 14724784 14561005 1.01
tpcds_q35/duckdb:duckdb 31186081 33826142 0.92
tpcds_q36/duckdb:duckdb 68863395 69296414 0.99
tpcds_q37/duckdb:duckdb 7546545 7809452 0.97
tpcds_q38/duckdb:duckdb 29983857 31075794 0.96
tpcds_q39/duckdb:duckdb 26291816 25764452 1.02
tpcds_q40/duckdb:duckdb 13550284 12995570 1.04
tpcds_q41/duckdb:duckdb 7840132 8271469 0.95
tpcds_q42/duckdb:duckdb 5930963 6050710 0.98
tpcds_q43/duckdb:duckdb 10818429 11134609 0.97
tpcds_q44/duckdb:duckdb 13259615 13816231 0.96
tpcds_q45/duckdb:duckdb 16433708 17854282 0.92
tpcds_q46/duckdb:duckdb 34666976 36031346 0.96
tpcds_q47/duckdb:duckdb 40002941 40824642 0.98
tpcds_q48/duckdb:duckdb 21273603 21800746 0.98
tpcds_q49/duckdb:duckdb 15110286 15824345 0.95
tpcds_q50/duckdb:duckdb 13474645 13877451 0.97
tpcds_q51/duckdb:duckdb 89551753 92288376 0.97
tpcds_q52/duckdb:duckdb 7058644 7048373 1.00
tpcds_q53/duckdb:duckdb 11446106 11749728 0.97
tpcds_q54/duckdb:duckdb 15692758 17066706 0.92
tpcds_q55/duckdb:duckdb 6639093 7005610 0.95
tpcds_q56/duckdb:duckdb 13790051 13000342 1.06
tpcds_q57/duckdb:duckdb 30748386 32691807 0.94
tpcds_q58/duckdb:duckdb 11777951 12219409 0.96
tpcds_q59/duckdb:duckdb 33559434 34370612 0.98
tpcds_q60/duckdb:duckdb 14194252 14506848 0.98
tpcds_q61/duckdb:duckdb 13952494 13844775 1.01
tpcds_q62/duckdb:duckdb 8979541 9121992 0.98
tpcds_q63/duckdb:duckdb 10897807 11241365 0.97
tpcds_q64/duckdb:duckdb 53040910 53348572 0.99
tpcds_q65/duckdb:duckdb 30681864 31393580 0.98
tpcds_q66/duckdb:duckdb 24784686 24961389 0.99
tpcds_q67/duckdb:duckdb 126892911 127332398 1.00
tpcds_q68/duckdb:duckdb 24146968 24780024 0.97
tpcds_q69/duckdb:duckdb 23281908 23743367 0.98
tpcds_q70/duckdb:duckdb 15083133 14254782 1.06
tpcds_q71/duckdb:duckdb 12266794 12601368 0.97
tpcds_q72/duckdb:duckdb 41514200 42341591 0.98
tpcds_q73/duckdb:duckdb 10320798 11233471 0.92
tpcds_q74/duckdb:duckdb 97977371 97166867 1.01
tpcds_q75/duckdb:duckdb 39325036 42821159 0.92
tpcds_q76/duckdb:duckdb 11922130 12423465 0.96
tpcds_q77/duckdb:duckdb 11186717 11803879 0.95
tpcds_q78/duckdb:duckdb 62931442 60681919 1.04
tpcds_q79/duckdb:duckdb 17184878 18001229 0.95
tpcds_q80/duckdb:duckdb 28471464 26943033 1.06
tpcds_q81/duckdb:duckdb 35983068 36742694 0.98
tpcds_q82/duckdb:duckdb 8388476 8863178 0.95
tpcds_q83/duckdb:duckdb 8999227 9099764 0.99
tpcds_q84/duckdb:duckdb 13862589 13237220 1.05
tpcds_q85/duckdb:duckdb 23128186 24274414 0.95
tpcds_q86/duckdb:duckdb 10719296 10559153 1.02
tpcds_q87/duckdb:duckdb 33250582 34574414 0.96
tpcds_q88/duckdb:duckdb 23914941 24784864 0.96
tpcds_q89/duckdb:duckdb 12804705 13075283 0.98
tpcds_q90/duckdb:duckdb 5337633 5442281 0.98
tpcds_q91/duckdb:duckdb 12402123 12604329 0.98
tpcds_q92/duckdb:duckdb 🚀 8363560 9356834 0.89
tpcds_q93/duckdb:duckdb 20975473 23179232 0.90
tpcds_q94/duckdb:duckdb 12825771 13168554 0.97
tpcds_q95/duckdb:duckdb 108038901 112093759 0.96
tpcds_q96/duckdb:duckdb 4367126 4676727 0.93
tpcds_q97/duckdb:duckdb 29703629 30638755 0.97
tpcds_q98/duckdb:duckdb 13506998 13231304 1.02
tpcds_q99/duckdb:duckdb 16659150 17097171 0.97
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
1 datafusion:vortex-compact -9.2% -2.4% -7.0% +14.8% ➖ noise
1 datafusion:vortex-file-compressed -11.6% -2.4% -9.4% +13.2% ➖ noise
1 duckdb:duckdb -5.1% -2.4% -2.8% +16.4% ➖ noise
1 duckdb:vortex-compact -3.5% -2.4% -1.1% +16.4% ➖ noise
1 duckdb:vortex-file-compressed -3.8% -2.4% -1.4% +11.5% ➖ noise
2 datafusion:vortex-compact -3.8% -2.8% -1.1% +10.0% ➖ noise
2 datafusion:vortex-file-compressed -9.7% -2.8% -7.1% +10.0% ➖ noise
2 duckdb:duckdb +1.7% -2.8% +4.6% +11.7% ➖ noise
2 duckdb:vortex-compact -3.5% -2.8% -0.7% +10.0% ➖ noise
2 duckdb:vortex-file-compressed +5.4% -2.8% +8.4% +13.0% ➖ noise
3 datafusion:vortex-compact -0.5% -7.2% +7.2% +10.0% ➖ noise
3 datafusion:vortex-file-compressed -7.6% -7.2% -0.4% +15.3% ➖ noise
3 duckdb:duckdb +3.0% -7.2% +10.9% +16.5% ➖ noise
3 duckdb:vortex-compact -2.7% -7.2% +4.8% +12.3% ➖ noise
3 duckdb:vortex-file-compressed -4.5% -7.2% +2.8% +11.4% ➖ noise
4 datafusion:vortex-compact -2.3% -3.1% +0.9% +10.0% ➖ noise
4 datafusion:vortex-file-compressed -6.3% -3.1% -3.2% +10.0% ➖ noise
4 duckdb:duckdb -3.0% -3.1% +0.2% +10.0% ➖ noise
4 duckdb:vortex-compact -1.3% -3.1% +1.9% +10.0% ➖ noise
4 duckdb:vortex-file-compressed -5.7% -3.1% -2.6% +10.0% ➖ noise
5 datafusion:vortex-compact -3.1% -2.6% -0.6% +11.9% ➖ noise
5 datafusion:vortex-file-compressed -11.2% -2.6% -8.8% +10.0% ➖ noise
5 duckdb:duckdb -6.7% -2.6% -4.3% +15.6% ➖ noise
5 duckdb:vortex-compact +0.5% -2.6% +3.1% +12.7% ➖ noise
5 duckdb:vortex-file-compressed -4.0% -2.6% -1.4% +12.5% ➖ noise
6 datafusion:vortex-compact -3.9% -4.9% +1.0% +13.1% ➖ noise
6 datafusion:vortex-file-compressed -4.8% -4.9% +0.1% +10.6% ➖ noise
6 duckdb:duckdb -10.5% -4.9% -5.9% +13.6% ➖ noise
6 duckdb:vortex-compact -2.8% -4.9% +2.2% +10.5% ➖ noise
6 duckdb:vortex-file-compressed -4.7% -4.9% +0.2% +10.6% ➖ noise
7 datafusion:vortex-compact -4.7% -4.3% -0.4% +10.0% ➖ noise
7 datafusion:vortex-file-compressed -4.2% -4.3% +0.2% +10.0% ➖ noise
7 duckdb:duckdb -6.2% -4.3% -2.0% +15.8% ➖ noise
7 duckdb:vortex-compact -3.0% -4.3% +1.4% +10.0% ➖ noise
7 duckdb:vortex-file-compressed -3.5% -4.3% +0.9% +10.0% ➖ noise
8 datafusion:vortex-compact -2.7% -2.9% +0.2% +10.0% ➖ noise
8 datafusion:vortex-file-compressed -8.7% -2.9% -5.9% +10.0% ➖ noise
8 duckdb:duckdb -0.7% -2.9% +2.2% +10.0% ➖ noise
8 duckdb:vortex-compact -6.3% -2.9% -3.5% +12.3% ➖ noise
8 duckdb:vortex-file-compressed -7.3% -2.9% -4.5% +10.3% ➖ noise
9 datafusion:vortex-compact -5.9% -0.9% -5.0% +10.8% ➖ noise
9 datafusion:vortex-file-compressed -5.0% -0.9% -4.1% +10.0% ➖ noise
9 duckdb:duckdb -1.4% -0.9% -0.5% +10.0% ➖ noise
9 duckdb:vortex-compact -2.6% -0.9% -1.7% +10.0% ➖ noise
9 duckdb:vortex-file-compressed -4.7% -0.9% -3.8% +10.0% ➖ noise
10 datafusion:vortex-compact -4.4% -1.6% -2.8% +10.0% ➖ noise
10 datafusion:vortex-file-compressed -6.4% -1.6% -4.9% +10.0% ➖ noise
10 duckdb:duckdb -2.6% -1.6% -1.0% +10.0% ➖ noise
10 duckdb:vortex-compact -7.7% -1.6% -6.2% +10.0% ➖ noise
10 duckdb:vortex-file-compressed -4.8% -1.6% -3.2% +10.0% ➖ noise
11 datafusion:vortex-compact -2.6% -5.6% +3.2% +10.0% ➖ noise
11 datafusion:vortex-file-compressed -5.3% -5.6% +0.3% +10.0% ➖ noise
11 duckdb:duckdb -2.5% -5.6% +3.3% +10.0% ➖ noise
11 duckdb:vortex-compact -2.6% -5.6% +3.2% +10.0% ➖ noise
11 duckdb:vortex-file-compressed -3.0% -5.6% +2.8% +10.0% ➖ noise
12 datafusion:vortex-compact -3.8% -2.3% -1.5% +10.0% ➖ noise
12 datafusion:vortex-file-compressed -0.7% -2.3% +1.7% +12.7% ➖ noise
12 duckdb:duckdb -3.9% -2.3% -1.6% +11.8% ➖ noise
12 duckdb:vortex-compact -8.5% -2.3% -6.3% +11.6% ➖ noise
12 duckdb:vortex-file-compressed -6.2% -2.3% -4.0% +11.5% ➖ noise
13 datafusion:vortex-compact -3.7% -3.4% -0.3% +10.0% ➖ noise
13 datafusion:vortex-file-compressed -2.0% -3.4% +1.5% +10.0% ➖ noise
13 duckdb:duckdb -2.3% -3.4% +1.2% +10.0% ➖ noise
13 duckdb:vortex-compact -1.8% -3.4% +1.7% +10.0% ➖ noise
13 duckdb:vortex-file-compressed -2.2% -3.4% +1.2% +12.4% ➖ noise
14 datafusion:vortex-compact -1.5% -4.4% +3.0% +10.0% ➖ noise
14 datafusion:vortex-file-compressed -5.5% -4.4% -1.2% +10.0% ➖ noise
14 duckdb:duckdb -5.4% -4.4% -1.1% +10.0% ➖ noise
14 duckdb:vortex-compact +1.2% -4.4% +5.8% +10.0% ➖ noise
14 duckdb:vortex-file-compressed -4.7% -4.4% -0.3% +10.0% ➖ noise
15 datafusion:vortex-compact -4.7% -0.9% -3.9% +15.5% ➖ noise
15 datafusion:vortex-file-compressed -9.0% -0.9% -8.3% +13.7% ➖ noise
15 duckdb:duckdb -7.5% -0.9% -6.7% +17.1% ➖ noise
15 duckdb:vortex-compact -7.7% -0.9% -6.9% +18.3% ➖ noise
15 duckdb:vortex-file-compressed -5.1% -0.9% -4.2% +14.6% ➖ noise
16 datafusion:vortex-compact +4.1% -4.6% +9.2% +10.6% ➖ noise
16 datafusion:vortex-file-compressed -10.6% -4.6% -6.2% +10.0% ➖ noise
16 duckdb:duckdb -9.0% -4.6% -4.6% +11.6% ➖ noise
16 duckdb:vortex-compact -4.4% -4.6% +0.2% +12.4% ➖ noise
16 duckdb:vortex-file-compressed -2.3% -4.6% +2.4% +10.0% ➖ noise
17 datafusion:vortex-compact +2.1% -5.5% +8.1% +10.0% ➖ noise
17 datafusion:vortex-file-compressed -8.3% -5.5% -3.0% +10.0% ➖ noise
17 duckdb:duckdb +1.3% -5.5% +7.2% +12.6% ➖ noise
17 duckdb:vortex-compact +1.6% -5.5% +7.5% +10.0% ➖ noise
17 duckdb:vortex-file-compressed +0.9% -5.5% +6.8% +10.0% ➖ noise
18 datafusion:vortex-compact +1.4% -2.1% +3.5% +10.0% ➖ noise
18 datafusion:vortex-file-compressed -3.9% -2.1% -1.9% +10.0% ➖ noise
18 duckdb:duckdb -4.7% -2.1% -2.7% +10.0% ➖ noise
18 duckdb:vortex-compact -4.6% -2.1% -2.6% +10.0% ➖ noise
18 duckdb:vortex-file-compressed -6.0% -2.1% -4.1% +10.0% ➖ noise
19 datafusion:vortex-compact +0.6% -6.5% +7.6% +10.0% ➖ noise
19 datafusion:vortex-file-compressed -5.3% -6.5% +1.2% +10.0% ➖ noise
19 duckdb:duckdb -2.0% -6.5% +4.8% +10.0% ➖ noise
19 duckdb:vortex-compact -2.3% -6.5% +4.5% +10.0% ➖ noise
19 duckdb:vortex-file-compressed -7.3% -6.5% -0.9% +10.0% ➖ noise
20 datafusion:vortex-compact -3.6% -1.6% -2.0% +10.0% ➖ noise
20 datafusion:vortex-file-compressed -6.3% -1.6% -4.7% +10.0% ➖ noise
20 duckdb:duckdb -4.8% -1.6% -3.2% +12.8% ➖ noise
20 duckdb:vortex-compact +0.8% -1.6% +2.4% +11.6% ➖ noise
20 duckdb:vortex-file-compressed +1.5% -1.6% +3.2% +11.9% ➖ noise
21 datafusion:vortex-compact -4.1% -7.7% +3.9% +11.2% ➖ noise
21 datafusion:vortex-file-compressed -1.8% -7.7% +6.4% +11.6% ➖ noise
21 duckdb:duckdb -2.1% -7.7% +6.1% +19.6% ➖ noise
21 duckdb:vortex-compact -0.7% -7.7% +7.5% +13.1% ➖ noise
21 duckdb:vortex-file-compressed +8.3% -7.7% +17.3% +13.4% 🚨 regression
22 datafusion:vortex-compact +3.8% -9.8% +15.1% +11.6% 🚨 regression
22 datafusion:vortex-file-compressed -8.3% -9.8% +1.7% +13.6% ➖ noise
22 duckdb:duckdb -3.4% -9.8% +7.1% +11.1% ➖ noise
22 duckdb:vortex-compact -1.3% -9.8% +9.4% +12.2% ➖ noise
22 duckdb:vortex-file-compressed -4.7% -9.8% +5.7% +13.7% ➖ noise
23 datafusion:vortex-compact -1.2% -1.5% +0.3% +10.0% ➖ noise
23 datafusion:vortex-file-compressed -4.6% -1.5% -3.1% +10.0% ➖ noise
23 duckdb:duckdb -10.1% -1.5% -8.7% +15.9% ➖ noise
23 duckdb:vortex-compact +2.2% -1.5% +3.8% +12.0% ➖ noise
23 duckdb:vortex-file-compressed +1.7% -1.5% +3.3% +10.0% ➖ noise
24 datafusion:vortex-compact -1.3% -1.4% +0.0% +10.0% ➖ noise
24 datafusion:vortex-file-compressed -5.6% -1.4% -4.2% +10.0% ➖ noise
24 duckdb:duckdb -0.8% -1.4% +0.6% +10.4% ➖ noise
24 duckdb:vortex-compact -4.1% -1.4% -2.8% +10.0% ➖ noise
24 duckdb:vortex-file-compressed -2.9% -1.4% -1.5% +10.0% ➖ noise
25 datafusion:vortex-compact -4.1% -1.0% -3.2% +10.0% ➖ noise
25 datafusion:vortex-file-compressed -6.7% -1.0% -5.8% +10.0% ➖ noise
25 duckdb:duckdb -2.4% -1.0% -1.4% +10.0% ➖ noise
25 duckdb:vortex-compact +2.0% -1.0% +3.1% +10.7% ➖ noise
25 duckdb:vortex-file-compressed -6.8% -1.0% -5.9% +10.0% ➖ noise
26 datafusion:vortex-compact -3.1% -3.3% +0.2% +10.8% ➖ noise
26 datafusion:vortex-file-compressed -4.8% -3.3% -1.6% +10.0% ➖ noise
26 duckdb:duckdb -1.9% -3.3% +1.4% +12.5% ➖ noise
26 duckdb:vortex-compact -0.5% -3.3% +2.8% +10.1% ➖ noise
26 duckdb:vortex-file-compressed +4.1% -3.3% +7.6% +15.9% ➖ noise
27 datafusion:vortex-compact -3.8% -3.4% -0.5% +10.0% ➖ noise
27 datafusion:vortex-file-compressed -4.6% -3.4% -1.3% +10.0% ➖ noise
27 duckdb:duckdb +3.5% -3.4% +7.2% +15.0% ➖ noise
27 duckdb:vortex-compact -4.9% -3.4% -1.6% +10.0% ➖ noise
27 duckdb:vortex-file-compressed -2.1% -3.4% +1.3% +10.0% ➖ noise
28 datafusion:vortex-compact -1.3% -2.8% +1.5% +10.0% ➖ noise
28 datafusion:vortex-file-compressed -3.3% -2.8% -0.6% +10.0% ➖ noise
28 duckdb:duckdb -4.1% -2.8% -1.4% +10.4% ➖ noise
28 duckdb:vortex-compact -1.3% -2.8% +1.5% +10.0% ➖ noise
28 duckdb:vortex-file-compressed -0.4% -2.8% +2.4% +10.0% ➖ noise
29 datafusion:vortex-compact -0.9% -5.3% +4.6% +10.0% ➖ noise
29 datafusion:vortex-file-compressed -6.5% -5.3% -1.3% +10.0% ➖ noise
29 duckdb:duckdb +0.9% -5.3% +6.5% +15.2% ➖ noise
29 duckdb:vortex-compact +4.2% -5.3% +10.0% +12.2% ➖ noise
29 duckdb:vortex-file-compressed -1.3% -5.3% +4.2% +10.0% ➖ noise
30 datafusion:vortex-compact -10.7% -2.2% -8.7% +10.3% ➖ noise
30 datafusion:vortex-file-compressed -6.0% -2.2% -3.9% +10.0% ➖ noise
30 duckdb:duckdb -6.9% -2.2% -4.8% +11.8% ➖ noise
30 duckdb:vortex-compact -3.5% -2.2% -1.3% +10.5% ➖ noise
30 duckdb:vortex-file-compressed +2.8% -2.2% +5.1% +14.3% ➖ noise
31 datafusion:vortex-compact -4.4% -3.6% -0.9% +10.0% ➖ noise
31 datafusion:vortex-file-compressed -6.5% -3.6% -3.0% +10.0% ➖ noise
31 duckdb:duckdb +4.5% -3.6% +8.4% +10.0% ➖ noise
31 duckdb:vortex-compact -13.3% -3.6% -10.1% +13.4% ➖ noise
31 duckdb:vortex-file-compressed -9.7% -3.6% -6.4% +12.7% ➖ noise
32 datafusion:vortex-compact -4.4% -2.6% -1.9% +10.0% ➖ noise
32 datafusion:vortex-file-compressed -7.7% -2.6% -5.2% +10.0% ➖ noise
32 duckdb:duckdb -2.1% -2.6% +0.5% +17.4% ➖ noise
32 duckdb:vortex-compact -3.7% -2.6% -1.1% +10.0% ➖ noise
32 duckdb:vortex-file-compressed -7.0% -2.6% -4.5% +10.0% ➖ noise
33 datafusion:vortex-compact -1.3% -1.6% +0.4% +10.0% ➖ noise
33 datafusion:vortex-file-compressed -7.4% -1.6% -5.9% +10.0% ➖ noise
33 duckdb:duckdb -11.8% -1.6% -10.3% +12.4% ➖ noise
33 duckdb:vortex-compact +2.8% -1.6% +4.5% +10.0% ➖ noise
33 duckdb:vortex-file-compressed -5.8% -1.6% -4.2% +10.8% ➖ noise
34 datafusion:vortex-compact -1.7% -0.8% -0.9% +10.0% ➖ noise
34 datafusion:vortex-file-compressed -12.1% -0.8% -11.4% +10.0% ✅ faster
34 duckdb:duckdb +1.1% -0.8% +1.9% +10.0% ➖ noise
34 duckdb:vortex-compact +5.8% -0.8% +6.6% +10.0% ➖ noise
34 duckdb:vortex-file-compressed -1.8% -0.8% -1.0% +10.0% ➖ noise
35 datafusion:vortex-compact -4.2% -2.7% -1.6% +10.0% ➖ noise
35 datafusion:vortex-file-compressed -5.6% -2.7% -3.0% +10.0% ➖ noise
35 duckdb:duckdb -7.8% -2.7% -5.3% +11.6% ➖ noise
35 duckdb:vortex-compact -4.8% -2.7% -2.2% +11.0% ➖ noise
35 duckdb:vortex-file-compressed -5.2% -2.7% -2.5% +10.0% ➖ noise
36 datafusion:vortex-compact -0.8% -3.3% +2.6% +10.0% ➖ noise
36 datafusion:vortex-file-compressed -4.8% -3.3% -1.5% +10.0% ➖ noise
36 duckdb:duckdb -0.6% -3.3% +2.7% +11.5% ➖ noise
36 duckdb:vortex-compact +2.0% -3.3% +5.5% +10.0% ➖ noise
36 duckdb:vortex-file-compressed -6.8% -3.3% -3.6% +10.3% ➖ noise
37 datafusion:vortex-compact -5.3% -5.8% +0.5% +10.0% ➖ noise
37 datafusion:vortex-file-compressed -2.7% -5.8% +3.3% +12.3% ➖ noise
37 duckdb:duckdb -3.4% -5.8% +2.6% +18.4% ➖ noise
37 duckdb:vortex-compact -4.1% -5.8% +1.8% +12.1% ➖ noise
37 duckdb:vortex-file-compressed -3.3% -5.8% +2.6% +10.5% ➖ noise
38 datafusion:vortex-compact -1.9% -9.0% +7.8% +10.0% ➖ noise
38 datafusion:vortex-file-compressed -7.7% -9.0% +1.4% +10.0% ➖ noise
38 duckdb:duckdb -3.5% -9.0% +6.0% +13.9% ➖ noise
38 duckdb:vortex-compact +1.8% -9.0% +11.8% +15.6% ➖ noise
38 duckdb:vortex-file-compressed -7.9% -9.0% +1.2% +21.5% ➖ noise
39 datafusion:vortex-compact -6.8% -5.0% -1.9% +10.6% ➖ noise
39 datafusion:vortex-file-compressed -7.4% -5.0% -2.5% +14.2% ➖ noise
39 duckdb:duckdb +2.0% -5.0% +7.4% +11.6% ➖ noise
39 duckdb:vortex-compact -5.1% -5.0% -0.1% +14.6% ➖ noise
39 duckdb:vortex-file-compressed +0.1% -5.0% +5.3% +19.6% ➖ noise
40 datafusion:vortex-compact -3.6% -5.3% +1.8% +10.0% ➖ noise
40 datafusion:vortex-file-compressed -5.1% -5.3% +0.3% +10.0% ➖ noise
40 duckdb:duckdb +4.3% -5.3% +10.2% +15.3% ➖ noise
40 duckdb:vortex-compact -11.9% -5.3% -6.9% +10.0% ➖ noise
40 duckdb:vortex-file-compressed -7.4% -5.3% -2.2% +15.3% ➖ noise
41 datafusion:vortex-compact -1.9% -6.2% +4.5% +15.0% ➖ noise
41 datafusion:vortex-file-compressed -7.7% -6.2% -1.6% +10.3% ➖ noise
41 duckdb:duckdb -5.2% -6.2% +1.0% +10.0% ➖ noise
41 duckdb:vortex-compact -8.4% -6.2% -2.4% +10.0% ➖ noise
41 duckdb:vortex-file-compressed -6.9% -6.2% -0.8% +10.0% ➖ noise
42 datafusion:vortex-compact -4.3% -7.5% +3.5% +10.0% ➖ noise
42 datafusion:vortex-file-compressed -15.5% -7.5% -8.6% +10.0% ➖ noise
42 duckdb:duckdb -2.0% -7.5% +6.0% +15.8% ➖ noise
42 duckdb:vortex-compact -10.1% -7.5% -2.8% +13.4% ➖ noise
42 duckdb:vortex-file-compressed -9.4% -7.5% -2.0% +16.8% ➖ noise
43 datafusion:vortex-compact -1.2% -3.3% +2.2% +10.0% ➖ noise
43 datafusion:vortex-file-compressed -5.2% -3.3% -1.9% +10.0% ➖ noise
43 duckdb:duckdb -2.8% -3.3% +0.5% +10.0% ➖ noise
43 duckdb:vortex-compact +2.1% -3.3% +5.5% +10.0% ➖ noise
43 duckdb:vortex-file-compressed -2.9% -3.3% +0.4% +10.0% ➖ noise
44 datafusion:vortex-compact -2.0% -0.4% -1.6% +10.0% ➖ noise
44 datafusion:vortex-file-compressed -11.6% -0.4% -11.3% +10.0% ✅ faster
44 duckdb:duckdb -4.0% -0.4% -3.7% +10.1% ➖ noise
44 duckdb:vortex-compact +4.8% -0.4% +5.2% +10.0% ➖ noise
44 duckdb:vortex-file-compressed -3.5% -0.4% -3.2% +10.6% ➖ noise
45 datafusion:vortex-compact -2.7% -2.4% -0.3% +10.0% ➖ noise
45 datafusion:vortex-file-compressed -1.8% -2.4% +0.6% +10.0% ➖ noise
45 duckdb:duckdb -8.0% -2.4% -5.7% +10.2% ➖ noise
45 duckdb:vortex-compact -7.2% -2.4% -4.9% +12.8% ➖ noise
45 duckdb:vortex-file-compressed -10.2% -2.4% -8.1% +10.0% ➖ noise
46 datafusion:vortex-compact +1.2% -2.8% +4.2% +10.0% ➖ noise
46 datafusion:vortex-file-compressed -9.9% -2.8% -7.3% +10.0% ➖ noise
46 duckdb:duckdb -3.8% -2.8% -1.0% +10.0% ➖ noise
46 duckdb:vortex-compact -7.2% -2.8% -4.5% +10.0% ➖ noise
46 duckdb:vortex-file-compressed -6.5% -2.8% -3.8% +10.0% ➖ noise
47 datafusion:vortex-compact -5.1% -0.2% -4.8% +10.0% ➖ noise
47 datafusion:vortex-file-compressed -6.3% -0.2% -6.0% +10.0% ➖ noise
47 duckdb:duckdb -2.0% -0.2% -1.8% +10.0% ➖ noise
47 duckdb:vortex-compact -1.4% -0.2% -1.2% +10.0% ➖ noise
47 duckdb:vortex-file-compressed -2.0% -0.2% -1.8% +10.0% ➖ noise
48 datafusion:vortex-compact -5.5% -6.9% +1.5% +10.5% ➖ noise
48 datafusion:vortex-file-compressed -2.3% -6.9% +4.9% +10.0% ➖ noise
48 duckdb:duckdb -2.4% -6.9% +4.8% +10.0% ➖ noise
48 duckdb:vortex-compact -3.3% -6.9% +3.8% +10.0% ➖ noise
48 duckdb:vortex-file-compressed -7.2% -6.9% -0.4% +14.2% ➖ noise
49 datafusion:vortex-compact -5.3% -0.6% -4.8% +10.0% ➖ noise
49 datafusion:vortex-file-compressed -4.9% -0.6% -4.4% +10.0% ➖ noise
49 duckdb:duckdb -4.5% -0.6% -4.0% +17.3% ➖ noise
49 duckdb:vortex-compact -1.5% -0.6% -1.0% +10.6% ➖ noise
49 duckdb:vortex-file-compressed +2.0% -0.6% +2.5% +10.2% ➖ noise
50 datafusion:vortex-compact -0.7% -3.8% +3.2% +10.0% ➖ noise
50 datafusion:vortex-file-compressed -5.8% -3.8% -2.0% +10.0% ➖ noise
50 duckdb:duckdb -2.9% -3.8% +1.0% +10.0% ➖ noise
50 duckdb:vortex-compact +9.0% -3.8% +13.3% +10.0% 🚨 regression
50 duckdb:vortex-file-compressed -0.4% -3.8% +3.6% +10.0% ➖ noise
51 datafusion:vortex-compact -1.8% +0.2% -2.0% +10.0% ➖ noise
51 datafusion:vortex-file-compressed -2.0% +0.2% -2.2% +10.0% ➖ noise
51 duckdb:duckdb -3.0% +0.2% -3.2% +15.5% ➖ noise
51 duckdb:vortex-compact -3.2% +0.2% -3.4% +16.1% ➖ noise
51 duckdb:vortex-file-compressed -0.6% +0.2% -0.8% +25.1% ➖ noise
52 datafusion:vortex-compact -9.0% -0.9% -8.1% +11.8% ➖ noise
52 datafusion:vortex-file-compressed -10.8% -0.9% -10.0% +11.2% ➖ noise
52 duckdb:duckdb +0.1% -0.9% +1.1% +14.5% ➖ noise
52 duckdb:vortex-compact -3.4% -0.9% -2.5% +22.5% ➖ noise
52 duckdb:vortex-file-compressed -7.9% -0.9% -7.0% +19.4% ➖ noise
53 datafusion:vortex-compact -0.0% -5.8% +6.1% +10.0% ➖ noise
53 datafusion:vortex-file-compressed -8.8% -5.8% -3.2% +10.0% ➖ noise
53 duckdb:duckdb -2.6% -5.8% +3.4% +10.4% ➖ noise
53 duckdb:vortex-compact -1.8% -5.8% +4.2% +10.0% ➖ noise
53 duckdb:vortex-file-compressed -3.1% -5.8% +2.8% +10.0% ➖ noise
54 datafusion:vortex-compact -1.5% +0.3% -1.8% +10.0% ➖ noise
54 datafusion:vortex-file-compressed -4.2% +0.3% -4.5% +10.0% ➖ noise
54 duckdb:duckdb -8.1% +0.3% -8.4% +12.9% ➖ noise
54 duckdb:vortex-compact -3.8% +0.3% -4.1% +10.0% ➖ noise
54 duckdb:vortex-file-compressed -8.1% +0.3% -8.4% +10.0% ➖ noise
55 datafusion:vortex-compact -4.4% -3.2% -1.2% +10.6% ➖ noise
55 datafusion:vortex-file-compressed -6.9% -3.2% -3.9% +14.1% ➖ noise
55 duckdb:duckdb -5.2% -3.2% -2.1% +16.6% ➖ noise
55 duckdb:vortex-compact -2.6% -3.2% +0.6% +12.6% ➖ noise
55 duckdb:vortex-file-compressed -1.0% -3.2% +2.3% +16.5% ➖ noise
56 datafusion:vortex-compact -2.0% -1.7% -0.2% +10.0% ➖ noise
56 datafusion:vortex-file-compressed -5.5% -1.7% -3.8% +10.4% ➖ noise
56 duckdb:duckdb +6.1% -1.7% +8.0% +14.3% ➖ noise
56 duckdb:vortex-compact -4.9% -1.7% -3.2% +10.3% ➖ noise
56 duckdb:vortex-file-compressed -6.7% -1.7% -5.1% +12.5% ➖ noise
57 datafusion:vortex-compact -6.0% -2.2% -3.9% +10.0% ➖ noise
57 datafusion:vortex-file-compressed -6.6% -2.2% -4.5% +10.7% ➖ noise
57 duckdb:duckdb -5.9% -2.2% -3.8% +14.4% ➖ noise
57 duckdb:vortex-compact -4.0% -2.2% -1.8% +17.0% ➖ noise
57 duckdb:vortex-file-compressed -4.1% -2.2% -1.9% +10.5% ➖ noise
58 datafusion:vortex-compact -5.2% -1.7% -3.6% +10.0% ➖ noise
58 datafusion:vortex-file-compressed -2.2% -1.7% -0.6% +10.0% ➖ noise
58 duckdb:duckdb -3.6% -1.7% -2.0% +15.1% ➖ noise
58 duckdb:vortex-compact -10.4% -1.7% -8.9% +17.1% ➖ noise
58 duckdb:vortex-file-compressed -9.0% -1.7% -7.4% +14.0% ➖ noise
59 datafusion:vortex-compact -0.7% -1.4% +0.8% +10.0% ➖ noise
59 datafusion:vortex-file-compressed -4.5% -1.4% -3.1% +10.0% ➖ noise
59 duckdb:duckdb -2.4% -1.4% -0.9% +12.5% ➖ noise
59 duckdb:vortex-compact -14.8% -1.4% -13.5% +10.0% ✅ faster
59 duckdb:vortex-file-compressed -11.0% -1.4% -9.7% +10.0% ✅ faster
60 datafusion:vortex-compact -4.0% -5.2% +1.3% +10.0% ➖ noise
60 datafusion:vortex-file-compressed -7.2% -5.2% -2.1% +10.0% ➖ noise
60 duckdb:duckdb -2.2% -5.2% +3.3% +10.0% ➖ noise
60 duckdb:vortex-compact +2.8% -5.2% +8.5% +10.0% ➖ noise
60 duckdb:vortex-file-compressed -6.4% -5.2% -1.2% +12.3% ➖ noise
61 datafusion:vortex-compact +1.4% -3.5% +5.1% +10.0% ➖ noise
61 datafusion:vortex-file-compressed -7.1% -3.5% -3.7% +10.0% ➖ noise
61 duckdb:duckdb +0.8% -3.5% +4.5% +10.3% ➖ noise
61 duckdb:vortex-compact +2.9% -3.5% +6.6% +11.4% ➖ noise
61 duckdb:vortex-file-compressed -0.4% -3.5% +3.2% +15.7% ➖ noise
62 datafusion:vortex-compact -5.9% -9.8% +4.4% +15.3% ➖ noise
62 datafusion:vortex-file-compressed -8.9% -9.8% +1.1% +14.2% ➖ noise
62 duckdb:duckdb -1.6% -9.8% +9.2% +12.6% ➖ noise
62 duckdb:vortex-compact -0.3% -9.8% +10.6% +27.4% ➖ noise
62 duckdb:vortex-file-compressed -6.2% -9.8% +4.0% +17.3% ➖ noise
63 datafusion:vortex-compact -0.7% -5.2% +4.7% +10.0% ➖ noise
63 datafusion:vortex-file-compressed -10.5% -5.2% -5.6% +10.0% ➖ noise
63 duckdb:duckdb -3.1% -5.2% +2.2% +10.0% ➖ noise
63 duckdb:vortex-compact +4.0% -5.2% +9.6% +10.0% ➖ noise
63 duckdb:vortex-file-compressed -3.2% -5.2% +2.1% +12.7% ➖ noise
64 datafusion:vortex-compact -5.8% -0.3% -5.5% +10.0% ➖ noise
64 datafusion:vortex-file-compressed -5.4% -0.3% -5.2% +10.0% ➖ noise
64 duckdb:duckdb -0.6% -0.3% -0.3% +10.0% ➖ noise
64 duckdb:vortex-compact -0.6% -0.3% -0.3% +10.0% ➖ noise
64 duckdb:vortex-file-compressed -2.0% -0.3% -1.7% +10.0% ➖ noise
65 datafusion:vortex-compact +0.9% -4.5% +5.6% +11.4% ➖ noise
65 datafusion:vortex-file-compressed -9.9% -4.5% -5.7% +10.2% ➖ noise
65 duckdb:duckdb -2.3% -4.5% +2.3% +12.9% ➖ noise
65 duckdb:vortex-compact -2.7% -4.5% +1.8% +10.6% ➖ noise
65 duckdb:vortex-file-compressed -3.3% -4.5% +1.2% +10.9% ➖ noise
66 datafusion:vortex-compact -7.8% -9.5% +1.8% +12.2% ➖ noise
66 datafusion:vortex-file-compressed -2.8% -9.5% +7.3% +11.5% ➖ noise
66 duckdb:duckdb -0.7% -9.5% +9.7% +13.5% ➖ noise
66 duckdb:vortex-compact -1.3% -9.5% +9.0% +10.6% ➖ noise
66 duckdb:vortex-file-compressed -1.3% -9.5% +9.0% +20.1% ➖ noise
67 datafusion:vortex-compact -3.0% -2.0% -1.0% +11.0% ➖ noise
67 datafusion:vortex-file-compressed -5.1% -2.0% -3.2% +10.6% ➖ noise
67 duckdb:duckdb -0.3% -2.0% +1.7% +10.0% ➖ noise
67 duckdb:vortex-compact -0.4% -2.0% +1.6% +10.0% ➖ noise
67 duckdb:vortex-file-compressed -0.7% -2.0% +1.3% +10.0% ➖ noise
68 datafusion:vortex-compact -4.1% -5.1% +1.0% +11.3% ➖ noise
68 datafusion:vortex-file-compressed -5.5% -5.1% -0.5% +10.0% ➖ noise
68 duckdb:duckdb -2.6% -5.1% +2.7% +14.1% ➖ noise
68 duckdb:vortex-compact +2.9% -5.1% +8.4% +11.4% ➖ noise
68 duckdb:vortex-file-compressed -3.1% -5.1% +2.0% +10.0% ➖ noise
69 datafusion:vortex-compact -4.8% -4.3% -0.5% +10.0% ➖ noise
69 datafusion:vortex-file-compressed -7.1% -4.3% -2.9% +10.0% ➖ noise
69 duckdb:duckdb -1.9% -4.3% +2.5% +10.0% ➖ noise
69 duckdb:vortex-compact -2.6% -4.3% +1.8% +10.0% ➖ noise
69 duckdb:vortex-file-compressed -6.6% -4.3% -2.4% +10.0% ➖ noise
70 datafusion:vortex-compact -5.3% -6.3% +1.1% +10.0% ➖ noise
70 datafusion:vortex-file-compressed -2.6% -6.3% +4.0% +10.0% ➖ noise
70 duckdb:duckdb +5.8% -6.3% +13.0% +14.0% ➖ noise
70 duckdb:vortex-compact +3.8% -6.3% +10.8% +10.7% 🚨 regression
70 duckdb:vortex-file-compressed -4.3% -6.3% +2.1% +10.0% ➖ noise
71 datafusion:vortex-compact -3.4% +0.3% -3.6% +10.3% ➖ noise
71 datafusion:vortex-file-compressed -7.5% +0.3% -7.7% +10.1% ➖ noise
71 duckdb:duckdb -2.7% +0.3% -2.9% +14.8% ➖ noise
71 duckdb:vortex-compact +5.4% +0.3% +5.1% +11.0% ➖ noise
71 duckdb:vortex-file-compressed +1.7% +0.3% +1.4% +12.7% ➖ noise
72 datafusion:vortex-compact -4.0% -1.8% -2.2% +10.0% ➖ noise
72 datafusion:vortex-file-compressed -2.9% -1.8% -1.2% +10.0% ➖ noise
72 duckdb:duckdb -2.0% -1.8% -0.2% +10.0% ➖ noise
72 duckdb:vortex-compact -1.1% -1.8% +0.7% +10.0% ➖ noise
72 duckdb:vortex-file-compressed -1.7% -1.8% +0.1% +10.0% ➖ noise
73 datafusion:vortex-compact -4.2% -7.7% +3.7% +17.2% ➖ noise
73 datafusion:vortex-file-compressed -7.0% -7.7% +0.7% +22.4% ➖ noise
73 duckdb:duckdb -8.1% -7.7% -0.5% +12.1% ➖ noise
73 duckdb:vortex-compact +4.5% -7.7% +13.2% +10.1% 🚨 regression
73 duckdb:vortex-file-compressed +5.2% -7.7% +14.0% +11.5% 🚨 regression
74 datafusion:vortex-compact -3.7% -3.0% -0.7% +10.0% ➖ noise
74 datafusion:vortex-file-compressed -5.6% -3.0% -2.7% +10.0% ➖ noise
74 duckdb:duckdb +0.8% -3.0% +4.0% +10.0% ➖ noise
74 duckdb:vortex-compact -3.6% -3.0% -0.6% +12.8% ➖ noise
74 duckdb:vortex-file-compressed -11.9% -3.0% -9.1% +10.0% ✅ faster
75 datafusion:vortex-compact -3.6% -5.8% +2.3% +10.0% ➖ noise
75 datafusion:vortex-file-compressed -4.6% -5.8% +1.3% +10.0% ➖ noise
75 duckdb:duckdb -8.2% -5.8% -2.5% +10.0% ➖ noise
75 duckdb:vortex-compact +1.7% -5.8% +8.0% +11.7% ➖ noise
75 duckdb:vortex-file-compressed -1.9% -5.8% +4.1% +10.0% ➖ noise
76 datafusion:vortex-compact -0.7% -0.1% -0.6% +11.0% ➖ noise
76 datafusion:vortex-file-compressed -9.7% -0.1% -9.6% +10.0% ✅ faster
76 duckdb:duckdb -4.0% -0.1% -4.0% +10.0% ➖ noise
76 duckdb:vortex-compact +2.9% -0.1% +3.0% +10.0% ➖ noise
76 duckdb:vortex-file-compressed -3.2% -0.1% -3.2% +10.4% ➖ noise
77 datafusion:vortex-compact +2.3% -4.5% +7.1% +10.0% ➖ noise
77 datafusion:vortex-file-compressed -4.5% -4.5% -0.0% +10.0% ➖ noise
77 duckdb:duckdb -5.2% -4.5% -0.8% +11.1% ➖ noise
77 duckdb:vortex-compact +4.1% -4.5% +9.0% +10.0% ➖ noise
77 duckdb:vortex-file-compressed -0.6% -4.5% +4.1% +10.3% ➖ noise
78 datafusion:vortex-compact -2.9% -2.4% -0.5% +10.0% ➖ noise
78 datafusion:vortex-file-compressed -0.8% -2.4% +1.6% +10.0% ➖ noise
78 duckdb:duckdb +3.7% -2.4% +6.2% +10.0% ➖ noise
78 duckdb:vortex-compact -3.0% -2.4% -0.7% +11.2% ➖ noise
78 duckdb:vortex-file-compressed +1.5% -2.4% +4.0% +10.0% ➖ noise
79 datafusion:vortex-compact -6.7% -1.3% -5.5% +10.0% ➖ noise
79 datafusion:vortex-file-compressed -5.1% -1.3% -3.9% +10.0% ➖ noise
79 duckdb:duckdb -4.5% -1.3% -3.3% +10.0% ➖ noise
79 duckdb:vortex-compact -15.9% -1.3% -14.8% +11.7% ✅ faster
79 duckdb:vortex-file-compressed -15.8% -1.3% -14.8% +15.1% ✅ faster
80 datafusion:vortex-compact -3.6% -3.5% -0.0% +10.0% ➖ noise
80 datafusion:vortex-file-compressed -3.4% -3.5% +0.2% +10.0% ➖ noise
80 duckdb:duckdb +5.7% -3.5% +9.5% +11.1% ➖ noise
80 duckdb:vortex-compact +2.7% -3.5% +6.5% +10.3% ➖ noise
80 duckdb:vortex-file-compressed -1.1% -3.5% +2.5% +10.0% ➖ noise
81 datafusion:vortex-compact -3.4% -2.5% -1.0% +12.4% ➖ noise
81 datafusion:vortex-file-compressed -7.1% -2.5% -4.8% +16.2% ➖ noise
81 duckdb:duckdb -2.1% -2.5% +0.4% +12.5% ➖ noise
81 duckdb:vortex-compact +0.3% -2.5% +2.9% +12.6% ➖ noise
81 duckdb:vortex-file-compressed -8.9% -2.5% -6.6% +11.2% ➖ noise
82 datafusion:vortex-compact -5.3% -2.5% -2.9% +10.0% ➖ noise
82 datafusion:vortex-file-compressed -4.6% -2.5% -2.2% +10.9% ➖ noise
82 duckdb:duckdb -5.4% -2.5% -2.9% +17.4% ➖ noise
82 duckdb:vortex-compact +7.1% -2.5% +9.8% +10.0% ➖ noise
82 duckdb:vortex-file-compressed -8.5% -2.5% -6.1% +10.0% ➖ noise
83 datafusion:vortex-compact -4.0% -5.7% +1.8% +10.0% ➖ noise
83 datafusion:vortex-file-compressed -9.0% -5.7% -3.5% +10.0% ➖ noise
83 duckdb:duckdb -1.1% -5.7% +4.8% +10.0% ➖ noise
83 duckdb:vortex-compact +1.7% -5.7% +7.8% +10.0% ➖ noise
83 duckdb:vortex-file-compressed -6.3% -5.7% -0.7% +10.0% ➖ noise
84 datafusion:vortex-compact -7.0% -6.0% -1.0% +10.0% ➖ noise
84 datafusion:vortex-file-compressed -3.9% -6.0% +2.2% +10.0% ➖ noise
84 duckdb:duckdb +4.7% -6.0% +11.4% +11.9% ➖ noise
84 duckdb:vortex-compact +1.8% -6.0% +8.3% +10.0% ➖ noise
84 duckdb:vortex-file-compressed -6.8% -6.0% -0.8% +10.0% ➖ noise
85 datafusion:vortex-compact -3.3% -5.8% +2.6% +10.0% ➖ noise
85 datafusion:vortex-file-compressed -3.8% -5.8% +2.1% +10.0% ➖ noise
85 duckdb:duckdb -4.7% -5.8% +1.1% +10.3% ➖ noise
85 duckdb:vortex-compact -2.6% -5.8% +3.4% +10.0% ➖ noise
85 duckdb:vortex-file-compressed -4.4% -5.8% +1.4% +10.0% ➖ noise
86 datafusion:vortex-compact -7.9% -4.1% -4.0% +12.1% ➖ noise
86 datafusion:vortex-file-compressed -10.4% -4.1% -6.6% +13.4% ➖ noise
86 duckdb:duckdb +1.5% -4.1% +5.8% +20.4% ➖ noise
86 duckdb:vortex-compact -1.3% -4.1% +2.8% +15.7% ➖ noise
86 duckdb:vortex-file-compressed -14.5% -4.1% -10.9% +18.5% ➖ noise
87 datafusion:vortex-compact -3.6% -3.7% +0.2% +10.0% ➖ noise
87 datafusion:vortex-file-compressed -10.6% -3.7% -7.1% +10.0% ➖ noise
87 duckdb:duckdb -3.8% -3.7% -0.1% +17.5% ➖ noise
87 duckdb:vortex-compact -2.6% -3.7% +1.2% +10.2% ➖ noise
87 duckdb:vortex-file-compressed -2.0% -3.7% +1.8% +11.7% ➖ noise
88 datafusion:vortex-compact +4.9% -3.1% +8.3% +10.0% ➖ noise
88 datafusion:vortex-file-compressed -10.0% -3.1% -7.2% +10.0% ➖ noise
88 duckdb:duckdb -3.5% -3.1% -0.5% +10.0% ➖ noise
88 duckdb:vortex-compact +13.7% -3.1% +17.3% +10.0% 🚨 regression
88 duckdb:vortex-file-compressed -5.4% -3.1% -2.4% +10.0% ➖ noise
89 datafusion:vortex-compact -1.0% -4.4% +3.6% +11.8% ➖ noise
89 datafusion:vortex-file-compressed -4.1% -4.4% +0.3% +10.9% ➖ noise
89 duckdb:duckdb -2.1% -4.4% +2.5% +13.3% ➖ noise
89 duckdb:vortex-compact +4.7% -4.4% +9.5% +13.7% ➖ noise
89 duckdb:vortex-file-compressed +1.0% -4.4% +5.7% +14.2% ➖ noise
90 datafusion:vortex-compact -5.2% -6.4% +1.3% +10.0% ➖ noise
90 datafusion:vortex-file-compressed -11.2% -6.4% -5.1% +10.0% ➖ noise
90 duckdb:duckdb -1.9% -6.4% +4.8% +10.0% ➖ noise
90 duckdb:vortex-compact -0.8% -6.4% +6.0% +10.0% ➖ noise
90 duckdb:vortex-file-compressed -7.7% -6.4% -1.5% +10.0% ➖ noise
91 datafusion:vortex-compact -4.7% -3.1% -1.7% +10.0% ➖ noise
91 datafusion:vortex-file-compressed -9.0% -3.1% -6.1% +10.0% ➖ noise
91 duckdb:duckdb -1.6% -3.1% +1.6% +13.8% ➖ noise
91 duckdb:vortex-compact +1.0% -3.1% +4.3% +10.3% ➖ noise
91 duckdb:vortex-file-compressed -10.3% -3.1% -7.4% +10.5% ➖ noise
92 datafusion:vortex-compact -7.0% -3.9% -3.2% +10.0% ➖ noise
92 datafusion:vortex-file-compressed -14.0% -3.9% -10.5% +10.0% ✅ faster
92 duckdb:duckdb -10.6% -3.9% -7.0% +11.5% ➖ noise
92 duckdb:vortex-compact +2.4% -3.9% +6.5% +12.4% ➖ noise
92 duckdb:vortex-file-compressed -4.4% -3.9% -0.6% +10.0% ➖ noise
93 datafusion:vortex-compact -0.3% -5.7% +5.7% +10.0% ➖ noise
93 datafusion:vortex-file-compressed -6.3% -5.7% -0.7% +10.0% ➖ noise
93 duckdb:duckdb -9.5% -5.7% -4.0% +13.6% ➖ noise
93 duckdb:vortex-compact +5.3% -5.7% +11.7% +15.3% ➖ noise
93 duckdb:vortex-file-compressed -7.4% -5.7% -1.8% +13.5% ➖ noise
94 datafusion:vortex-compact -4.7% -4.5% -0.2% +10.0% ➖ noise
94 datafusion:vortex-file-compressed -4.6% -4.5% -0.1% +10.0% ➖ noise
94 duckdb:duckdb -2.6% -4.5% +1.9% +10.1% ➖ noise
94 duckdb:vortex-compact -0.4% -4.5% +4.3% +19.7% ➖ noise
94 duckdb:vortex-file-compressed -4.9% -4.5% -0.4% +10.0% ➖ noise
95 datafusion:vortex-compact -6.6% -4.7% -1.9% +10.3% ➖ noise
95 datafusion:vortex-file-compressed -5.6% -4.7% -0.9% +11.4% ➖ noise
95 duckdb:duckdb -3.6% -4.7% +1.2% +18.0% ➖ noise
95 duckdb:vortex-compact -0.0% -4.7% +4.9% +21.0% ➖ noise
95 duckdb:vortex-file-compressed -5.0% -4.7% -0.3% +18.1% ➖ noise
96 datafusion:vortex-compact +0.8% -9.2% +11.0% +10.7% 🚨 regression
96 datafusion:vortex-file-compressed -11.4% -9.2% -2.4% +10.0% ➖ noise
96 duckdb:duckdb -6.6% -9.2% +2.8% +16.3% ➖ noise
96 duckdb:vortex-compact +26.6% -9.2% +39.4% +13.8% 🚨 regression
96 duckdb:vortex-file-compressed +2.0% -9.2% +12.3% +10.0% 🚨 regression
97 datafusion:vortex-compact -3.8% -4.3% +0.5% +10.0% ➖ noise
97 datafusion:vortex-file-compressed -7.1% -4.3% -2.9% +10.0% ➖ noise
97 duckdb:duckdb -3.1% -4.3% +1.3% +13.0% ➖ noise
97 duckdb:vortex-compact +9.1% -4.3% +14.0% +10.0% 🚨 regression
97 duckdb:vortex-file-compressed -9.2% -4.3% -5.1% +10.0% ➖ noise
98 datafusion:vortex-compact -5.5% -0.6% -5.0% +10.0% ➖ noise
98 datafusion:vortex-file-compressed -2.5% -0.6% -1.9% +10.0% ➖ noise
98 duckdb:duckdb +2.1% -0.6% +2.6% +11.0% ➖ noise
98 duckdb:vortex-compact +5.3% -0.6% +5.9% +14.6% ➖ noise
98 duckdb:vortex-file-compressed -2.8% -0.6% -2.2% +11.9% ➖ noise
99 datafusion:vortex-compact -3.7% -9.5% +6.4% +10.0% ➖ noise
99 datafusion:vortex-file-compressed -3.8% -9.5% +6.3% +10.8% ➖ noise
99 duckdb:duckdb -2.6% -9.5% +7.6% +46.6% ➖ noise
99 duckdb:vortex-compact -2.4% -9.5% +7.8% +21.9% ➖ noise
99 duckdb:vortex-file-compressed -1.3% -9.5% +9.0% +21.9% ➖ noise

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: PolarSignals Profiling

Vortex (geomean): 1.000x ➖

datafusion / vortex-file-compressed (1.000x ➖, 0↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
polarsignals_q00/datafusion:vortex-file-compressed 125605558 126769075 0.99
polarsignals_q01/datafusion:vortex-file-compressed 378834435 384196203 0.99
polarsignals_q02/datafusion:vortex-file-compressed 23618196 21710641 1.09
polarsignals_q03/datafusion:vortex-file-compressed 385123277 381811517 1.01
polarsignals_q04/datafusion:vortex-file-compressed 11781269 11462836 1.03
polarsignals_q05/datafusion:vortex-file-compressed 16713103 15985502 1.05
polarsignals_q06/datafusion:vortex-file-compressed 18467064 19444649 0.95
polarsignals_q07/datafusion:vortex-file-compressed 13900911 13454951 1.03
polarsignals_q08/datafusion:vortex-file-compressed 462284118 472953433 0.98
polarsignals_q09/datafusion:vortex-file-compressed 9888716 10906569 0.91

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark TPC-H SF=10 on NVME failed! Check the workflow run for details.

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: FineWeb S3

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +5.0%
Vortex (geomean): 1.112x ➖
Parquet (geomean): 1.058x ➖
Shifts: Parquet (control) +5.8% · Median polish +2.1%

datafusion / vortex-file-compressed (1.033x ➖, 0↑ 1↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-file-compressed 🚨 52007411 35658250 1.46
fineweb_q01/datafusion:vortex-file-compressed 495888273 596422574 0.83
fineweb_q02/datafusion:vortex-file-compressed 466466597 457526613 1.02
fineweb_q03/datafusion:vortex-file-compressed 1507434854 1461922943 1.03
fineweb_q04/datafusion:vortex-file-compressed 1389404261 1409933680 0.99
fineweb_q05/datafusion:vortex-file-compressed 1387062597 1361605158 1.02
fineweb_q06/datafusion:vortex-file-compressed 1534380353 1513394035 1.01
fineweb_q07/datafusion:vortex-file-compressed 1341666828 1337679344 1.00
fineweb_q08/datafusion:vortex-file-compressed 560239385 546447804 1.03
datafusion / vortex-compact (1.138x ➖, 0↑ 2↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-compact 37279695 33786376 1.10
fineweb_q01/datafusion:vortex-compact 🚨 1206452967 737498335 1.64
fineweb_q02/datafusion:vortex-compact 🚨 1264498080 643204158 1.97
fineweb_q03/datafusion:vortex-compact 1884850235 1882309732 1.00
fineweb_q04/datafusion:vortex-compact 2233604888 2191346969 1.02
fineweb_q05/datafusion:vortex-compact 1679926434 1731507957 0.97
fineweb_q06/datafusion:vortex-compact 1395858515 1472349360 0.95
fineweb_q07/datafusion:vortex-compact 1285323577 1336667597 0.96
fineweb_q08/datafusion:vortex-compact 412965649 414839765 1.00
datafusion / parquet (1.088x ➖, 0↑ 2↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/datafusion:parquet 1178441258 1182056161 1.00
fineweb_q01/datafusion:parquet 🚨 2378996726 1823704872 1.30
fineweb_q02/datafusion:parquet 2060257277 1989709356 1.04
fineweb_q03/datafusion:parquet 2119078264 1809128291 1.17
fineweb_q04/datafusion:parquet 1960467923 2025810776 0.97
fineweb_q05/datafusion:parquet 1880969739 2040248897 0.92
fineweb_q06/datafusion:parquet 2028100487 2238883191 0.91
fineweb_q07/datafusion:parquet 2198805502 1883274155 1.17
fineweb_q08/datafusion:parquet 🚨 2611212279 1817911653 1.44
duckdb / vortex-file-compressed (1.214x ➖, 0↑ 2↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-file-compressed 🚨 119060357 80071316 1.49
fineweb_q01/duckdb:vortex-file-compressed 722956044 564770299 1.28
fineweb_q02/duckdb:vortex-file-compressed 🚨 841811687 446282112 1.89
fineweb_q03/duckdb:vortex-file-compressed 1891862017 1649561360 1.15
fineweb_q04/duckdb:vortex-file-compressed 1737665680 1611506740 1.08
fineweb_q05/duckdb:vortex-file-compressed 1625967224 1640751965 0.99
fineweb_q06/duckdb:vortex-file-compressed 1767723342 1650182647 1.07
fineweb_q07/duckdb:vortex-file-compressed 1601602603 1581155887 1.01
fineweb_q08/duckdb:vortex-file-compressed 751699562 626942001 1.20
duckdb / vortex-compact (1.072x ➖, 0↑ 1↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-compact 🚨 113428190 69074830 1.64
fineweb_q01/duckdb:vortex-compact 582137973 671899543 0.87
fineweb_q02/duckdb:vortex-compact 577610275 593290923 0.97
fineweb_q03/duckdb:vortex-compact 1707553864 1820683116 0.94
fineweb_q04/duckdb:vortex-compact 1942092356 1848705094 1.05
fineweb_q05/duckdb:vortex-compact 1652479552 1662709324 0.99
fineweb_q06/duckdb:vortex-compact 1642234883 1512107404 1.09
fineweb_q07/duckdb:vortex-compact 1555776137 1382438107 1.13
fineweb_q08/duckdb:vortex-compact 551610760 489748384 1.13
duckdb / parquet (1.030x ➖, 0↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
fineweb_q00/duckdb:parquet 1171934212 1096599458 1.07
fineweb_q01/duckdb:parquet 1870555248 1675216030 1.12
fineweb_q02/duckdb:parquet 1694697552 1659254671 1.02
fineweb_q03/duckdb:parquet 4224546305 4247521740 0.99
fineweb_q04/duckdb:parquet 1959882688 2002162647 0.98
fineweb_q05/duckdb:parquet 2986506631 2668569773 1.12
fineweb_q06/duckdb:parquet 3540374622 3376298669 1.05
fineweb_q07/duckdb:parquet 3969969611 4184178073 0.95
fineweb_q08/duckdb:parquet 1183957307 1201980118 0.99
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
0 datafusion:vortex-compact +10.3% +3.2% +6.9% +195.0% ➖ noise
0 datafusion:vortex-file-compressed +45.8% +3.2% +41.3% +213.8% ➖ noise
0 duckdb:vortex-compact +64.2% +3.2% +59.1% +37.2% 🚨 regression
0 duckdb:vortex-file-compressed +48.7% +3.2% +44.1% +47.1% ➖ noise
1 datafusion:vortex-compact +63.6% +20.7% +35.5% +77.2% ➖ noise
1 datafusion:vortex-file-compressed -16.9% +20.7% -31.1% +71.8% ➖ noise
1 duckdb:vortex-compact -13.4% +20.7% -28.2% +30.0% ✅ faster
1 duckdb:vortex-file-compressed +28.0% +20.7% +6.1% +30.0% ➖ noise
2 datafusion:vortex-compact +96.6% +2.8% +91.2% +31.8% 🚨 regression
2 datafusion:vortex-file-compressed +2.0% +2.8% -0.9% +41.8% ➖ noise
2 duckdb:vortex-compact -2.6% +2.8% -5.3% +30.0% ➖ noise
2 duckdb:vortex-file-compressed +88.6% +2.8% +83.4% +39.9% 🚨 regression
3 datafusion:vortex-compact +0.1% +7.9% -7.2% +30.0% ➖ noise
3 datafusion:vortex-file-compressed +3.1% +7.9% -4.5% +30.0% ➖ noise
3 duckdb:vortex-compact -6.2% +7.9% -13.1% +30.0% ➖ noise
3 duckdb:vortex-file-compressed +14.7% +7.9% +6.3% +30.0% ➖ noise
4 datafusion:vortex-compact +1.9% -2.7% +4.7% +30.0% ➖ noise
4 datafusion:vortex-file-compressed -1.5% -2.7% +1.2% +48.0% ➖ noise
4 duckdb:vortex-compact +5.1% -2.7% +7.9% +30.0% ➖ noise
4 duckdb:vortex-file-compressed +7.8% -2.7% +10.8% +30.0% ➖ noise
5 datafusion:vortex-compact -3.0% +1.6% -4.5% +30.0% ➖ noise
5 datafusion:vortex-file-compressed +1.9% +1.6% +0.3% +30.0% ➖ noise
5 duckdb:vortex-compact -0.6% +1.6% -2.2% +30.0% ➖ noise
5 duckdb:vortex-file-compressed -0.9% +1.6% -2.4% +30.0% ➖ noise
6 datafusion:vortex-compact -5.2% -2.5% -2.7% +30.0% ➖ noise
6 datafusion:vortex-file-compressed +1.4% -2.5% +4.0% +30.0% ➖ noise
6 duckdb:vortex-compact +8.6% -2.5% +11.4% +30.0% ➖ noise
6 duckdb:vortex-file-compressed +7.1% -2.5% +9.9% +30.0% ➖ noise
7 datafusion:vortex-compact -3.8% +5.3% -8.6% +30.0% ➖ noise
7 datafusion:vortex-file-compressed +0.3% +5.3% -4.7% +30.0% ➖ noise
7 duckdb:vortex-compact +12.5% +5.3% +6.9% +30.0% ➖ noise
7 duckdb:vortex-file-compressed +1.3% +5.3% -3.8% +30.0% ➖ noise
8 datafusion:vortex-compact -0.5% +18.9% -16.3% +30.0% ➖ noise
8 datafusion:vortex-file-compressed +2.5% +18.9% -13.8% +36.3% ➖ noise
8 duckdb:vortex-compact +12.6% +18.9% -5.3% +30.0% ➖ noise
8 duckdb:vortex-file-compressed +19.9% +18.9% +0.8% +43.8% ➖ noise

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark TPC-H SF=10 on S3 failed! Check the workflow run for details.

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨

Benchmark Clickbench on NVME failed! Check the workflow run for details.

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Statistical and Population Genetics

Verdict: No clear signal (low confidence)
Attributed Vortex impact: -0.5%
Vortex (geomean): 0.980x ➖
Parquet (geomean): 0.985x ➖
Shifts: Parquet (control) -1.5% · Median polish +0.1%

duckdb / vortex-file-compressed (0.959x ➖, 2↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
statpopgen_q00/duckdb:vortex-file-compressed 10883061 11051176 0.98
statpopgen_q01/duckdb:vortex-file-compressed 19625763 19002866 1.03
statpopgen_q02/duckdb:vortex-file-compressed 1280038207 1315743190 0.97
statpopgen_q03/duckdb:vortex-file-compressed 3086724443 3042274574 1.01
statpopgen_q04/duckdb:vortex-file-compressed 3057298239 3045692437 1.00
statpopgen_q05/duckdb:vortex-file-compressed 1396536814 1411001301 0.99
statpopgen_q06/duckdb:vortex-file-compressed 2067429255 2064239737 1.00
statpopgen_q07/duckdb:vortex-file-compressed 🚀 197604048 262113913 0.75
statpopgen_q08/duckdb:vortex-file-compressed 🚀 222193572 273741246 0.81
statpopgen_q09/duckdb:vortex-file-compressed 2878602241 2815549204 1.02
statpopgen_q10/duckdb:vortex-file-compressed 4671636382 4607942066 1.01
duckdb / vortex-compact (1.001x ➖, 0↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
statpopgen_q00/duckdb:vortex-compact 10867173 10645010 1.02
statpopgen_q01/duckdb:vortex-compact 170277534 167249520 1.02
statpopgen_q02/duckdb:vortex-compact 1827654505 1806274787 1.01
statpopgen_q03/duckdb:vortex-compact 3492413473 3470444367 1.01
statpopgen_q04/duckdb:vortex-compact 3513136400 3530004876 1.00
statpopgen_q05/duckdb:vortex-compact 1818159320 1816330303 1.00
statpopgen_q06/duckdb:vortex-compact 2676160088 2655352478 1.01
statpopgen_q07/duckdb:vortex-compact 952307007 977023070 0.97
statpopgen_q08/duckdb:vortex-compact 989907340 1008662843 0.98
statpopgen_q09/duckdb:vortex-compact 3299624352 3312613885 1.00
statpopgen_q10/duckdb:vortex-compact 5414970305 5411216863 1.00
duckdb / parquet (0.985x ➖, 0↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
statpopgen_q00/duckdb:parquet 317643475 313594242 1.01
statpopgen_q01/duckdb:parquet 404631484 393394528 1.03
statpopgen_q02/duckdb:parquet 981924607 989401173 0.99
statpopgen_q03/duckdb:parquet 1430412646 1529335521 0.94
statpopgen_q04/duckdb:parquet 1458520349 1508647388 0.97
statpopgen_q05/duckdb:parquet 963925087 1013042632 0.95
statpopgen_q06/duckdb:parquet 1460198758 1484494803 0.98
statpopgen_q07/duckdb:parquet 1887471666 1874859213 1.01
statpopgen_q08/duckdb:parquet 1895170518 1879519515 1.01
statpopgen_q09/duckdb:parquet 1314063496 1351620216 0.97
statpopgen_q10/duckdb:parquet 2567982268 2630860106 0.98
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
0 duckdb:vortex-compact +2.1% +1.3% +0.8% +10.0% ➖ noise
0 duckdb:vortex-file-compressed -1.5% +1.3% -2.8% +10.0% ➖ noise
1 duckdb:vortex-compact +1.8% +2.9% -1.0% +24.8% ➖ noise
1 duckdb:vortex-file-compressed +3.3% +2.9% +0.4% +207.5% ➖ noise
2 duckdb:vortex-compact +1.2% -0.8% +2.0% +10.0% ➖ noise
2 duckdb:vortex-file-compressed -2.7% -0.8% -2.0% +10.0% ➖ noise
3 duckdb:vortex-compact +0.6% -6.5% +7.6% +10.0% ➖ noise
3 duckdb:vortex-file-compressed +1.5% -6.5% +8.5% +10.0% ➖ noise
4 duckdb:vortex-compact -0.5% -3.3% +2.9% +10.0% ➖ noise
4 duckdb:vortex-file-compressed +0.4% -3.3% +3.8% +10.0% ➖ noise
5 duckdb:vortex-compact +0.1% -4.8% +5.2% +10.0% ➖ noise
5 duckdb:vortex-file-compressed -1.0% -4.8% +4.0% +10.0% ➖ noise
6 duckdb:vortex-compact +0.8% -1.6% +2.5% +10.0% ➖ noise
6 duckdb:vortex-file-compressed +0.2% -1.6% +1.8% +10.0% ➖ noise
7 duckdb:vortex-compact -2.5% +0.7% -3.2% +10.0% ➖ noise
7 duckdb:vortex-file-compressed -24.6% +0.7% -25.1% +10.0% ✅ faster
8 duckdb:vortex-compact -1.9% +0.8% -2.7% +10.0% ➖ noise
8 duckdb:vortex-file-compressed -18.8% +0.8% -19.5% +10.0% ✅ faster
9 duckdb:vortex-compact -0.4% -2.8% +2.5% +10.0% ➖ noise
9 duckdb:vortex-file-compressed +2.2% -2.8% +5.2% +10.0% ➖ noise
10 duckdb:vortex-compact +0.1% -2.4% +2.5% +10.0% ➖ noise
10 duckdb:vortex-file-compressed +1.4% -2.4% +3.9% +10.0% ➖ noise

@joseph-isaacs joseph-isaacs added the action/benchmark Trigger full benchmarks to run on this PR label Mar 13, 2026
@github-actions github-actions Bot removed the action/benchmark Trigger full benchmarks to run on this PR label Mar 13, 2026
@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Random Access

Vortex (geomean): 0.804x ✅
Parquet (geomean): 0.958x ➖

unknown / unknown (0.887x ✅, 29↑ 0↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
random-access/feature-vectors/correlated/lance-tokio-local-disk 2340269 2563914 0.91
random-access/feature-vectors/correlated/lance-tokio-local-disk-footer 1325631 1372875 0.97
random-access/feature-vectors/correlated/parquet-tokio-local-disk 11758614824 11851348406 0.99
random-access/feature-vectors/correlated/parquet-tokio-local-disk-footer 11208140571 11460935157 0.98
random-access/feature-vectors/correlated/vortex-tokio-local-disk 8870023 8458983 1.05
random-access/feature-vectors/correlated/vortex-tokio-local-disk-footer 🚀 6550834 7318449 0.90
random-access/feature-vectors/uniform/lance-tokio-local-disk 1482323 1569527 0.94
random-access/feature-vectors/uniform/lance-tokio-local-disk-footer 2426585 2666333 0.91
random-access/feature-vectors/uniform/parquet-tokio-local-disk 11221025011 11387731168 0.99
random-access/feature-vectors/uniform/parquet-tokio-local-disk-footer 11163841866 11407495058 0.98
random-access/feature-vectors/uniform/vortex-tokio-local-disk 🚀 12761666 14987760 0.85
random-access/feature-vectors/uniform/vortex-tokio-local-disk-footer 🚀 12661294 14573263 0.87
random-access/lance-tokio-local-disk 750511 756499 0.99
random-access/lance-tokio-local-disk-footer 1358830 1422369 0.96
random-access/nested-lists/correlated/lance-tokio-local-disk 🚀 267961 299625 0.89
random-access/nested-lists/correlated/lance-tokio-local-disk-footer 🚀 716554 875678 0.82
random-access/nested-lists/correlated/parquet-tokio-local-disk 137275934 144594425 0.95
random-access/nested-lists/correlated/parquet-tokio-local-disk-footer 137474827 145265433 0.95
random-access/nested-lists/correlated/vortex-tokio-local-disk 🚀 702957 814527 0.86
random-access/nested-lists/correlated/vortex-tokio-local-disk-footer 🚀 709422 1085748 0.65
random-access/nested-lists/uniform/lance-tokio-local-disk 🚀 1165116 1337288 0.87
random-access/nested-lists/uniform/lance-tokio-local-disk-footer 🚀 1657914 1956669 0.85
random-access/nested-lists/uniform/parquet-tokio-local-disk 141335309 143827444 0.98
random-access/nested-lists/uniform/parquet-tokio-local-disk-footer 141049885 145639347 0.97
random-access/nested-lists/uniform/vortex-tokio-local-disk 🚀 2398231 2804127 0.86
random-access/nested-lists/uniform/vortex-tokio-local-disk-footer 🚀 2489849 3111695 0.80
random-access/nested-structs/correlated/lance-tokio-local-disk 🚀 432348 528125 0.82
random-access/nested-structs/correlated/lance-tokio-local-disk-footer 🚀 643559 823888 0.78
random-access/nested-structs/correlated/parquet-tokio-local-disk 🚀 27266364 31276636 0.87
random-access/nested-structs/correlated/parquet-tokio-local-disk-footer 🚀 26770098 31574176 0.85
random-access/nested-structs/correlated/vortex-tokio-local-disk 🚀 915908 1073870 0.85
random-access/nested-structs/correlated/vortex-tokio-local-disk-footer 🚀 919822 1079501 0.85
random-access/nested-structs/uniform/lance-tokio-local-disk 🚀 2798512 3350518 0.84
random-access/nested-structs/uniform/lance-tokio-local-disk-footer 🚀 3038578 3663229 0.83
random-access/nested-structs/uniform/parquet-tokio-local-disk 🚀 28065983 31416559 0.89
random-access/nested-structs/uniform/parquet-tokio-local-disk-footer 🚀 26896895 31981445 0.84
random-access/nested-structs/uniform/vortex-tokio-local-disk 🚀 1811964 2193020 0.83
random-access/nested-structs/uniform/vortex-tokio-local-disk-footer 🚀 1781494 2160283 0.82
random-access/parquet-tokio-local-disk 188838488 186108274 1.01
random-access/parquet-tokio-local-disk-footer 186448733 184783547 1.01
random-access/taxi/correlated/lance-tokio-local-disk 971306 978812 0.99
random-access/taxi/correlated/lance-tokio-local-disk-footer 1674543 1737587 0.96
random-access/taxi/correlated/parquet-tokio-local-disk 277247324 275670099 1.01
random-access/taxi/correlated/parquet-tokio-local-disk-footer 278619666 277253474 1.00
random-access/taxi/correlated/vortex-tokio-local-disk 🚀 1753946 2536825 0.69
random-access/taxi/correlated/vortex-tokio-local-disk-footer 🚀 1769643 2592192 0.68
random-access/taxi/uniform/lance-tokio-local-disk 8628721 8538294 1.01
random-access/taxi/uniform/lance-tokio-local-disk-footer 9337838 9091207 1.03
random-access/taxi/uniform/parquet-tokio-local-disk 288983630 292400758 0.99
random-access/taxi/uniform/parquet-tokio-local-disk-footer 296802312 291629822 1.02
random-access/taxi/uniform/vortex-tokio-local-disk 🚀 4495076 5779553 0.78
random-access/taxi/uniform/vortex-tokio-local-disk-footer 🚀 4577487 5585447 0.82
random-access/vortex-tokio-local-disk 🚀 1303048 1871859 0.70
random-access/vortex-tokio-local-disk-footer 🚀 1296142 1817508 0.71

@github-actions

github-actions Bot commented Mar 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Compression

Vortex (geomean): 0.958x ➖
Parquet (geomean): 0.966x ➖

unknown / unknown (0.955x ➖, 9↑ 14↓)
name PR 6076d52 (ns) base 59b0083 (ns) ratio (PR/base)
compress time/Arade 1207802655 1283195985 0.94
compress time/Bimbo 7561022476 7705082976 0.98
compress time/CMSprovider 3113130904 3075370230 1.01
compress time/Euro2016 454044082 454512544 1.00
compress time/Food 413819331 415055725 1.00
compress time/HashTags 🚨 963244144 847219892 1.14
compress time/TPC-H l_comment canonical 1385582727 1356918476 1.02
compress time/TPC-H l_comment chunked 1368520344 1393580786 0.98
compress time/taxi 776352859 739887590 1.05
compress time/wide table cols=100 chunks=1 rows=1000 🚨 15517143 11872850 1.31
compress time/wide table cols=100 chunks=50 rows=1000 🚨 16681961 12332984 1.35
compress time/wide table cols=1000 chunks=1 rows=1000 🚨 162446955 129235518 1.26
compress time/wide table cols=1000 chunks=50 rows=1000 🚨 156760008 132051638 1.19
compress time/wide table cols=10000 chunks=1 rows=1000 🚨 1569043293 1315283605 1.19
compress time/wide table cols=10000 chunks=50 rows=1000 🚨 1549677895 1278878317 1.21
decompress time/Arade 31791810 33170773 0.96
decompress time/Bimbo 103340266 105983143 0.98
decompress time/CMSprovider 88246586 88334798 1.00
decompress time/Euro2016 23025044 23417886 0.98
decompress time/Food 8864944 8805287 1.01
decompress time/HashTags 87374343 90883189 0.96
decompress time/TPC-H l_comment canonical 43927756 42661521 1.03
decompress time/TPC-H l_comment chunked 43247356 44161497 0.98
decompress time/taxi 19235941 18218693 1.06
decompress time/wide table cols=100 chunks=1 rows=1000 2678231 2784597 0.96
decompress time/wide table cols=100 chunks=50 rows=1000 2734950 2829604 0.97
decompress time/wide table cols=1000 chunks=1 rows=1000 🚀 23171091 35424723 0.65
decompress time/wide table cols=1000 chunks=50 rows=1000 🚀 23498236 36407750 0.65
decompress time/wide table cols=10000 chunks=1 rows=1000 🚀 247584299 1512773127 0.16
decompress time/wide table cols=10000 chunks=50 rows=1000 🚀 242942122 1495936087 0.16
parquet size/Arade 258014282 258014282 1.00
parquet size/Bimbo 384517292 384517292 1.00
parquet size/CMSprovider 376885545 376885545 1.00
parquet size/Euro2016 122975499 122975499 1.00
parquet size/Food 35699500 35699500 1.00
parquet size/HashTags 133510943 133510943 1.00
parquet size/TPC-H l_comment canonical 158358238 158358238 1.00
parquet size/TPC-H l_comment chunked 158358238 158358238 1.00
parquet size/taxi 55283635 55283635 1.00
parquet size/wide table cols=100 chunks=1 rows=1000 932404 932404 1.00
parquet size/wide table cols=100 chunks=50 rows=1000 932404 932404 1.00
parquet size/wide table cols=1000 chunks=1 rows=1000 9324004 9324004 1.00
parquet size/wide table cols=1000 chunks=50 rows=1000 9324004 9324004 1.00
parquet size/wide table cols=10000 chunks=1 rows=1000 93240004 93240004 1.00
parquet size/wide table cols=10000 chunks=50 rows=1000 93240004 93240004 1.00
parquet_rs-zstd compress time/Arade 3083976262 3114467969 0.99
parquet_rs-zstd compress time/Bimbo 17783895387 18101197733 0.98
parquet_rs-zstd compress time/CMSprovider 8084327597 7929986755 1.02
parquet_rs-zstd compress time/Euro2016 1368375316 1396632704 0.98
parquet_rs-zstd compress time/Food 962287012 967961436 0.99
parquet_rs-zstd compress time/HashTags 2531176690 2504329372 1.01
parquet_rs-zstd compress time/TPC-H l_comment canonical 3124868120 3148948856 0.99
parquet_rs-zstd compress time/TPC-H l_comment chunked 3131391526 3125643774 1.00
parquet_rs-zstd compress time/taxi 1534155199 1555850002 0.99
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 6850337 6884455 1.00
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 7220548 7597865 0.95
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 92691431 92480772 1.00
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 88487109 97270648 0.91
parquet_rs-zstd compress time/wide table cols=10000 chunks=1 rows=1000 969053463 981142095 0.99
parquet_rs-zstd compress time/wide table cols=10000 chunks=50 rows=1000 928841979 988857590 0.94
parquet_rs-zstd decompress time/Arade 652179759 659195956 0.99
parquet_rs-zstd decompress time/Bimbo 1820865431 1790135946 1.02
parquet_rs-zstd decompress time/CMSprovider 1955414297 1967006014 0.99
parquet_rs-zstd decompress time/Euro2016 389025640 396318991 0.98
parquet_rs-zstd decompress time/Food 224344017 225761253 0.99
parquet_rs-zstd decompress time/HashTags 640695161 634728438 1.01
parquet_rs-zstd decompress time/TPC-H l_comment canonical 571247292 571281598 1.00
parquet_rs-zstd decompress time/TPC-H l_comment chunked 572725369 580156407 0.99
parquet_rs-zstd decompress time/taxi 252468421 247274544 1.02
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 2712556 2758623 0.98
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 2789215 2776137 1.00
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 33078620 33614472 0.98
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 🚀 31730619 36199409 0.88
parquet_rs-zstd decompress time/wide table cols=10000 chunks=1 rows=1000 371039989 390364901 0.95
parquet_rs-zstd decompress time/wide table cols=10000 chunks=50 rows=1000 374099756 387415336 0.97
vortex-file-compressed size/Arade 143435604 143600332 1.00
vortex-file-compressed size/Bimbo 526882596 510400068 1.03
vortex-file-compressed size/CMSprovider 448399308 448304012 1.00
vortex-file-compressed size/Euro2016 164915236 164458756 1.00
vortex-file-compressed size/Food 44208064 44206912 1.00
vortex-file-compressed size/HashTags 195421908 195864476 1.00
vortex-file-compressed size/TPC-H l_comment canonical 179481808 179481896 1.00
vortex-file-compressed size/TPC-H l_comment chunked 179481808 179481896 1.00
vortex-file-compressed size/taxi 51543292 51040172 1.01
vortex-file-compressed size/wide table cols=100 chunks=1 rows=1000 884384 884416 1.00
vortex-file-compressed size/wide table cols=100 chunks=50 rows=1000 884384 884416 1.00
vortex-file-compressed size/wide table cols=1000 chunks=1 rows=1000 8829584 8829616 1.00
vortex-file-compressed size/wide table cols=1000 chunks=50 rows=1000 8829584 8829616 1.00
vortex-file-compressed size/wide table cols=10000 chunks=1 rows=1000 88317584 88317616 1.00
vortex-file-compressed size/wide table cols=10000 chunks=50 rows=1000 88317584 88317616 1.00
vortex:parquet-zstd ratio compress time/Arade 0 0 0.95
vortex:parquet-zstd ratio compress time/Bimbo 0 0 1.00
vortex:parquet-zstd ratio compress time/CMSprovider 0 0 0.99
vortex:parquet-zstd ratio compress time/Euro2016 0 0 1.02
vortex:parquet-zstd ratio compress time/Food 0 0 1.00
vortex:parquet-zstd ratio compress time/HashTags 🚨 0 0 1.12
vortex:parquet-zstd ratio compress time/TPC-H l_comment canonical 0 0 1.03
vortex:parquet-zstd ratio compress time/TPC-H l_comment chunked 0 0 0.98
vortex:parquet-zstd ratio compress time/taxi 0 0 1.06
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=1 rows=1000 🚨 2 1 1.31
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=50 rows=1000 🚨 2 1 1.42
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=1 rows=1000 🚨 1 1 1.25
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=50 rows=1000 🚨 1 1 1.30
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=1 rows=1000 🚨 1 1 1.21
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=50 rows=1000 🚨 1 1 1.29
vortex:parquet-zstd ratio decompress time/Arade 0 0 0.97
vortex:parquet-zstd ratio decompress time/Bimbo 0 0 0.96
vortex:parquet-zstd ratio decompress time/CMSprovider 0 0 1.00
vortex:parquet-zstd ratio decompress time/Euro2016 0 0 1.00
vortex:parquet-zstd ratio decompress time/Food 0 0 1.01
vortex:parquet-zstd ratio decompress time/HashTags 0 0 0.95
vortex:parquet-zstd ratio decompress time/TPC-H l_comment canonical 0 0 1.03
vortex:parquet-zstd ratio decompress time/TPC-H l_comment chunked 0 0 0.99
vortex:parquet-zstd ratio decompress time/taxi 0 0 1.03
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=1 rows=1000 0 1 0.98
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=50 rows=1000 0 1 0.96
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=1 rows=1000 🚀 0 1 0.66
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=50 rows=1000 🚀 0 1 0.74
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=1 rows=1000 🚀 0 3 0.17
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=50 rows=1000 🚀 0 3 0.17
vortex:parquet-zstd size/Arade 0 0 1.00
vortex:parquet-zstd size/Bimbo 1 1 1.03
vortex:parquet-zstd size/CMSprovider 1 1 1.00
vortex:parquet-zstd size/Euro2016 1 1 1.00
vortex:parquet-zstd size/Food 1 1 1.00
vortex:parquet-zstd size/HashTags 1 1 1.00
vortex:parquet-zstd size/TPC-H l_comment canonical 1 1 1.00
vortex:parquet-zstd size/TPC-H l_comment chunked 1 1 1.00
vortex:parquet-zstd size/taxi 0 0 1.01
vortex:parquet-zstd size/wide table cols=100 chunks=1 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=100 chunks=50 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=1000 chunks=1 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=1000 chunks=50 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=10000 chunks=1 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=10000 chunks=50 rows=1000 0 0 1.00

@joseph-isaacs
validate all
04ddf0f 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the action/benchmark Trigger full benchmarks to run on this PR label Mar 13, 2026
@joseph-isaacs joseph-isaacs removed the action/benchmark Trigger full benchmarks to run on this PR label Mar 13, 2026
@joseph-isaacs joseph-isaacs changed the title validate all validate all benchmark runs Mar 19, 2026
@joseph-isaacs joseph-isaacs added the action/benchmark Trigger full benchmarks to run on this PR label Mar 19, 2026
@joseph-isaacs
Merge remote-tracking branch 'origin/develop' into ji/validate-sql-bench
4bb214d 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

# Conflicts:
#	Cargo.lock
@joseph-isaacs
fixup
e2e379c 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs marked this pull request as ready for review March 30, 2026 13:40
@joseph-isaacs
fixup
ef1dadf 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added action/benchmark Trigger full benchmarks to run on this PR and removed action/benchmark Trigger full benchmarks to run on this PR labels Mar 30, 2026
@github-actions github-actions Bot removed the action/benchmark Trigger full benchmarks to run on this PR label Mar 30, 2026
@joseph-isaacs
fixup
6542c62 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the action/benchmark Trigger full benchmarks to run on this PR label Mar 30, 2026
@github-actions github-actions Bot removed the action/benchmark Trigger full benchmarks to run on this PR label Mar 30, 2026
@joseph-isaacs
fixup
323fc00 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs
fixup
6a58b21 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the action/benchmark Trigger full benchmarks to run on this PR label Apr 1, 2026
@github-actions github-actions Bot removed the action/benchmark Trigger full benchmarks to run on this PR label Apr 1, 2026
@joseph-isaacs
fixup
6076d52 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the action/benchmark Trigger full benchmarks to run on this PR label Apr 2, 2026
@github-actions github-actions Bot removed the action/benchmark Trigger full benchmarks to run on this PR label Apr 2, 2026
AdamGS
AdamGS reviewed Apr 15, 2026
View reviewed changes
Comment thread vortex-bench/Cargo.toml
reqwest = { workspace = true, features = ["stream"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
sqllogictest = "0.28"

@AdamGS AdamGS Apr 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IDK how this works, we're on 0.29

AdamGS
AdamGS reviewed Apr 15, 2026
View reviewed changes
Comment thread vortex-bench/tpch/slt/tpch_datafusion.slt
@@ -0,0 +1,24 @@
# TPC-H benchmark queries — datafusion expected results.

include queries/bench_01_datafusion.slt

@AdamGS AdamGS Apr 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slt should support queries/*.slt

AdamGS
AdamGS reviewed Apr 15, 2026
View reviewed changes
Comment thread vortex-bench/tpch/slt/tpch_duckdb.slt
@@ -0,0 +1,24 @@
# TPC-H benchmark queries — duckdb expected results.

include queries/bench_01_duckdb.slt

@AdamGS AdamGS Apr 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slt should support queries/*.slt

AdamGS
AdamGS reviewed Apr 15, 2026
View reviewed changes
Comment thread vortex-bench/clickbench/slt/clickbench_duckdb.slt
@@ -0,0 +1,36 @@
# ClickBench benchmark queries — duckdb expected results.

include queries/bench_00.slt

@AdamGS AdamGS Apr 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slt should support queries/*.slt

AdamGS
AdamGS reviewed Apr 15, 2026
View reviewed changes
Comment thread vortex-bench/clickbench/slt/clickbench_datafusion.slt
@@ -0,0 +1,37 @@
# ClickBench benchmark queries — datafusion expected results.

include queries/bench_00.slt

@AdamGS AdamGS Apr 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slt should support queries/*.slt

@AdamGS

AdamGS commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

SLT has a onlyif syntax that should allow you to filter things at a statement level instead of splitting things into DF files and DuckDB files

AdamGS
AdamGS approved these changes Apr 15, 2026
View reviewed changes

@AdamGS AdamGS left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM overall, just some relatively minor comments

@joseph-isaacs joseph-isaacs added action/benchmark Trigger full benchmarks to run on this PR and removed action/benchmark Trigger full benchmarks to run on this PR labels Apr 15, 2026
@joseph-isaacs
Merge remote-tracking branch 'origin/develop' into ji/validate-sql-bench
94104ed 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

# Conflicts:
#	Cargo.lock
#	benchmarks/datafusion-bench/src/main.rs
@joseph-isaacs
wip
959e6f0 
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@github-actions

github-actions Bot commented May 1, 2026

Copy link
Copy Markdown
Contributor

This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days

@github-actions github-actions Bot added the stale This PR is stale and will be auto-closed soon label May 1, 2026
@github-actions

github-actions Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor

This PR was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions Bot closed this May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AdamGS AdamGS AdamGS approved these changes

Assignees

No one assigned

Labels

changelog/chore A trivial change stale This PR is stale and will be auto-closed soon

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@joseph-isaacs @AdamGS