Skip to content

feat: Reuse intermediate materialization results #686

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat: Reuse intermediate materialization results #686

wants to merge 4 commits into from
+96 −9

Conversation

@krlmlr
Copy link
Member

@krlmlr krlmlr commented Apr 10, 2025 •
edited
Loading

Closes #442.

@krlmlr krlmlr enabled auto-merge (squash) April 10, 2025 05:42
@krlmlr krlmlr disabled auto-merge April 10, 2025 05:43
@krlmlr
Copy link
Member Author

krlmlr commented Apr 10, 2025

Let's see what the benchmark results are here.

@github-actions github-actions bot mentioned this pull request Apr 10, 2025
Closed
@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 21a09b3 is merged into main:

  • ❗🐌001_tpch_01: 24.7ms -> 26.1ms [+0.73%, +9.95%]
  • ❗🐌001_tpch_02: 69.7ms -> 82.5ms [+17.44%, +19.3%]
  • ❗🐌001_tpch_03: 35.7ms -> 40.9ms [+12.21%, +17.07%]
  • ❗🐌001_tpch_04: 23.6ms -> 25.8ms [+6.04%, +12.97%]
  • ❗🐌001_tpch_05: 62.8ms -> 70.2ms [+9.21%, +14.45%]
  • ❗🐌001_tpch_06: 11.9ms -> 12.7ms [+3.88%, +8.16%]
  • ❗🐌001_tpch_07: 75.5ms -> 85.9ms [+12.76%, +14.64%]
  • ❗🐌001_tpch_08: 96.5ms -> 113ms [+16.15%, +17.67%]
  • ❗🐌001_tpch_09: 72.6ms -> 82.8ms [+12.13%, +15.98%]
  • ❗🐌001_tpch_10: 49.7ms -> 56.5ms [+12.63%, +14.88%]
  • ❗🐌001_tpch_11: 34.5ms -> 41.2ms [+17.35%, +21.76%]
  • ❗🐌001_tpch_12: 22.5ms -> 23.8ms [+3.2%, +8.46%]
  • ❗🐌001_tpch_13: 18.4ms -> 20.4ms [+10.01%, +11.64%]
  • ❗🐌001_tpch_14: 17.6ms -> 18.4ms [+3.29%, +5.11%]
  • ❗🐌001_tpch_15: 32.1ms -> 36.7ms [+13.4%, +15.34%]
  • ❗🐌001_tpch_16: 31.3ms -> 35.4ms [+10.48%, +15.61%]
  • ❗🐌001_tpch_17: 27.9ms -> 31.8ms [+12.28%, +15.71%]
  • ❗🐌001_tpch_18: 23.1ms -> 25.6ms [+8.64%, +12.48%]
  • ❗🐌001_tpch_19: 46.3ms -> 48ms [+1.59%, +5.75%]
  • ❗🐌001_tpch_20: 47.1ms -> 58.8ms [+23.3%, +26.55%]
  • ❗🐌001_tpch_21: 84ms -> 99.5ms [+17%, +19.77%]
  • ❗🐌001_tpch_22: 48.9ms -> 55.8ms [+12.21%, +15.83%]
  • ✔️010_tpch_01: 90.3ms -> 89.2ms [-6.37%, +3.98%]
  • ❗🐌010_tpch_02: 123ms -> 137ms [+8.52%, +14.57%]
  • ❗🐌010_tpch_03: 62.5ms -> 69.7ms [+5.57%, +17.5%]
  • ❗🐌010_tpch_04: 46ms -> 48.3ms [+2.93%, +7.43%]
  • ❗🐌010_tpch_05: 139ms -> 147ms [+5.24%, +6.83%]
  • ✔️010_tpch_06: 34.2ms -> 31.9ms [-17.12%, +3.94%]
  • ❗🐌010_tpch_07: 149ms -> 162ms [+5.91%, +12.28%]
  • ❗🐌010_tpch_08: 118ms -> 134ms [+12.47%, +14.11%]
  • ❗🐌010_tpch_09: 162ms -> 173ms [+6.01%, +7.42%]
  • ❗🐌010_tpch_10: 127ms -> 133ms [+3.86%, +5.87%]
  • ❗🐌010_tpch_11: 44.4ms -> 50.6ms [+11.77%, +15.86%]
  • ✔️010_tpch_12: 55ms -> 56ms [-0.02%, +3.7%]
  • ❗🐌010_tpch_13: 52ms -> 54ms [+1.21%, +6.25%]
  • ✔️010_tpch_14: 36.8ms -> 38.3ms [-1.24%, +9.65%]
  • ✔️010_tpch_15: 59.6ms -> 62.9ms [-0.45%, +11.36%]
  • ❗🐌010_tpch_16: 47.2ms -> 51.4ms [+4.3%, +13.31%]
  • ❗🐌010_tpch_17: 60.8ms -> 66ms [+4.51%, +12.53%]
  • ✔️010_tpch_18: 59.4ms -> 62.7ms [-0.06%, +10.95%]
  • ✔️010_tpch_19: 147ms -> 152ms [-0.51%, +8.02%]
  • ❗🐌010_tpch_20: 75.4ms -> 87.8ms [+10.58%, +22.33%]
  • ❗🐌010_tpch_21: 331ms -> 358ms [+4.2%, +11.98%]
  • ❗🐌010_tpch_22: 62.3ms -> 69.9ms [+9.49%, +14.96%]
  • ✔️100_tpch_01: 355ms -> 353ms [-22.05%, +21.16%]
  • ✔️100_tpch_02: 137ms -> 145ms [-6.36%, +17.59%]
  • ✔️100_tpch_03: 168ms -> 179ms [-2.64%, +15.4%]
  • ✔️100_tpch_04: 156ms -> 157ms [-10.19%, +11.9%]
  • ✔️100_tpch_05: 263ms -> 262ms [-12.47%, +11.38%]
  • ✔️100_tpch_06: 105ms -> 98.6ms [-21.95%, +10.09%]
  • ✔️100_tpch_07: 229ms -> 252ms [-0.17%, +19.85%]
  • ✔️100_tpch_08: 256ms -> 281ms [-5.16%, +24.26%]
  • ✔️100_tpch_09: 320ms -> 313ms [-13.21%, +9.1%]
  • ✔️100_tpch_10: 212ms -> 211ms [-6.66%, +5.23%]
  • ✔️100_tpch_11: 92.9ms -> 93.4ms [-30.21%, +31.38%]
  • ✔️100_tpch_12: 194ms -> 194ms [-14.2%, +14.61%]
  • ✔️100_tpch_13: 342ms -> 334ms [-15.03%, +9.93%]
  • ✔️100_tpch_14: 117ms -> 108ms [-20.31%, +4.91%]
  • ✔️100_tpch_15: 210ms -> 213ms [-3.87%, +6.94%]
  • ✔️100_tpch_16: 131ms -> 136ms [-2.08%, +8.8%]
  • ✔️100_tpch_17: 177ms -> 177ms [-6.2%, +5.86%]
  • ❗🐌100_tpch_18: 204ms -> 212ms [+0.39%, +7.12%]
  • ✔️100_tpch_19: 259ms -> 254ms [-14.62%, +10.66%]
  • ✔️100_tpch_20: 183ms -> 190ms [-17.55%, +25.16%]
  • ❗🐌100_tpch_21: 1.41s -> 1.51s [+3.66%, +11.16%]
  • ✔️100_tpch_22: 158ms -> 158ms [-2.69%, +2%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 27992b1 is merged into main:

  • ❗🐌001_tpch_01: 23.6ms -> 24.9ms [+1.11%, +10.15%]
  • ❗🐌001_tpch_02: 68.3ms -> 80.8ms [+17.22%, +19.2%]
  • ❗🐌001_tpch_03: 34.7ms -> 39.6ms [+12.67%, +15.22%]
  • ❗🐌001_tpch_04: 22.3ms -> 25.1ms [+11.77%, +13.26%]
  • ❗🐌001_tpch_05: 59.2ms -> 67ms [+12.3%, +13.95%]
  • ❗🐌001_tpch_06: 11.9ms -> 12.5ms [+2.44%, +7.05%]
  • ❗🐌001_tpch_07: 75.9ms -> 85.1ms [+10.12%, +14.25%]
  • ❗🐌001_tpch_08: 96.1ms -> 112ms [+15.35%, +17.47%]
  • ❗🐌001_tpch_09: 71.6ms -> 81.3ms [+12.9%, +14.17%]
  • ❗🐌001_tpch_10: 50.1ms -> 56.1ms [+10.37%, +13.29%]
  • ❗🐌001_tpch_11: 33.8ms -> 39.9ms [+17.56%, +18.79%]
  • ❗🐌001_tpch_12: 21.6ms -> 23.1ms [+5.66%, +8.02%]
  • ❗🐌001_tpch_13: 18.2ms -> 20ms [+8.62%, +11.07%]
  • ❗🐌001_tpch_14: 17.4ms -> 18ms [+2.42%, +4.33%]
  • ❗🐌001_tpch_15: 31.5ms -> 36ms [+12.59%, +16.02%]
  • ❗🐌001_tpch_16: 30.5ms -> 34.9ms [+13.1%, +15.47%]
  • ❗🐌001_tpch_17: 27.1ms -> 31.1ms [+13.42%, +15.86%]
  • ❗🐌001_tpch_18: 22.8ms -> 25.1ms [+8.44%, +12.2%]
  • ❗🐌001_tpch_19: 46.1ms -> 47.2ms [+1.31%, +3.59%]
  • ❗🐌001_tpch_20: 46.9ms -> 58.6ms [+24.08%, +25.82%]
  • ❗🐌001_tpch_21: 84.4ms -> 101ms [+17.91%, +21.77%]
  • ❗🐌001_tpch_22: 48.7ms -> 55.5ms [+12.35%, +15.31%]
  • ✔️010_tpch_01: 88.3ms -> 92.1ms [-1.22%, +9.98%]
  • ❗🐌010_tpch_02: 124ms -> 138ms [+9.26%, +13.69%]
  • ❗🐌010_tpch_03: 63.1ms -> 68.4ms [+7.24%, +9.43%]
  • ❗🐌010_tpch_04: 46.1ms -> 48.8ms [+1.47%, +9.95%]
  • ❗🐌010_tpch_05: 138ms -> 146ms [+4.65%, +6.94%]
  • ✔️010_tpch_06: 32.8ms -> 33.2ms [-6.21%, +8.67%]
  • ❗🐌010_tpch_07: 150ms -> 159ms [+4.71%, +7.57%]
  • ❗🐌010_tpch_08: 117ms -> 137ms [+12.89%, +20.08%]
  • ❗🐌010_tpch_09: 161ms -> 172ms [+5.19%, +7.37%]
  • ❗🐌010_tpch_10: 127ms -> 134ms [+4.74%, +6.44%]
  • ❗🐌010_tpch_11: 44.6ms -> 50.7ms [+10.86%, +16.3%]
  • ❗🐌010_tpch_12: 55.2ms -> 57.5ms [+0.4%, +7.9%]
  • ❗🐌010_tpch_13: 52.2ms -> 53.8ms [+1.33%, +4.63%]
  • ✔️010_tpch_14: 36.9ms -> 38.6ms [-1.21%, +10.42%]
  • ✔️010_tpch_15: 58.6ms -> 61.9ms [-1.21%, +12.25%]
  • ❗🐌010_tpch_16: 45.4ms -> 50.1ms [+7.71%, +13.04%]
  • ✔️010_tpch_17: 60.5ms -> 63.7ms [-0.65%, +11.11%]
  • ✔️010_tpch_18: 57.8ms -> 57.8ms [-5.38%, +5.64%]
  • ❗🐌010_tpch_19: 141ms -> 144ms [+1.28%, +2.64%]
  • ❗🐌010_tpch_20: 73ms -> 84.8ms [+12.76%, +19.55%]
  • ❗🐌010_tpch_21: 324ms -> 344ms [+3.29%, +8.84%]
  • ❗🐌010_tpch_22: 62.5ms -> 68.1ms [+6.09%, +11.74%]
  • ✔️100_tpch_01: 342ms -> 363ms [-15.87%, +27.91%]
  • ❗🐌100_tpch_02: 128ms -> 145ms [+2.79%, +23.66%]
  • ✔️100_tpch_03: 171ms -> 178ms [-4.63%, +12.85%]
  • ✔️100_tpch_04: 156ms -> 162ms [-9.34%, +17.04%]
  • ✔️100_tpch_05: 245ms -> 261ms [-3.64%, +16.83%]
  • ✔️100_tpch_06: 93.9ms -> 93.3ms [-6.5%, +5.15%]
  • ❗🐌100_tpch_07: 234ms -> 245ms [+3.28%, +6.2%]
  • ✔️100_tpch_08: 264ms -> 267ms [-5.55%, +8.12%]
  • ✔️100_tpch_09: 303ms -> 318ms [-4.2%, +14.12%]
  • ❗🐌100_tpch_10: 198ms -> 217ms [+1.03%, +18.98%]
  • ❗🐌100_tpch_11: 83.1ms -> 91.2ms [+3.66%, +15.73%]
  • 🚀100_tpch_12: 204ms -> 181ms [-16.24%, -5.99%]
  • ✔️100_tpch_13: 338ms -> 334ms [-8.86%, +6.66%]
  • ✔️100_tpch_14: 117ms -> 116ms [-14.32%, +13.34%]
  • ✔️100_tpch_15: 208ms -> 206ms [-7.5%, +5.14%]
  • ✔️100_tpch_16: 132ms -> 134ms [-0.48%, +3.7%]
  • ✔️100_tpch_17: 178ms -> 178ms [-8.05%, +7.79%]
  • ✔️100_tpch_18: 201ms -> 206ms [-6.17%, +10.35%]
  • ✔️100_tpch_19: 261ms -> 281ms [-3.97%, +19.39%]
  • ❗🐌100_tpch_20: 173ms -> 196ms [+1.12%, +25.77%]
  • ✔️100_tpch_21: 1.46s -> 1.47s [-2.24%, +4.71%]
  • ✔️100_tpch_22: 159ms -> 162ms [-12.64%, +16.59%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@krlmlr
Copy link
Member Author

krlmlr commented Apr 11, 2025

This probably should be opt-in. Could be an option because it doesn't affect behavior, only performance tradeoffs.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if eaa39b2 is merged into main:

  • ✔️001_tpch_01: 24.1ms -> 24.9ms [-0.79%, +7.46%]
  • ❗🐌001_tpch_02: 72ms -> 85ms [+16.25%, +20.05%]
  • ❗🐌001_tpch_03: 35.9ms -> 41.3ms [+13.65%, +16.39%]
  • ❗🐌001_tpch_04: 23ms -> 25.8ms [+10.88%, +13.19%]
  • ❗🐌001_tpch_05: 61.4ms -> 68.8ms [+10.64%, +13.33%]
  • ❗🐌001_tpch_06: 12.4ms -> 13ms [+0.69%, +7.8%]
  • ❗🐌001_tpch_07: 77.8ms -> 88.3ms [+12.56%, +14.64%]
  • ❗🐌001_tpch_08: 99.9ms -> 116ms [+14.88%, +17.73%]
  • ❗🐌001_tpch_09: 73.6ms -> 83ms [+11.78%, +13.7%]
  • ❗🐌001_tpch_10: 52.3ms -> 58.7ms [+9.71%, +14.64%]
  • ❗🐌001_tpch_11: 36.2ms -> 42.5ms [+15.73%, +19.03%]
  • ❗🐌001_tpch_12: 24ms -> 25.3ms [+3.21%, +8.14%]
  • ❗🐌001_tpch_13: 19.3ms -> 21.6ms [+9.38%, +13.81%]
  • ❗🐌001_tpch_14: 18.5ms -> 19ms [+0.43%, +5.03%]
  • ❗🐌001_tpch_15: 32.1ms -> 36.4ms [+11.91%, +14.62%]
  • ❗🐌001_tpch_16: 30.7ms -> 35.4ms [+13.49%, +17%]
  • ❗🐌001_tpch_17: 28.5ms -> 31.7ms [+6.03%, +16.47%]
  • ❗🐌001_tpch_18: 23.5ms -> 25.9ms [+8.36%, +12.37%]
  • ❗🐌001_tpch_19: 45.4ms -> 47.2ms [+2.85%, +5.23%]
  • ❗🐌001_tpch_20: 47.3ms -> 58.8ms [+23.81%, +25%]
  • ❗🐌001_tpch_21: 84.6ms -> 101ms [+17.72%, +21.34%]
  • ❗🐌001_tpch_22: 48.2ms -> 54.9ms [+12.87%, +14.69%]
  • ❗🐌010_tpch_01: 87.3ms -> 90.1ms [+0.22%, +6.24%]
  • ❗🐌010_tpch_02: 127ms -> 142ms [+7.6%, +15.89%]
  • ❗🐌010_tpch_03: 65.7ms -> 69.7ms [+4.25%, +8.07%]
  • ❗🐌010_tpch_04: 47.3ms -> 50.4ms [+3.62%, +9.34%]
  • ❗🐌010_tpch_05: 144ms -> 154ms [+5.28%, +8.9%]
  • ✔️010_tpch_06: 34.2ms -> 34.7ms [-5.5%, +8.13%]
  • ❗🐌010_tpch_07: 156ms -> 166ms [+4.37%, +7.99%]
  • ❗🐌010_tpch_08: 121ms -> 140ms [+12.67%, +17.8%]
  • ❗🐌010_tpch_09: 170ms -> 181ms [+4%, +8.39%]
  • ❗🐌010_tpch_10: 129ms -> 133ms [+0.4%, +5.25%]
  • ❗🐌010_tpch_11: 45.5ms -> 51.3ms [+10.05%, +15.81%]
  • ✔️010_tpch_12: 56.6ms -> 60ms [-6.16%, +17.89%]
  • ❗🐌010_tpch_13: 52.4ms -> 55.1ms [+2.15%, +8.06%]
  • ✔️010_tpch_14: 37.9ms -> 38.5ms [-0.26%, +3.52%]
  • ❗🐌010_tpch_15: 60.3ms -> 64.4ms [+0.25%, +13.38%]
  • ❗🐌010_tpch_16: 46.5ms -> 50ms [+4.64%, +10.67%]
  • ❗🐌010_tpch_17: 59.3ms -> 63.2ms [+4.59%, +8.37%]
  • ✔️010_tpch_18: 59.3ms -> 61.4ms [-3.83%, +10.87%]
  • ✔️010_tpch_19: 145ms -> 144ms [-3.05%, +1.7%]
  • ❗🐌010_tpch_20: 73.4ms -> 85ms [+13.72%, +17.83%]
  • ❗🐌010_tpch_21: 336ms -> 347ms [+0.43%, +6.34%]
  • ❗🐌010_tpch_22: 115ms -> 123ms [+5.47%, +8.63%]
  • ✔️100_tpch_01: 377ms -> 372ms [-5.29%, +2.37%]
  • ❗🐌100_tpch_02: 131ms -> 144ms [+8.64%, +11.88%]
  • ✔️100_tpch_03: 183ms -> 179ms [-8.72%, +4.82%]
  • ✔️100_tpch_04: 152ms -> 151ms [-12.73%, +11.46%]
  • ✔️100_tpch_05: 266ms -> 262ms [-18.5%, +15.6%]
  • ✔️100_tpch_06: 99.8ms -> 99.6ms [-10.92%, +10.51%]
  • ✔️100_tpch_07: 225ms -> 239ms [-2.11%, +13.85%]
  • ✔️100_tpch_08: 278ms -> 287ms [-5.2%, +12.12%]
  • ✔️100_tpch_09: 320ms -> 330ms [-11.47%, +17.85%]
  • ✔️100_tpch_10: 215ms -> 215ms [-9.84%, +9.93%]
  • ✔️100_tpch_11: 84.3ms -> 96.9ms [-7.97%, +37.78%]
  • ✔️100_tpch_12: 199ms -> 199ms [-3.77%, +3.76%]
  • ✔️100_tpch_13: 336ms -> 348ms [-5.55%, +12.38%]
  • ✔️100_tpch_14: 116ms -> 118ms [-15.31%, +17.45%]
  • ✔️100_tpch_15: 211ms -> 220ms [-9.24%, +17%]
  • ✔️100_tpch_16: 137ms -> 142ms [-1.31%, +9.54%]
  • ✔️100_tpch_17: 181ms -> 185ms [-9.11%, +13.41%]
  • ✔️100_tpch_18: 213ms -> 217ms [-12.4%, +16.85%]
  • ✔️100_tpch_19: 257ms -> 266ms [-7.8%, +14.64%]
  • ✔️100_tpch_20: 180ms -> 184ms [-3.11%, +8.37%]
  • ✔️100_tpch_21: 1.44s -> 1.46s [-6.3%, +8.52%]
  • ✔️100_tpch_22: 158ms -> 166ms [-7%, +16.81%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@krlmlr
Copy link
Member Author

krlmlr commented Sep 17, 2025

Strangely, this seems to work, see nondeterministic in posit-conf notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Reuse already materialized data frames

1 participant

@krlmlr