Andrew Lamb Weekly-ish Open Source plan - 2025-11-24

[alamb]

This is my weekly plan, mostly for my own organizational needs (as I am dropping too many things). I am making it public in the hopes that helps others to see what I am working on -- also I spend so much time in github the interface is very familiar to me and I can cross link all the issues I am working

(it is also my excuse as to why I haven't reviewed many good looking PRs)

Notes to myself: a duplicate entry unchecked means I need to go back and re-review

PR review queue (rough order)

• add specialized InList implementations for common scalar types #18832
• Optimize planning for projected nested union #18713
• Support reverse parquet scan and fast parquet order inversion at row group level #18817
• [Variant] Improve variant_get performance on a perfect shredding arrow-rs#8887
• Add union to opaque comparisons arrow-rs#8896
• Add validated constructors for UnionFields arrow-rs#8891
• chore: update Repartition DisplayAs to indicate maintained sort order #18673
• Enable Parallel Aggregation for Non-Overlapping Partitioned Data #18826
• optimizer: Support dynamic filter in MIN/MAX aggregates #18644
• [DRAFT] Extension Type Registry Draft #18552
• move projection handling into FileSource #18627
• Make Parquet SBBF serialize/deserialize helpers public for external reuse arrow-rs#8762
• feat: implement GroupArrayAggAccumulator attempt 3 #17915
• feat: Deduplicating recursive CTE implementation #18254
• Add relation planner extension support #17843

Projects I am supporting actively (high on my priority list)

• Improve DataFusion ClickBench performance: [EPIC] Make DataFusion the top of the ClickBench Parquet leaderboard #18489
• Help integrate Variant with @friendlymatthew [EPIC] Support VARIANT type for unstructured data #16116
• Release object store 0.13.0: Release object store 0.13.0 (breaking) - Target Nov 2025 arrow-rs-object-store#367
• DataFusion object store requests go faster with @BlakeOrth [EPIC] ListingTable object store usage improvements #17214

Projects on my backlog

These are ones I would like to support but don't have the capacity at the moment to push, in relative order

• Epic: Join Order Enumeration #18249 from @NGA-TRAN

PRs that look great but need a thorough review (looking for help here 🎣 from anyone else)

• Examples of extending SQL syntax #17824 from the @theirix
• external tables for multiple locations: feat(cli): support external tables on multiple locations #17702
• relation extension planner: Add relation planner extension support #17843
• writing REE arrays to parquet: Support writing RunEndEncoded as Parquet arrow-rs#8069