Skip to content

Conversation

@kumarUjjawal
Copy link
Contributor

Which issue does this PR close?

Rationale for this change

  • The goal is to replace the old statistics API with the new partition_statistics

What changes are included in this PR?

  • implemented the partition_statistics API for SortMergeJoinExec
  • Removed the special case that returned unknown statistics for specific partitions.
  • Added test_partition_statistics to verify both aggregate and specific partition statistics work correctly

Are these changes tested?

  • Addes new tests

Are there any user-facing changes?

@github-actions github-actions bot added the physical-plan Changes to the physical-plan crate label Dec 30, 2025
@kumarUjjawal
Copy link
Contributor Author

cc @xudong963

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kumarUjjawal I feel the test might be expanded

For aggregate statistics (partition = None): Should return meaningful statistics computed from both inputs

For partition-specific statistics (partition = Some(0)): Currently returns unknown statistics, but ideally should return statistics specific to that partition

Currently it just checks column num?

@kumarUjjawal
Copy link
Contributor Author

kumarUjjawal commented Dec 31, 2025

Thanks @kumarUjjawal I feel the test might be expanded

For aggregate statistics (partition = None): Should return meaningful statistics computed from both inputs

For partition-specific statistics (partition = Some(0)): Currently returns unknown statistics, but ideally should return statistics specific to that partition

Currently it just checks column num?

Good point. Thank you for the feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants