Skip to content

Conversation

zhjwpku
Copy link
Collaborator

@zhjwpku zhjwpku commented Jul 27, 2025

No description provided.

@@ -51,6 +61,45 @@ Result<ArrowArray> BucketTransform::Transform(const ArrowArray& input) {
return NotImplemented("BucketTransform::Transform");
}

Result<std::optional<Literal>> BucketTransform::Transform(const Literal& literal) {
assert(literal.type() == source_type());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to this PR: we should introduce something like ARROW_DCHECK to add some explicit debug-level checks.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we may incorporate this alongside our logging infrastructure.

Copy link
Member

@wgtmac wgtmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for bearing with me! I think we still need some changes to fully conform to the spec.

@zhjwpku zhjwpku force-pushed the implement_transform branch from 853fab3 to 4c0766a Compare August 7, 2025 15:02
Copy link
Member

@wgtmac wgtmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 Thanks!

My only concern is that we need to have better test coverage of those transform utility functions. We can improve them in followup PRs as this one is getting large enough.

using namespace std::chrono; // NOLINT
switch (source_type()->type_id()) {
case TypeId::kDate: {
auto value = std::get<int32_t>(literal.value());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we need also move all of these transform functions to a dedicated utility file and add exhaustive test cases. Could you create an issue for this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea.
#164

@zhjwpku
Copy link
Collaborator Author

zhjwpku commented Aug 8, 2025

@Fokko @zeroshade Could you help review this? Thanks!

@zhjwpku zhjwpku requested a review from mapleFU August 9, 2025 02:40
Copy link
Member

@mapleFU mapleFU left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can assert return literal type is equal to result type?

@zhjwpku
Copy link
Collaborator Author

zhjwpku commented Aug 9, 2025

Perhaps we can assert return literal type is equal to result type?

That would require introducing a temporary variable before the return statement. I think if we have thorough test cases, the assert might not be necessary.


namespace iceberg {

ICEBERG_EXPORT class TruncateUtils {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ICEBERG_EXPORT class TruncateUtils {
class ICEBERG_EXPORT TruncateUtils {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially used this approach, but the pre-commit clang-format hook misformatted the file, so I followed the StringUtils approach instead.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I found that upgrade https://github.com/pre-commit/mirrors-clang-format to v20.1.8 works, so changed to the suggested approach, thanks.

@zeroshade zeroshade merged commit a13eed2 into apache:main Aug 11, 2025
6 checks passed
@zhjwpku zhjwpku deleted the implement_transform branch August 13, 2025 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants