-
Notifications
You must be signed in to change notification settings - Fork 45
feat: implement Literal Transform #156
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@@ -51,6 +61,45 @@ Result<ArrowArray> BucketTransform::Transform(const ArrowArray& input) { | |||
return NotImplemented("BucketTransform::Transform"); | |||
} | |||
|
|||
Result<std::optional<Literal>> BucketTransform::Transform(const Literal& literal) { | |||
assert(literal.type() == source_type()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not related to this PR: we should introduce something like ARROW_DCHECK
to add some explicit debug-level checks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, we may incorporate this alongside our logging infrastructure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for bearing with me! I think we still need some changes to fully conform to the spec.
also fix some lint warnings
853fab3
to
4c0766a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 Thanks!
My only concern is that we need to have better test coverage of those transform utility functions. We can improve them in followup PRs as this one is getting large enough.
using namespace std::chrono; // NOLINT | ||
switch (source_type()->type_id()) { | ||
case TypeId::kDate: { | ||
auto value = std::get<int32_t>(literal.value()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I think we need also move all of these transform functions to a dedicated utility file and add exhaustive test cases. Could you create an issue for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea.
#164
@Fokko @zeroshade Could you help review this? Thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we can assert return literal type is equal to result type?
That would require introducing a temporary variable before the return statement. I think if we have thorough test cases, the assert might not be necessary. |
src/iceberg/util/truncate_utils.h
Outdated
|
||
namespace iceberg { | ||
|
||
ICEBERG_EXPORT class TruncateUtils { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ICEBERG_EXPORT class TruncateUtils { | |
class ICEBERG_EXPORT TruncateUtils { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I initially used this approach, but the pre-commit clang-format hook misformatted the file, so I followed the StringUtils approach instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I found that upgrade https://github.com/pre-commit/mirrors-clang-format to v20.1.8 works, so changed to the suggested approach, thanks.
No description provided.