[Enhancement] support struct column late materialized #28222
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Seaven
force-pushed
the
dev2
branch
4 times, most recently
from
6f6a817
to
b637626
Compare
stdpain
reviewed
Aug 14, 2023
| void resize(size_t n) override { _size = n; } | ||
| void resize(size_t n) override { | ||
| if (_size == 0) { | ||
| _data->resize(1); |
There was a problem hiding this comment.
why we need call resize when _size == 0
There was a problem hiding this comment.
I need resize column to keep same rows when fill default value
stdpain
reviewed
Aug 14, 2023
fzhedu
reviewed
Aug 14, 2023
Comment on lines
+143
to
+148
| void append_default(size_t count) override { | ||
| if (_size == 0) { | ||
| _data->append_default(1); | ||
| } | ||
| _size += count; | ||
| } |
There was a problem hiding this comment.
reject if data is not emtpy?
There was a problem hiding this comment.
no, don't need reject, only need modified size
fzhedu
reviewed
Aug 14, 2023
Comment on lines
+90
to
+91
| array_column->elements_column()->append_default(1); | ||
| array_column->elements_column() = ConstColumn::create(array_column->elements_column(), num_to_read); |
There was a problem hiding this comment.
array_column must be emtpy before append_default()?
fzhedu
reviewed
Aug 14, 2023
Comment on lines
+222
to
+231
|
|
||
| size_t size_to_read = 0; | ||
| for (size_t i = 0; i < size; ++i) { | ||
| RETURN_IF_ERROR(_array_size_iterator->seek_to_ordinal_and_calc_element_ordinal(rowids[i])); | ||
| size_t element_ordinal = _array_size_iterator->element_ordinal(); | ||
| RETURN_IF_ERROR(_element_iterator->seek_to_ordinal(element_ordinal)); | ||
| size_to_read += array_size.get_data()[i]; | ||
| } | ||
|
|
||
| array_column->elements_column()->append_default(size_to_read); |
There was a problem hiding this comment.
yeah, for mock read data
fzhedu
reviewed
Aug 14, 2023
|
|
||
| StructColumn* struct_column = nullptr; | ||
| NullColumn* null_column = nullptr; | ||
| if (dst->is_nullable()) { |
There was a problem hiding this comment.
would dst be set const ? may be a subfield in struct{struct{},struct{stuct{}}}
There was a problem hiding this comment.
the method only call once on empty chunk, so I think never be const
Seaven
force-pushed
the
dev2
branch
2 times, most recently
from
2626df8
to
a5740fe
Compare
[FE PR Coverage Check]😍 pass : 4 / 5 (80.00%) file detail
|
Signed-off-by: Seaven <[email protected]>
Signed-off-by: Seaven <[email protected]>
Signed-off-by: Seaven <[email protected]>
Signed-off-by: Seaven <[email protected]>
Signed-off-by: Seaven <[email protected]>
|
SonarCloud Quality Gate failed.
|
[FE Incremental Coverage Report]😍 pass : 4 / 5 (80.00%) file detail
|
[BE Incremental Coverage Report]😞 fail : 45 / 246 (18.29%) file detail
|
stdpain
approved these changes
Sep 4, 2023
fzhedu
reviewed
Sep 5, 2023
| @@ -288,6 +288,7 @@ public class SessionVariable implements Serializable, Writable, Cloneable { | |||
| "enable_rewrite_groupingsets_to_union_all"; | |||
|
|
|||
| public static final String CBO_USE_DB_LOCK = "cbo_use_lock_db"; | |||
| public static final String CBO_PREDICATE_SUBFIELD_PATH = "cbo_enable_predicate_subfield_path"; | |||
fzhedu
approved these changes
Sep 5, 2023
Jay-ju
pushed a commit
to Jay-ju/starrocks
that referenced
this pull request
Sep 7, 2023
* [Enhancement] support struct column late materialized --------- Signed-off-by: Seaven <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #issue
support late materialized on struct column
/structA/structB/structC, can't prune the subfield ofstructB(/structA/array/structB), because the interface ofarray/mapdoesn't support now,array/mapneed flatten rows when fetch values by rowid, it's too complexDoesn't support on map/array, because only few scenarios when use
MAP_SIZE/ARRAY_LENGTH = xxas scan predicateWhat type of PR is this:
Checklist:
Bugfix cherry-pick branch check: