You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After all the other tasks associated with the overall predicate pushdown issue are complete, there will still be a number of cases where we will need to improve pruning triggers for certain engine types. Scenarios we may need to address with follow-ups include:
Cases where the partition filter granularity is different from the dimension itself and the query spec in such a way that our granularity adjustments bypass pruning - i.e., if we do date_trunc then partition pruning may not apply.
Cases where the partition filter column is tied to the agg_time_dimension but is not the agg_time_dimension itself (e.g., a PARTITION_LOAD_DATE might be mapped on to a daily granularity time field, and we'd want to render a separate filter expression)
Cases where there's a categorical partition that can be mapped from some other partition, like maybe a table is partitioned by REGION_CODE so any filter on COUNTRY can easily be made more efficient
Cases where a partition filter must always be provided, even if it is a partition IS NOT NULL constraint (like 3 above but where users want to group by country instead of pruning out input)
Some of these can be handled with more robust documentation around existing syntax and use of the is_partition flag on dimension columns, but others will likely require expanded support for partition specification in the semantic models themselves.
tlento
changed the title
Ensure predicate rendering is compatible with partition pruning directives across relevant engines
[SL-1631] Ensure predicate rendering is compatible with partition pruning directives across relevant engines
Jan 29, 2024
After all the other tasks associated with the overall predicate pushdown issue are complete, there will still be a number of cases where we will need to improve pruning triggers for certain engine types. Scenarios we may need to address with follow-ups include:
partition IS NOT NULL
constraint (like 3 above but where users want to group by country instead of pruning out input)Some of these can be handled with more robust documentation around existing syntax and use of the
is_partition
flag on dimension columns, but others will likely require expanded support for partition specification in the semantic models themselves.From SyncLinear.com | SL-1631
The text was updated successfully, but these errors were encountered: