Add Trino 479 release notes #27153
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,193 @@ | ||
| # Release 479 (TBD 2025) | ||
|
|
||
| ## General | ||
|
|
||
| * Generate TLS certificates for internal cluster communications automatically when node discovery type is set to `ANNOUNCE`. ({issue}`27030`) | ||
| * Add support for specifying the location of the Exchange Manager configuration file. ({issue}`26611`) | ||
| * Publish time taken by query in `FINISHING` state in event listener. ({issue}`27202`) | ||
| * Add lineage support for output columns in `SELECT` queries. ({issue}`26241`) | ||
| * Require JDK 25 to build and run Trino. ({issue}`27153`) | ||
| * Add support for setting and dropping column defaults via ALTER TABLE ... ALTER COLUMN" at current. ({issue}`26162`) | ||
| * Add {func}`array_first` and {func}`array_last` functions. ({issue}`27295`) | ||
chenjian2664 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * {{breaking}} The configuration property `task.statistics-cpu-timer-enabled` is now defunct and must be removed. ({issue}`27504`) | ||
| * Allow field name declaration in row literals. For example, `row(1 as a, 2 as b)` is now legal. ({issue}`25261`) | ||
| * Deprecate `EXPLAIN (TYPE LOGICAL)`. `EXPLAIN (TYPE DISTRIBUTED)` should be used instead. ({issue}`27434`) | ||
|
Member
There was a problem hiding this comment. No need for the second sentence. That is the default behavior.
Member
There was a problem hiding this comment. "Deprecate
Member
There was a problem hiding this comment. We don't want users to use
Member
There was a problem hiding this comment. If using |
||
| * Remove `prefer_streaming_operators` session property. `SET SESSION task_concurrency=1` can be used to achieve the same behaviour as `prefer_streaming_operators`. ({issue}`27506`) | ||
|
Member
There was a problem hiding this comment. Remove this. The option hasn't really done anything since we removed support for multiple table layouts years ago.
Member
There was a problem hiding this comment. It may not have been intended to do anything after removal of multiple table layouts support. However, the code was having the same effect as setting task_concurrency to 1 until now. We can drop the second line if we don't want to advertise the effect of that flag, but we should retain the first line about its removal. |
||
| * Improve performance of remote data exchanges on CPUs supporting the required SIMD extensions. This can be disabled by setting `exchange.experimental.vectorized-serde.enabled=false`. ({issue}`27426`, {issue}`26919`) | ||
|
Member
There was a problem hiding this comment. "required SIMD extensions" is low level and not very actionable. Do we have a list of CPUs and architectures, even if partial?
Member
There was a problem hiding this comment. ARM: CPU's supporting SVE (no NEON) Or to enumerate specific hardware generations: |
||
| * Fix query failure when one of the branches of a `UNION` is known to produce zero rows during query planning. ({issue}`21506`) | ||
| * Improve performance of {func}`array_sort` function. ({issue}`27272`) | ||
| * Improve performance of {func}`repeat` function. ({issue}`27369`) | ||
| * Improve performance of data exchanges involving variable width data. ({issue}`27377`) | ||
| * Fix configuring partitioned layout for spooling protocol. ({issue}`27247`) | ||
ebyhr marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * Fix `EXPLAIN (TYPE IO)` failure when query constraint contains type which cannot be cast to `varchar`. ({issue}`27433`) | ||
|
Member
There was a problem hiding this comment. What does "query constraint" refer to? Also, "contains a type"
Contributor
Author
There was a problem hiding this comment. "query constraint" may comes from here https://github.com/trinodb/trino/pull/27433/files#diff-7dad18a4dbe9d9a536d45655875f47ffd0d8e3b5d5c6d2f3447246763841fb44R1673-R1677 , Constraint, ColumnConstraint, so it is from "predicate" ? |
||
|
|
||
| ## Security | ||
|
|
||
| ## Web UI | ||
|
|
||
| * Fix preview UI to render single line queries. ({issue}`27328`) | ||
|
Member
There was a problem hiding this comment. What's the fix? Did single line queries not work before?
Contributor
Author
Member
There was a problem hiding this comment. how about: "Fix preview UI to correctly wrap long single-line query strings on the query details page"
Contributor
Author
There was a problem hiding this comment. Also what about "Render long single-line query string correctly in preview UI query details page" |
||
| * Render catalog properties correctly in preview UI. ({issue}`27327`) | ||
|
|
||
| ## JDBC driver | ||
chenjian2664 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| * Add a `extraHeaders` option to support sending arbitrary HTTP headers. ({issue}`15826`) | ||
|
|
||
| ## Docker image | ||
|
|
||
| * Update JDK to 25.0.1. ({issue}`27117`) | ||
|
|
||
| ## CLI | ||
|
|
||
| * Add a `--extra-header` option to support sending arbitrary HTTP headers. ({issue}`15826`) | ||
| * Fix TLS connection failures when using unqualified (single-label, without ".") hostnames. ({issue} `27478`) | ||
|
|
||
| ## BigQuery connector | ||
|
|
||
| ## Blackhole connector | ||
|
|
||
| ## Cassandra connector | ||
|
|
||
| ## ClickHouse connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## Delta Lake connector | ||
|
|
||
| * Add `APPLICATION_DEFAULT` authentication type for GCS. ({issue}`26984`) | ||
| * {{breaking}} Remove support for unauthenticated access when GCS authentication type is set to `SERVICE_ACCOUNT`. ({issue}`26984`) | ||
| * Rename `s3.exclusive-create` config to `delta.s3.transaction-log-conditional-writes.enabled`. ({issue}`27372`) | ||
| * Fix incorrect results for queries involving `IS NOT DISTINCT FROM`. ({issue}`27213`) | ||
| * Fix failure when writing to tables created by Databricks 17.3. ({issue}`27100`) | ||
| * Harden hierarchical namespace check in Azure with root blob fallback check. ({issue}`27278`) | ||
|
Member
There was a problem hiding this comment. What does this mean? What's the user visible impact? |
||
| * Avoid worker crashes by failing queries attempting to read columns with huge values in parquet files. ({issue}`27148`) | ||
| * Fix potential `FileAlreadyExistsException` failure when network failures occur during writes to S3. ({issue}`27330`, {issue}`27388`) | ||
| * Fix potential failure when reading [cloned tables](https://docs.databricks.com/aws/en/delta/clone). ({issue}`27098`) | ||
|
|
||
| ## Druid connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## DuckDB connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## Elasticsearch connector | ||
|
|
||
| ## Exasol connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## Faker connector | ||
|
|
||
| ## Google Sheets connector | ||
|
|
||
| ## Hive connector | ||
|
|
||
ebyhr marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * Add `APPLICATION_DEFAULT` authentication type for GCS. ({issue}`26984`) | ||
| * {{breaking}} Remove support for unauthenticated access when GCS authentication type is set to `SERVICE_ACCOUNT`. ({issue}`26984`) | ||
| * Remove `s3.exclusive-create` config. ({issue}`27372`) | ||
|
Member
There was a problem hiding this comment. What functionality is being removed?
Contributor
Author
There was a problem hiding this comment. Just remove the config from Hive, previously the configuration was added but only used by Delta |
||
| * Improve accuracy of table statistics written by INSERT queries with OVERWRITE behaviour. ({issue}`26517`) | ||
| * Harden hierarchical namespace check in Azure with root blob fallback check. ({issue}`27278`) | ||
|
Member
There was a problem hiding this comment. This entry should start with "Fix failure when ...". Same for other connectors.
Contributor
Author
There was a problem hiding this comment. How about : Fix failure when checking Azure hierarchical namespaces by introducing a root blob fallback. @findinpath |
||
| * Avoid worker crashes by failing queries attempting to read columns with huge values in parquet files. ({issue}`27148`) | ||
| * Fix potential `FileAlreadyExistsException` failure when network failures occur during writes to S3. ({issue}`27330`) | ||
| * Fix failure when listing tables with invalid table metadata in AWS Glue. ({issue}`27525`) | ||
|
|
||
| ## Hudi connector | ||
|
|
||
| * Add `APPLICATION_DEFAULT` authentication type for GCS. ({issue}`26984`) | ||
| * {{breaking}} Remove support for unauthenticated access when GCS authentication type is set to `SERVICE_ACCOUNT`. ({issue}`26984`) | ||
| * Harden hierarchical namespace check in Azure with root blob fallback check. ({issue}`27278`) | ||
| * Avoid worker crashes by failing queries attempting to read columns with huge values in parquet files. ({issue}`27148`) | ||
| * Fix potential `FileAlreadyExistsException` failure when network failures occur during writes to S3. ({issue}`27330`) | ||
| * Fix failure when querying HDFS that includes empty directories. ({issue}`26897`) | ||
|
|
||
| ## Iceberg connector | ||
|
|
||
| * Add support for disabling token exchange via the `iceberg.rest-catalog.oauth2.token-exchange-enabled` config property. ({issue}`27174`) | ||
|
Contributor
Author
There was a problem hiding this comment. It is the "token-exchange-enabled" in https://iceberg.apache.org/docs/nightly/configuration/#oauth2-auth-properties How about :"Add support for updating token-exchange-enabled via the |
||
| * Add `retain_last` and `clean_expired_metadata` options to `expire_snapshots` command. ({issue}`27357`) | ||
| * Remove `s3.exclusive-create` config. ({issue}`27372`) | ||
| * Add `APPLICATION_DEFAULT` authentication type for GCS. ({issue}`26984`) | ||
| * {{breaking}} Remove support for unauthenticated access when GCS authentication type is set to `SERVICE_ACCOUNT`. ({issue}`26984`) | ||
| * Fix failures when querying `$files` table after changes to table partitioning. ({issue}`26746`) | ||
| * Fix incorrect results for queries involving `IS NOT DISTINCT FROM`. ({issue}`27213`) | ||
| * Harden hierarchical namespace check in Azure with root blob fallback check. ({issue}`27278`) | ||
| * Avoid worker crashes by failing queries attempting to read columns with huge values in parquet files. ({issue}`27148`) | ||
| * Fix potential `FileAlreadyExistsException` failure when network failures occur during writes to S3. ({issue}`27330`) | ||
| * Fix potential failure when dropping a schema with `CASCADE` option. ({issue}`27361`) | ||
|
|
||
| ## Ignite connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## JMX connector | ||
|
|
||
| ## Kafka connector | ||
|
|
||
| ## Loki connector | ||
|
|
||
| * Fix failure when initializing the connector. ({issue}`27180`) | ||
|
|
||
| ## MariaDB connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## Memory connector | ||
|
|
||
| * Add support for setting and dropping column defaults via `ALTER TABLE ... ALTER COLUMN`. ({issue}`26162`) | ||
| * Fix concurrent modification exception in `RENAME SCHEMA`. ({issue}`27205`) | ||
|
Member
There was a problem hiding this comment. What's "concurrent modification exception"? When does it happen? Describe those conditions, instead.
Contributor
Author
|
||
|
|
||
| ## MongoDB connector | ||
|
|
||
| ## MySQL connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
| * Fix incorrect results for queries involving `IS NOT DISTINCT FROM`. ({issue}`27213`) | ||
|
|
||
| ## OpenSearch connector | ||
|
|
||
| ## Oracle connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## Pinot connector | ||
|
|
||
| ## PostgreSQL connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
| * Fix incorrect results for queries involving `IS NOT DISTINCT FROM`. ({issue}`27213`) | ||
|
|
||
| ## Prometheus connector | ||
|
|
||
| ## Redis connector | ||
|
|
||
| ## Redshift connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
| * Fix failure when reading Redshift `character varying` type. ({issue}`27224`) | ||
|
|
||
| ## SingleStore connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## Snowflake connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## SQL Server connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
| * Fix potential failure when listing tables and columns. ({issue}`10846`) | ||
|
|
||
| ## TPC-H connector | ||
|
|
||
| ## TPC-DS connector | ||
|
|
||
| ## Vertica connector | ||
|
|
||
| * {{breaking}} Replace `join-pushdown.with-expressions` configuration property with `deprecated.join-pushdown.with-expressions`. The new property will be removed in a future version. | ||
|
|
||
| ## SPI | ||
|
|
||
| * Add non-callback based entry builder to RowBlockBuilder. ({issue}`27198`) | ||
| * Add non-callback based entry builder to ArrayBlockBuilder. ({issue}`27198`) | ||
| * Fix `ColumnMetadata.builderFrom` to retain column default value. ({issue}`27503`) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is unclear. What's the user-visible impact of this? What does it mean to have lineage for output columns?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Praveen2112 @behrooz-stripe
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Users will have access to a new field (named
selectColumnsLineageInfo) in QueryCompletedEvent of EventListeners that includes lineage information for each output column of select queries. This lineage info will indicate from where each output column was sourced from. This was historically only available for insert, or create queries only.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so, it is more likely : "Add lineage support for output columns of
SELECTqueries"