Skip to content

[BP-2.4][FLINK-41468][table] Keep partition keys in materialized table conversion - #29993

Merged
raminqaf merged 1 commit into
apache:release-2.4from
Timm0:FLINK-41468-bp-2.4
Nov 18, 2026
Merged

raminqaf merged 1 commit into
apache:release-2.4from
Timm0:FLINK-41468-bp-2.4

Conversation

@Timm0

@Timm0 Timm0 commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Backport of #29968 to release-2.4. When CREATE OR ALTER MATERIALIZED TABLE converted a partitioned table, the new materialized table could get different partition keys, which made the existing data unreadable. The conversion now requires PARTITIONED BY with the table's partition keys in the same order.

Brief change log

Cherry-pick of 13cc6d08d96 with one conflict:

  • MaterializedTableOperationFactory does not exist on release-2.4. The partition-key check went into SqlCreateOrAlterMaterializedTableConverter#handleConvert instead, with the same error message. The conversion now resolves the new table through getResolvedCatalogMaterializedTable, the release-2.4 equivalent of resolveNewMaterializedTable, so it runs CREATE's partition validation, including the partition.fields.#.date-formatter options.
  • Tests and docs applied cleanly.

Verifying this change

  • SqlNodeToOperationConvertTableToMaterializedTableTest and the other materialized table tests in flink-table-planner pass on this branch.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Claude Opus 5.5)

…sion

- Require `PARTITIONED BY` with a partitioned table's partition keys, in the same order, when `CREATE OR ALTER MATERIALIZED TABLE` converts it (unpartitioned tables still take the statement's keys)
- Run CREATE's partition validation on conversion, including `partition.fields.#.date-formatter` options
- Add `PartitionKeys` tests to `SqlNodeToOperationConvertTableToMaterializedTableTest`
- Document the partition-key rule in the conversion's `Partition keys` subsection of `statements.md`

(cherry picked from commit 13cc6d0)
Generated-by: Claude Code (Claude Opus 5.5)
@Timm0
Timm0 marked this pull request as ready for review November 17, 2026 16:10
@flinkbot

flinkbot commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@raminqaf raminqaf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@raminqaf
raminqaf merged commit aaf0ebd into apache:release-2.4 Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants