Add `WHEN STALE` syntax for `CREATE MATERIALIZED VIEW` #27356

piotrrzysko · 2025-11-18T09:27:40Z

Description

This PR introduces a new optional clause for CREATE MATERIALIZED VIEW that allows users to control the behavior of materialized views when they are stale.

I will updated the documentation once the proposed syntax and behavior are approved.

New syntax

CREATE MATERIALIZED VIEW mv_name
WHEN STALE { FAIL | INLINE }
AS SELECT ...

Behavior

FAIL - If the MV is stale, queries referencing it will fail. Analysis of the underlying query is never performed while querying the MV.
INLINE (default) - Preserves the current behavior. Even if the MV is stale, it is expanded like a logical view, and the underlying query is analyzed.

Motivation

Avoid analyzing the underlying query every time, which can be costly when it references many tables.
Allow using fresh MVs even when some data sources are temporarily unavailable.

Additional context and related issues

The proposed clause was originally discussed in Define grace/staleness period for materialized views #15326
Materialized view should only check permissions of underlying query for refresh #23747

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## Section
* Add `WHEN STALE` option to `CREATE MATERIALIZED VIEW` ({issue}`issuenumber`)

piotrrzysko · 2025-11-18T09:40:05Z

core/trino-main/src/main/java/io/trino/sql/analyzer/StatementAnalyzer.java

                    TableHandle tableHandle = metadata.getTableHandle(session, storageTableName)
                            .orElseThrow(() -> semanticException(INVALID_VIEW, table, "Storage table '%s' does not exist", storageTableName));
-                    return createScopeForMaterializedView(table, name, scope, materializedViewDefinition, Optional.of(tableHandle));
+                    return createScopeForMaterializedView(table, name, scope, materializedViewDefinition, Optional.of(tableHandle), useLogicalViewSemantics);


I'm a bit concerned about this line: the new clause affects the path for fresh MVs (WHEN STALE FAIL skips analysis, while WHEN STALE INLINE runs the analysis), even though the syntax suggests it should only define behavior for stale MVs.

Is analysis of the underlying query even necessary for fresh MVs, regardless of the WHEN STALE clause? I understand that it can detect schema changes in the base tables or changes to the MV definer’s access permissions - but is that the intended behavior?

Praveen2112 · 2025-11-18T10:18:58Z

What is our plan on changing this behaviour for existing or new MVs ?

Praveen2112 · 2025-11-18T10:22:59Z

core/trino-main/src/main/java/io/trino/execution/CreateMaterializedViewTask.java

+                    if (!plannerContext.getMetadata().getConnectorCapabilities(session, catalogHandle).contains(MATERIALIZED_VIEW_WHEN_STALE_BEHAVIOR)) {
+                        throw semanticException(NOT_SUPPORTED, statement, "Catalog '%s' does not support WHEN STALE", catalogName);
+                    }


Should catalogs need to handle anything in order to support it ? If it is some sort of MV property (like security mode in case of views) - It should be supported for all connectors supporting MVs right ? Or we could fail during the creation or alter phase ?

I don't understand why we need a connector capability for this.
What's the default behavior of WHEN STALE going to be?

oh, i get it -- storing the clause requires a capability.

Praveen2112 · 2025-11-18T10:31:25Z

core/trino-main/src/main/java/io/trino/sql/analyzer/StatementAnalyzer.java

                }
                // This is a stale materialized view and should be expanded like a logical view
-                return createScopeForMaterializedView(table, name, scope, materializedViewDefinition, Optional.empty());
+                return createScopeForMaterializedView(table, name, scope, materializedViewDefinition, Optional.empty(), useLogicalViewSemantics);


Instead of passing a useLogicalViewSemantics can we create a scope for the storage table ?

Praveen2112 · 2025-11-18T10:39:08Z

Apart from the benefit on skipping the analysis what are the other benefits would I get if a run a query as WHEN STALE FAIL ? Now when I create a MV with WHEN STALE - I won't be able to use it unless I run a REFRESH MV. In case of pipeline which uses this MV might be affected if a MV went stale ? Is this expected ? Should we add this as an extension of WHEN FRESH SKIP ANALYSIS or something similar ?

martint · 2025-11-18T16:46:53Z

For context, #15326

martint · 2025-11-18T16:51:16Z

Also, #23387 (reply in thread) and #23747

piotrrzysko · 2025-11-19T10:40:01Z

@Praveen2112

What is our plan on changing this behaviour for existing or new MVs ?

This change is meant to be backward-compatible. The new clause is optional. If it is not specified, or if WHEN STALE INLINE is used, the MV will behave as it does currently.

Apart from the benefit on skipping the analysis what are the other benefits would I get if a run a query as WHEN STALE FAIL ? Now when I create a MV with WHEN STALE - I won't be able to use it unless I run a REFRESH MV. In case of pipeline which uses this MV might be affected if a MV went stale ? Is this expected ?

Skipping the analysis is not the goal in itself. The goal is to achieve better availability when the source tables are not available and to get closer to the semantics mentioned in the issues that Martin and I linked in this PR - specifically, that within the grace period, the MV should be treated as if it were a regular storage table. And today, as far as I know, the only blocker for this is the analysis performed every time the MV is queried.

Should we add this as an extension of WHEN FRESH SKIP ANALYSIS or something similar ?

This is more or less what I wanted to express in this comment: #27356 (comment). I’m happy to discuss what the syntax should look like to achieve the goals described above. Personally, WHEN FRESH SKIP ANALYSIS feels too low-level - as mentioned in my comment, I'd consider always skipping analysis when the MV is fresh.

piotrrzysko · 2025-11-26T13:54:21Z

@martint just a gentle reminder about the PR when you have a moment

findepi · 2025-11-26T16:26:45Z

@piotrrzysko can you please rebase? there is a conflict.

findepi · 2025-11-26T16:29:40Z

core/trino-main/src/main/java/io/trino/sql/rewrite/ShowQueriesRewrite.java

                    query,
                    false,
                    false,
                    Optional.empty(), // TODO support GRACE PERIOD


did we miss doing this?

Yeah, it looks like someone (6d5215c) forgot about it 😀.

do you want to create a separate PR to have this addressed?

findepi

Add WHEN STALE FAIL/INLINE syntax

lgtm

findepi

Add when stale behavior in ConnectorMaterializedViewDefinition

findepi · 2025-11-26T16:38:55Z

core/trino-main/src/main/java/io/trino/execution/CreateMaterializedViewTask.java

+                    if (!plannerContext.getMetadata().getConnectorCapabilities(session, catalogHandle).contains(MATERIALIZED_VIEW_WHEN_STALE_BEHAVIOR)) {
+                        throw semanticException(NOT_SUPPORTED, statement, "Catalog '%s' does not support WHEN STALE", catalogName);
+                    }


I don't understand why we need a connector capability for this.
What's the default behavior of WHEN STALE going to be?

core/trino-main/src/test/java/io/trino/sql/analyzer/TestAnalyzer.java

This is a preparatory change that allows connectors to store information about the WHEN STALE behavior. The option cannot yet be used in practice, as execution will still fail with a NOT_SUPPORTED error.

findepi · 2025-11-28T10:28:34Z

testing/trino-testing/src/main/java/io/trino/testing/BaseConnectorTest.java

+            // Change underlying state
+            assertUpdate("INSERT INTO " + baseTable + " (regionkey, name) VALUES (42, 'foo new region')", 1);
+            String updatedResults = initialResults + " UNION ALL VALUES (42, 'foo new region')";
+
+            // The materialization is stale now
+            assertThat(query(defaultSession, "TABLE " + viewName))
+                    .hasPlan(readFromStorageTable)
+                    .matches(initialResults);
+            assertThat(query(futureSession, "TABLE " + viewName))
+                    .hasPlan(readFromBaseTables)
+                    .matches(updatedResults);


The MV has GRACE PERIOD INTERVAL '1' HOUR.
The GRACE is defined earlier in the MV definition and should take precedence. I.e. we should keep querying the MV storage when it's known to be stale but grace period did not elapse.

cla-bot bot added the cla-signed label Nov 18, 2025

github-actions bot added iceberg Iceberg connector lakehouse labels Nov 18, 2025

piotrrzysko added the syntax-needs-review label Nov 18, 2025

piotrrzysko commented Nov 18, 2025

View reviewed changes

piotrrzysko requested review from findepi, kasiafi and martint November 18, 2025 10:15

piotrrzysko marked this pull request as ready for review November 18, 2025 10:15

Praveen2112 reviewed Nov 18, 2025

View reviewed changes

findepi reviewed Nov 26, 2025

View reviewed changes

core/trino-main/src/test/java/io/trino/sql/analyzer/TestAnalyzer.java Show resolved Hide resolved

piotrrzysko mentioned this pull request Nov 27, 2025

Fix testAnalyzeMaterializedViewWithAccessControl #27477

Merged

piotrrzysko added 2 commits November 27, 2025 15:43

Add WHEN STALE FAIL/INLINE syntax

df2503a

Add when stale behavior in ConnectorMaterializedViewDefinition

976cbc3

This is a preparatory change that allows connectors to store information about the WHEN STALE behavior. The option cannot yet be used in practice, as execution will still fail with a NOT_SUPPORTED error.

piotrrzysko force-pushed the when_stale branch from da71e4a to 8d70622 Compare November 27, 2025 14:44

findepi changed the title ~~Add WHEN STALE option to CREATE MATERIALIZED VIEW~~ Add WHEN STALE option to CREATE MATERIALIZED VIEW and implement in Iceberg Nov 28, 2025

findepi reviewed Nov 28, 2025

View reviewed changes

piotrrzysko force-pushed the when_stale branch from 8d70622 to 976cbc3 Compare November 28, 2025 11:13

piotrrzysko mentioned this pull request Nov 28, 2025

Support WHEN STALE in analyzer #27502

Draft

findepi changed the title ~~Add WHEN STALE option to CREATE MATERIALIZED VIEW and implement in Iceberg~~ Add WHEN STALE option to CREATE MATERIALIZED VIEW Nov 28, 2025

findepi changed the title ~~Add WHEN STALE option to CREATE MATERIALIZED VIEW~~ Add WHEN STALE syntax for CREATE MATERIALIZED VIEW Nov 28, 2025

findepi approved these changes Nov 28, 2025

View reviewed changes

Praveen2112 approved these changes Nov 28, 2025

View reviewed changes

Add WHEN STALE syntax for CREATE MATERIALIZED VIEW #27356

Are you sure you want to change the base?

Add WHEN STALE syntax for CREATE MATERIALIZED VIEW #27356

Conversation

piotrrzysko commented Nov 18, 2025

Description

New syntax

Behavior

Motivation

Additional context and related issues

Release notes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Praveen2112 commented Nov 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Praveen2112 commented Nov 18, 2025

Uh oh!

martint commented Nov 18, 2025

Uh oh!

martint commented Nov 18, 2025

Uh oh!

piotrrzysko commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

piotrrzysko commented Nov 26, 2025

Uh oh!

findepi commented Nov 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

findepi left a comment

Choose a reason for hiding this comment

Uh oh!

findepi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Add `WHEN STALE` syntax for `CREATE MATERIALIZED VIEW` #27356

Add `WHEN STALE` syntax for `CREATE MATERIALIZED VIEW` #27356

piotrrzysko commented Nov 19, 2025 •

edited

Loading