Skip to content

Document period materialized views #202

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 60 additions & 21 deletions documentation/guides/mat-views.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,10 @@ As data grows in size, the performance of certain queries can degrade.
Materialized views store the result of a `SAMPLE BY` or time-based `GROUP BY`
query on disk, and keep it automatically up to date.

The refresh of a materialized view is `INCREMENTAL` and very efficient, and
using materialized views can offer 100x or higher query speedups. If you require
the lowest latency queries, for example, for charts and dashboards, use
materialized views!
The refresh of a materialized view is incremental and very efficient, and using
materialized views can offer 100x or higher query speedups. If you require the
lowest latency queries, for example, for charts and dashboards, use materialized
views!

For a better understanding of what materialized views are for, read the
[introduction to materialized views](/docs/concept/mat-views/) documentation.
Expand Down Expand Up @@ -106,7 +106,7 @@ If you are unfamiliar with the OHLC concept, please see our

```questdb-sql title="trades_OHLC_15m DDL"
CREATE MATERIALIZED VIEW 'trades_OHLC_15m'
WITH BASE 'trades' REFRESH INCREMENTAL AS
WITH BASE 'trades' REFRESH IMMEDIATE AS
SELECT
timestamp, symbol,
first(price) AS open,
Expand All @@ -124,9 +124,9 @@ In this example:
2. The base table is `trades`
- This is the data source, and will trigger incremental refresh when new data
is written.
3. The refresh strategy is `INCREMENTAL`
- The data is automatically refreshed and incrementally written; efficient,
fast, low maintenance.
3. The refresh strategy is `IMMEDIATE`
- The data is automatically refreshed and incrementally written after a base
table transaction occurs; efficient, fast, low maintenance.
4. The `SAMPLE BY` query contains two key column (`timestamp`, `symbol`) and
five aggregates (`first`, `max`, `min`, `last`, `price`) calculated in `15m`
time buckets.
Expand Down Expand Up @@ -172,13 +172,61 @@ will not trigger any sort of refresh.

#### Refresh strategies

Currently, only `INCREMENTAL` refresh is supported. This strategy incrementally
updates the view when new data is inserted into the base table. This means that
only new data is written to the view, so there is minimal write overhead.
The `IMMEDIATE` refresh strategy incrementally updates the view when new data is
inserted into the base table. This means that only new data is written to the
view, so there is minimal write overhead.

Upon creation, or when the view is invalidated, a full refresh will occur, which
rebuilds the view from scratch.

Other than `IMMEDIATE` refresh, QuestDB supports `MANUAL` and timer
(`EVERY <interval>`) strategies for materialized views. Manual strategy means
that to refresh the view, you need to run the
[`REFRESH` SQL](/docs/reference/sql/refresh-mat-view/) explicitly. In case of
timer-based refresh the view is refreshed periodically, at the specified
interval.

The refresh strategy of an existing view can be changed any time with the
[`ALTER SET REFRESH`](/docs/reference/sql/alter-mat-view-set-refresh/) command.

## Period materialized views

In certain use cases, like storing trading day information, the data becomes
available at fixed time intervals. In this case, `PERIOD` variant of
materialized views can be used:

```questdb-sql title="Period materialized view"
CREATE MATERIALIZED VIEW trades_daily_prices
REFRESH PERIOD (LENGTH 1d TIME ZONE 'Europe/London' DELAY 2h) AS
SELECT
timestamp,
symbol,
avg(price) AS avg_price
FROM trades
SAMPLE BY 1d;
```

Refer to the following
[documentation page](/docs/reference/sql/create-mat-view/#period-materialized-views)
to learn more on period materialized views.

## Initial refresh

As soon as a materialized view is created an asynchronous refresh is started. In
situations when this is not desirable, `DEFERRED` keyword can be specified along
with the refresh strategy:

```questdb-sql title="Deferred manual refresh"
CREATE MATERIALIZED VIEW trades_daily_prices
REFRESH MANUAL DEFERRED AS
...
```

The `DEFERRED` keyword can be specified for any refresh strategy. Refer to the
following
[documentation page](/docs/reference/sql/create-mat-view/#initial-refresh) to
learn more on the keyword.

#### SAMPLE BY

Materialized views are populated using `SAMPLE BY` or time-based `GROUP BY`
Expand Down Expand Up @@ -357,7 +405,7 @@ useful.
## Limitations

- Not all `SAMPLE BY` syntax is supported, for example, `FILL`.
- `INCREMENTAL` refresh is only triggered by inserts into the `base` table, not
- `IMMEDIATE` refresh is only triggered by inserts into the `base` table, not
join tables.

## LATEST ON materialized views
Expand Down Expand Up @@ -555,15 +603,6 @@ partitioning capabilities.

### Refresh mechanism

:::note

Currently, QuestDB only supports **incremental refresh** for materialized views.

Future releases will include additional refresh types, such as time-interval and
manual refreshes.

:::

Unlike regular views, which recompute their results at query time, materialized
views in QuestDB are incrementally refreshed as new data is added to the base
table. This approach ensures that only the **relevant time slices** of the view
Expand Down
4 changes: 2 additions & 2 deletions documentation/reference/function/meta.md
Original file line number Diff line number Diff line change
Expand Up @@ -518,8 +518,8 @@ materialized_views();

| view_name | refresh_type | base_table_name | last_refresh_start_timestamp | last_refresh_finish_timestamp | view_sql | view_table_dir_name | invalidation_reason | view_status | refresh_base_table_txn | base_table_txn | refresh_limit_value | refresh_limit_unit | timer_start | timer_interval_value | timer_interval_unit |
|------------------|--------------|-----------------|------------------------------|-------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|---------------------|-------------|------------------------|----------------|---------------------|--------------------|-------------|----------------------|---------------------|
| trades_OHLC_15m | incremental | trades | 2025-05-30T16:40:37.562421Z | 2025-05-30T16:40:37.568800Z | SELECT timestamp, symbol, first(price) AS open, max(price) as high, min(price) as low, last(price) AS close, sum(amount) AS volume FROM trades SAMPLE BY 15m | trades_OHLC_15m~27 | null | valid | 55141609 | 55141609 | 0 | null | null | 0 | null |
| trades_latest_1d | incremental | trades | 2025-05-30T16:40:37.554274Z | 2025-05-30T16:40:37.562049Z | SELECT timestamp, symbol, side, last(price) AS price, last(amount) AS amount, last(timestamp) as latest FROM trades SAMPLE BY 1d | trades_latest_1d~28 | null | valid | 55141609 | 55141609 | 0 | null | null | 0 | null |
| trades_OHLC_15m | immediate | trades | 2025-05-30T16:40:37.562421Z | 2025-05-30T16:40:37.568800Z | SELECT timestamp, symbol, first(price) AS open, max(price) as high, min(price) as low, last(price) AS close, sum(amount) AS volume FROM trades SAMPLE BY 15m | trades_OHLC_15m~27 | null | valid | 55141609 | 55141609 | 0 | null | null | 0 | null |
| trades_latest_1d | immediate | trades | 2025-05-30T16:40:37.554274Z | 2025-05-30T16:40:37.562049Z | SELECT timestamp, symbol, side, last(price) AS price, last(amount) AS amount, last(timestamp) as latest FROM trades SAMPLE BY 1d | trades_latest_1d~28 | null | valid | 55141609 | 55141609 | 0 | null | null | 0 | null |


## version/pg_catalog.version
Expand Down
31 changes: 0 additions & 31 deletions documentation/reference/sql/alter-mat-view-set-refresh-start.md

This file was deleted.

38 changes: 38 additions & 0 deletions documentation/reference/sql/alter-mat-view-set-refresh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
title: ALTER MATERIALIZED VIEW SET REFRESH
sidebar_label: SET REFRESH
description:
ALTER MATERIALIZED VIEW SET REFRESH SQL keyword reference documentation.
---

Changes a materialized view's refresh strategy and parameters.

## Syntax

![Flow chart showing the syntax of ALTER MATERIALIZED VIEW SET REFRESH command](/images/docs/diagrams/alterMatViewSetRefresh.svg)

## Description

Sometimes, the view's refresh strategy and its parameters may need to be changed.
Say, you may want to change the view to be timer refreshed instead of immediate
refresh.

The `REFRESH` follows the same format as [CREATE MATERIALIZED VIEW](/docs/reference/sql/create-mat-view/).

## Examples

```questdb-sql
ALTER MATERIALIZED VIEW trades_hourly_prices SET REFRESH EVERY '1h';
```

```questdb-sql
ALTER MATERIALIZED VIEW trades_hourly_prices SET REFRESH PERIOD (LENGTH 1d DELAY 1h);
```

```questdb-sql
ALTER MATERIALIZED VIEW trades_hourly_prices SET REFRESH IMMEDIATE;
```

```questdb-sql
ALTER MATERIALIZED VIEW trades_hourly_prices SET REFRESH MANUAL;
```
Loading