Skip to content

Commit c1937ea

Browse files
committed
pr review changes
1 parent cc37f15 commit c1937ea

File tree

9 files changed

+164
-289
lines changed

9 files changed

+164
-289
lines changed

doc/user/content/installation/_index.md

Lines changed: 0 additions & 169 deletions
Original file line numberDiff line numberDiff line change
@@ -16,175 +16,6 @@ aliases:
1616

1717
{{< include-md file="shared-content/self-managed/general-rules-for-upgrades.md" >}}
1818

19-
### Upgrade guides
20-
21-
The following upgrade guides are available:
22-
23-
| | Notes |
24-
| ------------- | -------|
25-
| [Upgrade on kind](/installation/install-on-local-kind/upgrade-on-local-kind/) |
26-
| [Upgrade on AWS](/installation/install-on-aws/upgrade-on-aws/) | Uses Materialize provided Terraform |
27-
| [Upgrade on Azure Kubernetes Service (AKS)](/installation/install-on-azure/upgrade-on-azure/) | Uses Materialize provided Terraform |
28-
| [Upgrade on Google Kubernetes Engine (GKE)](/installation/install-on-gcp/upgrade-on-gcp/) | Uses Materialize provided Terraform |
29-
30-
31-
### General notes for upgrades
32-
33-
The following provides some general notes for upgrades. For specific examples,
34-
see the [Upgrade guides](#upgrade-guides)
35-
36-
37-
#### Upgrading the Helm Chart and Kubernetes Operator
38-
39-
{{< important >}}
40-
41-
When upgrading Materialize, always upgrade the operator first.
42-
43-
{{</ important >}}
44-
45-
The Materialize Kubernetes operator is deployed via Helm and can be updated through standard Helm upgrade commands.
46-
47-
```shell
48-
helm upgrade my-materialize-operator materialize/misc/helm-charts/operator
49-
```
50-
51-
If you have custom values, make sure to include your values file:
52-
53-
```shell
54-
helm upgrade my-materialize-operator materialize/misc/helm-charts/operator -f my-values.yaml
55-
```
56-
57-
#### Upgrading Materialize Instances
58-
59-
In order to minimize unexpected downtime and avoid connection drops at critical
60-
periods for your application, changes are not immediately and automatically
61-
rolled out by the Operator. Instead, the upgrade process involves two steps:
62-
- First, staging spec changes to the Materialize custom resource.
63-
- Second, applying the changes via a `requestRollout`.
64-
65-
When upgrading your Materialize instances, you'll first want to update the `environmentdImageRef` field in the Materialize custom resource spec.
66-
67-
##### Updating the `environmentdImageRef`
68-
To find a compatible version with your currently deployed Materialize operator, check the `appVersion` in the Helm repository.
69-
70-
```shell
71-
helm list -n materialize
72-
```
73-
74-
Using the returned version, we can construct an image ref.
75-
We always recommend using the official Materialize image repository
76-
`docker.io/materialize/environmentd`.
77-
78-
```
79-
environmentdImageRef: docker.io/materialize/environmentd:v26.0.0
80-
```
81-
82-
The following is an example of how to patch the version.
83-
```shell
84-
# For version updates, first update the image reference
85-
kubectl patch materialize <instance-name> \
86-
-n <materialize-instance-namespace> \
87-
--type='merge' \
88-
-p "{\"spec\": {\"environmentdImageRef\": \"materialize/environmentd:v26.0.0\"}}"
89-
```
90-
91-
##### Applying the changes via `requestRollout`
92-
93-
To apply changes and kick off the Materialize instance upgrade, you must update the `requestRollout` field in the Materialize custom resource spec to a new UUID.
94-
Be sure to consult the [Rollout Configurations](#rollout-configuration) to ensure you've selected the correct rollout behavior.
95-
```shell
96-
# Then trigger the rollout with a new UUID
97-
kubectl patch materialize <instance-name> \
98-
-n <materialize-instance-namespace> \
99-
--type='merge' \
100-
-p "{\"spec\": {\"requestRollout\": \"$(uuidgen)\"}}"
101-
```
102-
103-
104-
It is possible to combine both operations in a single command if preferred:
105-
106-
```shell
107-
kubectl patch materialize <instance-name> \
108-
-n materialize-environment \
109-
--type='merge' \
110-
-p "{\"spec\": {\"environmentdImageRef\": \"materialize/environmentd:v26.0.0\", \"requestRollout\": \"$(uuidgen)\"}}"
111-
```
112-
113-
##### Using YAML Definition
114-
115-
Alternatively, you can update your Materialize custom resource definition directly:
116-
117-
```yaml
118-
apiVersion: materialize.cloud/v1alpha1
119-
kind: Materialize
120-
metadata:
121-
name: 12345678-1234-1234-1234-123456789012
122-
namespace: materialize-environment
123-
spec:
124-
environmentdImageRef: materialize/environmentd:v26.0.0 # Update version as needed
125-
requestRollout: 22222222-2222-2222-2222-222222222222 # Generate new UUID
126-
forceRollout: 33333333-3333-3333-3333-333333333333 # Optional: for forced rollouts
127-
inPlaceRollout: false # In Place rollout is deprecated and ignored. Please use rolloutStrategy
128-
rolloutStrategy: WaitUntilReady # The mechanism to use when rolling out the new version. Can be WaitUntilReady or ImmediatelyPromoteCausingDowntime
129-
backendSecretName: materialize-backend
130-
```
131-
132-
Apply the updated definition:
133-
134-
```shell
135-
kubectl apply -f materialize.yaml
136-
```
137-
138-
#### Rollout Configuration
139-
140-
##### Forced Rollouts
141-
142-
If you need to force a rollout even when there are no changes to the instance:
143-
144-
```shell
145-
kubectl patch materialize <instance-name> \
146-
-n materialize-environment \
147-
--type='merge' \
148-
-p "{\"spec\": {\"requestRollout\": \"$(uuidgen)\", \"forceRollout\": \"$(uuidgen)\"}}"
149-
```
150-
151-
##### Rollout Strategies
152-
153-
The behavior of the new version rollout follows your `rolloutStrategy` setting:
154-
155-
`WaitUntilReady` (default):
156-
157-
New instances are created and all dataflows are determined to be ready before cutover and terminating the old version, temporarily requiring twice the resources during the transition.
158-
159-
`ImmediatelyPromoteCausingDowntime`:
160-
161-
Tears down the prior version before creating and promoting the new version. This causes downtime equal to the duration it takes for dataflows to hydrate, but does not require additional resources.
162-
163-
##### In Place Rollout
164-
165-
The `inPlaceRollout` setting has been deprecated and will be ignored.
166-
167-
### Verifying the Upgrade
168-
169-
After initiating the rollout, you can monitor the status field of the Materialize custom resource to check on the upgrade.
170-
171-
```shell
172-
# Watch the status of your Materialize environment
173-
kubectl get materialize -n materialize-environment -w
174-
175-
# Check the logs of the operator
176-
kubectl logs -l app.kubernetes.io/name=materialize-operator -n materialize
177-
```
178-
### Version Specific Upgrade Notes
179-
180-
#### Upgrading to `v26.0`
181-
182-
{{< include-md file="shared-content/self-managed/upgrade-notes/v26.0.md" >}}
183-
184-
#### Upgrading between minor versions less than `v26`
185-
- Prior to `v26`, you must upgrade at most one minor version at a time. For
186-
example, upgrading from `v25.1.5` to `v25.2.15` is permitted.
187-
18819
## See also
18920

19021
- [Materialize Operator Configuration](/installation/configuration/)

doc/user/content/installation/install-on-aws/appendix-deployment-guidelines.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,14 +15,14 @@ menu:
1515
As a general guideline, we recommend:
1616

1717
- ARM-based CPU
18-
- A 1:8 ratio of vCPU to GiB memory is recommended.
19-
- When using swap, it is recommended to use a 8:1 ratio of GiB local instance storage to GiB Ram.
18+
- A 1:8 ratio of vCPU to GiB memory.
19+
- When using swap, use a 8:1 ratio of GiB local instance storage to GiB memory.
2020

2121
{{% self-managed/aws-recommended-instances %}}
2222

2323
## Locally-attached NVMe storage
2424

25-
Configuring swap on nodes to using locally-attached NVMe storage allows
25+
Configuring swap on nodes to use locally-attached NVMe storage allows
2626
Materialize to spill to disk when operating on datasets larger than main memory.
2727
This setup can provide significant cost savings and provides a more graceful
2828
degradation rather than OOMing. Network-attached storage (like EBS volumes) can
@@ -45,7 +45,7 @@ With this change, the Terraform:
4545
See [Upgrade Notes](https://github.com/MaterializeInc/terraform-aws-materialize?tab=readme-ov-file#v061).
4646

4747
{{< note >}}
48-
If deploying `v25.2` Materialize clusters will not automatically use swap unless they are configured with a `memory_request` less than their `memory_limit`. In `v26` this will be handled automatically.
48+
If deploying `v25.2`, Materialize clusters will not automatically use swap unless they are configured with a `memory_request` less than their `memory_limit`. In `v26`, this will be handled automatically.
4949
{{< /note >}}
5050

5151
## TLS

doc/user/content/installation/install-on-aws/terraform-module/_index.md

Lines changed: 47 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -9,17 +9,14 @@ menu:
99
---
1010

1111
Materialize provides a set of modular Terraform modules that can be used to
12-
deploy all services required for a production ready Materialize database.
12+
deploy all services required for Materialize to run on AWS.
1313
The module is intended to provide a simple set of examples on how to deploy
14-
materialize. It can be used as is or modules can be taken from the example and
14+
Materialize. It can be used as is or modules can be taken from the example and
1515
integrated with existing DevOps tooling.
1616

17-
The repository can be found at:
18-
19-
***[Materialize Terraform Self-Managed AWS](https://github.com/MaterializeInc/materialize-terraform-self-managed/tree/main/aws)***
20-
21-
Please see the [top level](https://github.com/MaterializeInc/materialize-terraform-self-managed/tree/main) and [cloud specific](https://github.com/MaterializeInc/materialize-terraform-self-managed/tree/main/aws) documentation for a full understanding
22-
of the module structure and customizations.
17+
For details on the module structure and customization, see:
18+
* [top level](https://github.com/MaterializeInc/materialize-terraform-self-managed/tree/main)
19+
* [cloud specific](https://github.com/MaterializeInc/materialize-terraform-self-managed/tree/main/aws)
2320

2421
Also check out the [AWS deployment guide](/installation/install-on-aws/appendix-deployment-guidelines/) for details on recommended instance sizing and configuration.
2522

@@ -43,7 +40,7 @@ Also check out the [AWS deployment guide](/installation/install-on-aws/appendix-
4340

4441
#### License key
4542

46-
{{< include-md file="shared-content/license-key-required.md" >}}
43+
{{< yaml-table data="self_managed/license_key" >}}
4744

4845
---
4946

@@ -64,40 +61,56 @@ cd materialize-terraform-self-managed/aws/examples/simple
6461
This example provisions the following infrastructure:
6562

6663
### Networking
67-
- **VPC**: 10.0.0.0/16 with DNS hostnames and support enabled
68-
- **Subnets**: 3 private subnets (10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24) and 3 public subnets (10.0.101.0/24, 10.0.102.0/24, 10.0.103.0/24) across availability zones us-east-1a, us-east-1b, us-east-1c
69-
- **NAT Gateway**: Single NAT Gateway for all private subnets
70-
- **Internet Gateway**: For public subnet connectivity
64+
65+
| Resource | Description |
66+
|----------|-------------|
67+
| VPC | 10.0.0.0/16 with DNS hostnames and support enabled |
68+
| Subnets | 3 private subnets (10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24) and 3 public subnets (10.0.101.0/24, 10.0.102.0/24, 10.0.103.0/24) across availability zones us-east-1a, us-east-1b, us-east-1c |
69+
| NAT Gateway | Single NAT Gateway for all private subnets |
70+
| Internet Gateway | For public subnet connectivity |
7171

7272
### Compute
73-
- **EKS Cluster**: Version 1.32 with CloudWatch logging (API, audit)
74-
- **Base Node Group**: 2 nodes (t4g.medium) for Karpenter and CoreDNS
75-
- **Karpenter**: Auto-scaling controller with two node classes:
76-
- Generic nodepool: t4g.xlarge instances for general workloads
77-
- Materialize nodepool: r7gd.2xlarge instances with swap enabled and dedicated taints to run materialize instance workloads.
73+
74+
| Resource | Description |
75+
|----------|-------------|
76+
| EKS Cluster | Version 1.32 with CloudWatch logging (API, audit) |
77+
| Base Node Group | 2 nodes (t4g.medium) for Karpenter and CoreDNS |
78+
| Karpenter | Auto-scaling controller with two node classes: Generic nodepool (t4g.xlarge instances for general workloads) and Materialize nodepool (r7gd.2xlarge instances with swap enabled and dedicated taints to run materialize instance workloads) |
7879

7980
### Database
80-
- **RDS PostgreSQL**: Version 15, db.t3.large instance
81-
- **Storage**: 50GB allocated, autoscaling up to 100GB
82-
- **Deployment**: Single-AZ (non-production configuration)
83-
- **Backups**: 7-day retention
84-
- **Security**: Dedicated security group with access from EKS cluster and nodes
81+
82+
| Resource | Description |
83+
|----------|-------------|
84+
| RDS PostgreSQL | Version 15, db.t3.large instance |
85+
| Storage | 50GB allocated, autoscaling up to 100GB |
86+
| Deployment | Single-AZ (non-production configuration) |
87+
| Backups | 7-day retention |
88+
| Security | Dedicated security group with access from EKS cluster and nodes |
8589

8690
### Storage
87-
- **S3 Bucket**: Dedicated bucket for Materialize persistence
88-
- **Encryption**: Disabled (for testing; enable in production)
89-
- **Versioning**: Disabled (for testing; enable in production)
90-
- **IAM Role**: IRSA role for Kubernetes service account access
91+
92+
| Resource | Description |
93+
|----------|-------------|
94+
| S3 Bucket | Dedicated bucket for Materialize persistence |
95+
| Encryption | Disabled (for testing; enable in production) |
96+
| Versioning | Disabled (for testing; enable in production) |
97+
| IAM Role | IRSA role for Kubernetes service account access |
9198

9299
### Kubernetes Add-ons
93-
- **AWS Load Balancer Controller**: For managing Network Load Balancers
94-
- **cert-manager**: Certificate management controller for Kubernetes that automates TLS certificate provisioning and renewal
95-
- **Self-signed ClusterIssuer**: Provides self-signed TLS certificates for Materialize instance internal communication (balancerd, console). Used by the Materialize instance for secure inter-component communication.
100+
101+
| Resource | Description |
102+
|----------|-------------|
103+
| AWS Load Balancer Controller | For managing Network Load Balancers |
104+
| cert-manager | Certificate management controller for Kubernetes that automates TLS certificate provisioning and renewal |
105+
| Self-signed ClusterIssuer | Provides self-signed TLS certificates for Materialize instance internal communication (balancerd, console). Used by the Materialize instance for secure inter-component communication. |
96106

97107
### Materialize
98-
- **Operator**: Materialize Kubernetes operator
99-
- **Instance**: Single Materialize instance in `materialize-environment` namespace
100-
- **Network Load Balancer**: Dedicated internal NLB for Materialize access (ports 6875, 6876, 8080)
108+
109+
| Resource | Description |
110+
|----------|-------------|
111+
| Operator | Materialize Kubernetes operator |
112+
| Instance | Single Materialize instance in `materialize-environment` namespace |
113+
| Network Load Balancer | Dedicated internal NLB for Materialize access (ports 6875, 6876, 8080) |
101114

102115
---
103116

@@ -111,7 +124,7 @@ Before running Terraform, create a `terraform.tfvars` file with the following va
111124
name_prefix = "simple-demo"
112125
aws_region = "us-east-1"
113126
aws_profile = "your-aws-profile"
114-
license_key = "your-materialize-license-key" # Get from https://materialize.com/self-managed/
127+
license_key = "your-materialize-license-key"
115128
tags = {
116129
environment = "demo"
117130
}

doc/user/content/installation/install-on-azure/appendix-deployment-guidelines.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,8 @@ menu:
1313
As a general guideline, we recommend:
1414

1515
- ARM-based CPU
16-
- A 1:8 ratio of vCPU to GiB memory is recommended.
17-
- When using swap, it is recommended to use a 8:1 ratio of GiB local instance storage to GiB Ram.
16+
- A 1:8 ratio of vCPU to GiB memory.
17+
- When using swap, use a 8:1 ratio of GiB local instance storage to GiB memory.
1818

1919
### Recommended Azure VM Types with Local NVMe Disks
2020

@@ -39,7 +39,7 @@ when the VM is stopped or deleted.
3939

4040
## Locally-attached NVMe storage
4141

42-
Configuring swap on nodes to using locally-attached NVMe storage allows
42+
Configuring swap on nodes to use locally-attached NVMe storage allows
4343
Materialize to spill to disk when operating on datasets larger than main memory.
4444
This setup can provide significant cost savings and provides a more graceful
4545
degradation rather than OOMing. Network-attached storage (like EBS volumes) can
@@ -62,7 +62,7 @@ With this change, the Terraform:
6262
See [Upgrade Notes](https://github.com/MaterializeInc/terraform-azurerm-materialize?tab=readme-ov-file#v061).
6363

6464
{{< note >}}
65-
If deploying `v25.2` Materialize clusters will not automatically use swap unless they are configured with a `memory_request` less than their `memory_limit`. In `v26` this will be handled automatically.
65+
If deploying `v25.2`, Materialize clusters will not automatically use swap unless they are configured with a `memory_request` less than their `memory_limit`. In `v26`, this will be handled automatically.
6666
{{< /note >}}
6767

6868
## Recommended Azure Blob Storage

0 commit comments

Comments
 (0)