Skip to content

Commit c546ce5

Browse files
authored
first-release (#119)
* first-release Signed-off-by: Michael Kalantar <[email protected]> * spelling Signed-off-by: Michael Kalantar <[email protected]> * updates to first release Signed-off-by: Michael Kalantar <[email protected]> * updates to wordlist Signed-off-by: Michael Kalantar <[email protected]> * updates to wordlist Signed-off-by: Michael Kalantar <[email protected]> * spelling Signed-off-by: Michael Kalantar <[email protected]> * more release tutorials Signed-off-by: Michael Kalantar <[email protected]> * mm-canary tutorial Signed-off-by: Michael Kalantar <[email protected]> * more release docs Signed-off-by: Michael Kalantar <[email protected]> * more release docs Signed-off-by: Michael Kalantar <[email protected]> * refine first release Signed-off-by: Michael Kalantar <[email protected]> * replace routing with release Signed-off-by: Michael Kalantar <[email protected]> * gateway sample Signed-off-by: Michael Kalantar <[email protected]> * gateway sample Signed-off-by: Michael Kalantar <[email protected]> * update release tutorials Signed-off-by: Michael Kalantar <[email protected]> * spelling Signed-off-by: Michael Kalantar <[email protected]> * update references to release chart Signed-off-by: Michael Kalantar <[email protected]> * applied review feedback Signed-off-by: Michael Kalantar <[email protected]> * spelling Signed-off-by: Michael Kalantar <[email protected]> * undo mkdocs changes Signed-off-by: Michael Kalantar <[email protected]> * update mkdocs-material Signed-off-by: Michael Kalantar <[email protected]> * should be canary Signed-off-by: Michael Kalantar <[email protected]> --------- Signed-off-by: Michael Kalantar <[email protected]>
1 parent 6f04b40 commit c546ce5

32 files changed

+1260
-5827
lines changed

.github/wordlist.txt

+4-1
Original file line numberDiff line numberDiff line change
@@ -190,4 +190,7 @@ Hyperparameter
190190
LLM
191191
odh
192192
contrib
193-
opendatahub
193+
opendatahub
194+
serviceSpecification
195+
deploymentSpecification
196+
kennethreitz

docs/getting-started/concepts.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Iter8 provides three inter-related components to support the above use-cases.
2525

2626
The following picture illustrates a blue-green rollout scenario that is orchestrated by this controller.
2727

28-
![Blue-green](../tutorials/integrations/kserve-mm/images/blue-green.png)
28+
![Blue-green](../tutorials/images/blue-green.png)
2929

3030
As part of the dynamic reconfiguration of route resources, the Iter8 controller also checks for readiness (for e.g., in KServe ModelMesh), availability (for e.g., in Kubernetes deployments) and other relevant status conditions before configuring traffic splits to candidate versions. Similarly, before candidate versions are deleted, the Iter8 controller uses finalizers to first ensure that all traffic flows to the primary version of the ML model. This makes for a very high-degree of reliability and zero-downtime/loss-less rollouts of new app/ML model versions. Users do not get this level of reliability out-of-the-box with a vanilla service mesh.
3131

@@ -39,7 +39,7 @@ Iter8 provides three inter-related components to support the above use-cases.
3939

4040
The following picture illustrates the use of the SDK for A/B testing.
4141

42-
![A/B testing](images/abn.png)
42+
![A/B testing](../tutorials/images/abn.png)
4343

4444
Iter8's SDK is designed to handle user stickiness, collection of business metrics, and decoupling of front-end and back-end releases processes during A/B/n testing.
4545

docs/getting-started/first-abn.md

+41-59
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ template: main.html
66

77
This tutorial describes how to do A/B testing of a backend component using the [Iter8 SDK](../user-guide/topics/ab_testing.md).
88

9-
![A/B/n testing](images/abn.png)
9+
![A/B/n testing](../tutorials/images/abn.png)
1010

1111
***
1212

@@ -24,7 +24,7 @@ This tutorial describes how to do A/B testing of a backend component using the [
2424

2525
## Deploy the sample application
2626

27-
A sample application using the Iter8 SDK is provided. Deploy both the frontend and backend components of this application as described in each tab:
27+
A simple sample two-tier application using the Iter8 SDK is provided. Note that only the frontend component uses the Iter8 SDK. Deploy both the frontend and backend components of this application as described in each tab:
2828

2929
=== "frontend"
3030
Install the frontend component using an implementation in the language of your choice:
@@ -44,32 +44,20 @@ A sample application using the Iter8 SDK is provided. Deploy both the frontend a
4444
The frontend component is implemented to call `Lookup()` before each call to the backend component. The frontend component uses the returned version number to route the request to the recommended version of the backend component.
4545

4646
=== "backend"
47-
Deploy an initial version of the *backend* component:
47+
Release an initial version of the backend named `backend`:
4848

4949
```shell
50-
kubectl create deployment backend --image=iter8/abn-sample-backend:0.17-v1
51-
kubectl label deployment backend iter8.tools/watch="true"
52-
53-
kubectl expose deployment backend --name=backend --port=8091
50+
cat <<EOF | helm upgrade --install backend --repo https://iter8-tools.github.io/iter8 release --version 0.18 -f -
51+
environment: deployment
52+
application:
53+
port: 8091
54+
versions:
55+
- metadata:
56+
name: backend
57+
image: iter8/abn-sample-backend:0.17-v1
58+
EOF
5459
```
5560

56-
## Describe the application
57-
58-
In order to support `Lookup()`, Iter8 needs to know what the application component versions look like. A _routemap_ is created to do this. A routemap contains a description of each version of an application and may contain [routing templates](../user-guide/topics/routemap.md). To create the routemap:
59-
60-
```shell
61-
cat <<EOF | helm template routing --repo https://iter8-tools.github.io/iter8 routing-actions --version 0.18 -f - | kubectl apply -f -
62-
appType: deployment
63-
appName: backend
64-
action: initialize
65-
appVersions:
66-
- name: backend
67-
- name: backend-candidate-1
68-
EOF
69-
```
70-
71-
The `initialize` action (with strategy `none`) creates a routemap that only defines the resources that make up each version of the application. In this case, two versions: `backend` and `backend-candidate-1`. Each version is comprised of a `Service` and a `Deployment`. Iter8 uses this information to identify when any of the versions of the application are available. It can then respond appropriately to `Lookup()` requests.
72-
7361
## Generate load
7462

7563
In one shell, port-forward requests to the frontend component:
@@ -83,17 +71,25 @@ In another shell, run a script to generate load from multiple users:
8371

8472
## Deploy candidate
8573

86-
Deploy the candidate version of the *backend* component, naming it `backend-candidate-1`.
74+
A candidate version of the *backend* component can be deployed simply by adding a second version to the list of versions:
8775

8876
```shell
89-
kubectl create deployment backend-candidate-1 --image=iter8/abn-sample-backend:0.17-v2
90-
kubectl label deployment backend-candidate-1 iter8.tools/watch="true"
91-
92-
kubectl expose deployment backend-candidate-1 --name=backend-candidate-1 --port=8091
77+
cat <<EOF | helm upgrade --install backend --repo https://iter8-tools.github.io/iter8 release --version 0.18 -f -
78+
environment: deployment
79+
application:
80+
port: 8091
81+
versions:
82+
- metadata:
83+
name: backend
84+
image: iter8/abn-sample-backend:0.17-v1
85+
- metadata:
86+
name: backend-candidate-1
87+
image: iter8/abn-sample-backend:0.17-v2
88+
EOF
9389
```
9490

95-
Until the candidate version is ready; that is, until all expected resources are deployed and available, calls to `Lookup()` will return only the version number `0`; the existing version.
96-
Once the candidate version is ready, `Lookup()` will return both version numbers (`0` and `1`) so that requests can be distributed across versions.
91+
While the candidate version is deploying, `Lookup()` will return only the version index number `0`; that is, the first, or primary, version of the model.
92+
Once the candidate version is ready, `Lookup()` will return both `0` and `1`, the indices of both versions, so that requests can be distributed across both versions.
9793

9894
## Compare versions using Grafana
9995

@@ -114,49 +110,35 @@ Open Grafana in a browser by going to [http://localhost:3000](http://localhost:3
114110

115111
The Iter8 dashboard allows you to compare the behavior of the two versions of the backend component against each other and select a winner. Since user requests are being sent by the load generation script, the values in the report may change over time. The Iter8 dashboard will look like the following:
116112

117-
![A/B dashboard](images/dashboard.png)
113+
![A/B dashboard](../tutorials/images/abnDashboard.png)
118114

119115
Once you identify a winner, it can be promoted, and the candidate version deleted.
120116

121117
## Promote candidate
122118

123-
To promote the candidate version (`backend-candidate-1`), first update the primary version, `backend`, using the new image. You can also overwrite any metadata describing the version.
119+
To promote the candidate version (`backend-candidate-1`), re-release the application, updating the image of the primary (the first) version to use the image of the candidate version and remove the candidate version:
124120

125121
```shell
126-
kubectl set image deployment/backend abn-sample-backend=iter8/abn-sample-backend:0.17-v2
127-
```
128-
129-
Finally, delete the candidate version:
130-
131-
```shell
132-
kubectl delete svc/backend-candidate-1 deploy/backend-candidate-1
122+
cat <<EOF | helm upgrade --install backend --repo https://iter8-tools.github.io/iter8 release --version 0.18 -f -
123+
environment: deployment
124+
application:
125+
port: 8091
126+
versions:
127+
- metadata:
128+
name: backend
129+
image: iter8/abn-sample-backend:0.17-v2
130+
EOF
133131
```
134132

135-
Calls to `Lookup()` will now recommend that all traffic be sent to the primary version `backend` (currently serving the promoted version of the code).
133+
Calls to `Lookup()` will now recommend that all traffic be sent to the new primary version `backend` (currently serving the promoted version of the code).
136134

137135
## Cleanup
138136

139137
Delete the sample application:
140138

141139
```shell
142-
kubectl delete \
143-
svc/frontend deploy/frontend \
144-
svc/backend deploy/backend \
145-
svc/backend-candidate-1 deploy/backend-candidate-1
146-
```
147-
148-
Delete the application routemap:
149-
150-
```shell
151-
cat <<EOF | helm template routing --repo https://iter8-tools.github.io/iter8 routing-actions --version 0.18 -f - | kubectl delete -f -
152-
appType: deployment
153-
appName: backend
154-
action: initialize
155-
appVersions:
156-
- name: backend
157-
- name: backend-candidate-1
158-
EOF
159-
140+
kubectl delete svc/frontend deploy/frontend
141+
helm delete backend
160142
```
161143

162144
Uninstall the Iter8 controller:

docs/getting-started/first-release.md

+192
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
---
2+
template: main.html
3+
---
4+
5+
# Your first blue-green release
6+
7+
This tutorial shows how Iter8 can be used to release a basic Kubernetes application using a blue-green rollout strategy.
8+
In a blue-green rollout, a percentage of requests are directed to a candidate version of the model.
9+
This percentage can be changed over time.
10+
The user declaratively describes the desired application state at any given moment.
11+
An Iter8 `release` chart assists users who describe the application state at any given moment.
12+
The chart provides the configuration needed for Iter8 to automatically deploy application versions and configure the routing to implement the blue-green rollout strategy.
13+
14+
![Blue-green rollout](../tutorials/images/blue-green.png)
15+
16+
???+ warning "Before you begin"
17+
1. Ensure that you have a Kubernetes cluster and the [`kubectl`](https://kubernetes.io/docs/reference/kubectl/) and [`helm`](https://helm.sh/) CLIs. You can create a local Kubernetes cluster using tools like [Kind](https://kind.sigs.k8s.io/) or [Minikube](https://minikube.sigs.k8s.io/docs/).
18+
2. Install [Istio](https://istio.io). It suffices to install the [demo profile](https://istio.io/latest/docs/setup/getting-started/), for example by using:
19+
```shell
20+
istioctl install --set profile=demo -y
21+
```
22+
23+
## Install the Iter8 controller
24+
25+
--8<-- "docs/getting-started/install.md"
26+
27+
## Deploy initial version
28+
29+
Deploy the initial version of the application using the Iter8 `release` chart by identifying the environment into which it should be deployed, a list of the versions to be deployed (only one here), and the rollout strategy to be used:
30+
31+
```shell
32+
cat <<EOF | helm upgrade --install httpbin --repo https://iter8-tools.github.io/iter8 release --version 0.18 -f -
33+
environment: deployment-istio
34+
application:
35+
versions:
36+
- metadata:
37+
labels:
38+
app.kubernetes.io/version: v0
39+
image: kennethreitz/httpbin
40+
strategy: blue-green
41+
EOF
42+
```
43+
44+
??? note "What happens?"
45+
Because `environment` is set to `deployment-istio`, a `Deployment` and a `Service` object are created.
46+
- The namespace `default` is inherited from the Helm release namespace since it is not specified in the version or in `application.metadata`.
47+
- The name `httpbin-0` is derived from the Helm release name since it is not specified in the version or in `application.metadata`. The name is derived by appending the index of the version in the list of versions; `-0` in this case.
48+
- Alternatively, a `deploymentSpecification` and/or a `serviceSpecification` could have been specified.
49+
50+
To support routing, a `Service` (of type `ExternalName`) named `default/httpbin` pointing at the Istio gateway, `istio-ingressgateway.istio-system`, is deployed. The name is the Helm release name since it not specified in `application.metadata`. Further, an Iter8 [routemap](../user-guide/topics/routemap.md) is created. Finally, to support the blue-green rollout, a `ConfigMap` (`httpbin-0-weight-config`) is created to be used to manage the proportion of traffic sent to this version.
51+
52+
Once the application components are ready, the Iter8 controller automatically configures the routing by creating an Istio `VirtualService`. It is configured to route all traffic to the only deployed version, `httpbin-0`.
53+
54+
### Verify routing
55+
56+
You can verify the routing configuration by inspecting the `VirtualService`:
57+
58+
```shell
59+
kubectl get virtualservice httpbin -o yaml
60+
```
61+
62+
You can also send requests from a pod within the cluster:
63+
64+
1. Create a `sleep` pod in the cluster from which requests can be made:
65+
```shell
66+
curl -s https://raw.githubusercontent.com/iter8-tools/docs/v0.17.3/samples/kserve-serving/sleep.sh | sh -
67+
```
68+
69+
2. Exec into the sleep pod:
70+
```shell
71+
kubectl exec --stdin --tty "$(kubectl get pod --sort-by={metadata.creationTimestamp} -l app=sleep -o jsonpath={.items..metadata.name} | rev | cut -d' ' -f 1 | rev)" -c sleep -- /bin/sh
72+
```
73+
74+
3. Send requests:
75+
```shell
76+
curl httpbin.default -s -D - | grep -e '^HTTP' -e app-version
77+
```
78+
79+
The output includes the success of the request (the HTTP return code) and the version of the application that responded (the `app-version` response header). For example:
80+
81+
```
82+
HTTP/1.1 200 OK
83+
app-version: httpbin-0
84+
```
85+
86+
??? note "To send requests from outside the cluster"
87+
To configure the release for traffic from outside the cluster, a suitable Istio `Gateway` is required. For example, this [sample gateway](https://raw.githubusercontent.com/kalantar/docs/release/samples/iter8-sample-gateway.yaml). When using the Iter8 `release` chart, set the `gateway` field to the name of your `Gateway`. Finally, to send traffic:
88+
89+
(a) In a separate terminal, port-forward the ingress gateway:
90+
```shell
91+
kubectl -n istio-system port-forward svc/istio-ingressgateway 8080:80
92+
```
93+
(b) Send requests using the `Host` header:
94+
```shell
95+
curl -H 'Host: httpbin.default' localhost:8080 -s -D - | grep -e '^HTTP' -e app-version
96+
```
97+
98+
## Deploy candidate
99+
100+
A candidate can deployed by simply adding a second version to the list of versions comprising the application:
101+
102+
```shell
103+
cat <<EOF | helm upgrade --install httpbin --repo https://iter8-tools.github.io/iter8 release --version 0.18 -f -
104+
environment: deployment-istio
105+
application:
106+
versions:
107+
- metadata:
108+
labels:
109+
app.kubernetes.io/version: v0
110+
image: kennethreitz/httpbin
111+
- metadata:
112+
labels:
113+
app.kubernetes.io/version: v1
114+
image: kennethreitz/httpbin
115+
strategy: blue-green
116+
EOF
117+
```
118+
119+
??? note "About the candidate version"
120+
In this tutorial, the candidate image is the same as the one for the primary version. In a real world example, it would be different. The version label (`app.kubernetes.io/version`) can be used to distinguish between versions.
121+
122+
When the second version is deployed and ready, the Iter8 controller automatically reconfigures the routing; the `VirtualService` is updated to distribute traffic between versions based on the weights.
123+
124+
### Verify routing
125+
126+
You can verify the routing configuration by inspecting the `VirtualService` and/or by sending requests as described above. Requests will now be handled equally by both versions.
127+
128+
## Modify weights (optional)
129+
130+
To modify the request distribution between the versions, add a `weight` to each version. The weights are relative to each other.
131+
132+
```shell
133+
cat <<EOF | helm upgrade --install httpbin --repo https://iter8-tools.github.io/iter8 release --version 0.18 -f -
134+
environment: deployment-istio
135+
application:
136+
versions:
137+
- metadata:
138+
labels:
139+
app.kubernetes.io/version: v0
140+
image: kennethreitz/httpbin
141+
weight: 30
142+
- metadata:
143+
labels:
144+
app.kubernetes.io/version: v1
145+
image: kennethreitz/httpbin
146+
weight: 70
147+
strategy: blue-green
148+
EOF
149+
```
150+
151+
Iter8 automatically reconfigures the routing to distribute traffic between the versions based on the new weights.
152+
153+
### Verify routing
154+
155+
You can verify the routing configuration by inspecting the `VirtualService` and/or by sending requests as described above. 70 percent of requests will now be handled by the candidate version; the remaining 30 percent by the primary version.
156+
157+
## Promote candidate
158+
159+
The candidate can be promoted by redefining the primary version and removing the candidate:
160+
161+
```shell
162+
cat <<EOF | helm upgrade --install httpbin --repo https://iter8-tools.github.io/iter8 release --version 0.18 -f -
163+
environment: deployment-istio
164+
application:
165+
versions:
166+
- metadata:
167+
labels:
168+
app.kubernetes.io/version: v1
169+
image: kennethreitz/httpbin
170+
strategy: blue-green
171+
EOF
172+
```
173+
??? note "What is different?"
174+
The version label (`app.kubernetes.io/version`) of the primary version was updated. In a real world example, the image would also have been updated (with that from the candidate version).
175+
176+
Once the (reconfigured) primary version ready, the Iter8 controller will automatically reconfigure the routing to send all requests to it.
177+
178+
### Verify routing
179+
180+
You can verify the routing configuration by inspecting the `VirtualService` and/or by sending requests as described above. They will all be handled by the primary version.
181+
182+
## Cleanup
183+
184+
Delete the application and its routing configuration:
185+
186+
```shell
187+
helm delete httpbin
188+
```
189+
190+
Uninstall Iter8 controller:
191+
192+
--8<-- "docs/getting-started/uninstall.md"

0 commit comments

Comments
 (0)