You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/getting-started/advantages.md
+12-7
Original file line number
Diff line number
Diff line change
@@ -3,19 +3,24 @@ template: main.html
3
3
---
4
4
5
5
# Advantages
6
-
First, Iter8 has no restrictions on the types of resources that make up a version of an application. This includes custom resources; that is, those defined by custom resource definitions (CRDs). Because the set of resources that comprise a version is declarative, it is easy to [extend](../user-guide/controller/extensions.md) Iter8 to work with new resource types. This same extension mechanism also allows Iter8 to be used with any service mesh.
7
6
8
-
Second, the Iter8 client SDK addresses a key challenge to [A/B/n testing](../user-guide/abn/about.md): the decoupling of the front-end release process from that of the back-end. Iter8 allows the front-end to reliably associate business metrics with the contributing version of the back-end.
7
+
## Use any resources
9
8
10
-
Finally, Iter8 simplifies performance testing by reducing the set up time needed to start testing. Tests can be easily specified as a sequence of [easily configured tasks](../user-guide/performance/parameters.md). Further, there is no need to setup and configure an external metrics database -- Iter8 captures the metrics data and provides a REST API, allowing it to be visualized and evaluated in Grafana.
9
+
Iter8 allows an application to be composed of resources of any type, including custom resources defined by custom resource definitions (CRDs). In Iter8, the set of resources that make up an application version is declarative making it easy to extend for new resource types. The same extension mechanism also allows Iter8 to be used with any service mesh or ingress.
10
+
11
+
## A/B/n testing of backend components
12
+
13
+
Using the Iter8 SDK enables frontend application components to reliably associate business metrics with the contributing versions of the backend. This addresses a key challenge of [A/B/n testing](../user-guide/abn/about.md) of backend application components/ML models.
14
+
15
+
## Simplified performance testing
16
+
17
+
Iter8 makes it easy to start running performance tests. Tests are composed with [easily configurable tasks](../user-guide/performance/parameters.md). Furthermore, there is no need to setup and configure an external metrics database -- Iter8 captures the metrics data and provides a REST API to access it, allowing it to be visualized and evaluated in Grafana.
11
18
12
19
# Comparison to other tools
13
20
14
-
[Flagger](https://flagger.app/) and [Argo Rollouts](https://argo-rollouts.readthedocs.io/en/stable/) share similarities with Iter8.
15
-
Both provide support advanced application rollout on Kubernetes with blue-green and canary analysis. They work with many service meshes and ingress products to provide this support.
16
-
Users specify the desired rollout using a Kubernetes custom resource.
21
+
[Flagger](https://flagger.app/) and [Argo Rollouts](https://argo-rollouts.readthedocs.io/en/stable/) share similarities with Iter8. Both provide support for advanced application rollout on Kubernetes with blue-green and canary analysis. They work with many service meshes and ingress products to provide this support. Users specify the desired rollout using a Kubernetes custom resource.
17
22
18
-
Iter8 was heavily inspired by both projects. However, Iter8 differs in several regards. For example, with Iter8:
23
+
Iter8 is inspired by both projects. However, Iter8 differs in several regards. For example, with Iter8:
19
24
20
25
- Applications can be composed of any resource type. For example, it works with machine learning applications built using KServe `InferenceService` resources out of the box. To do so, Iter8 allows the user to specify the resources being deployed as part of the specification of the rollout instead of assuming a particular pattern.
Copy file name to clipboardExpand all lines: docs/tutorials/blue-green.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,6 @@
2
2
template: main.html
3
3
---
4
4
5
-
# Blue-green rollout
5
+
# Blue-green release
6
6
7
-
[Your first automated release](../getting-started/first-release.md) describes how to implement a blue-green rollout of a Kubernetes application.
7
+
[Your first automated release](../getting-started/first-release.md) describes how to implement a progressive blue-green release of a Kubernetes application.
Like progressive release, A/B/n tests use the Iter8 `release` chart. Extending this chart for a new resource type is similar to [extending it for progressive release](../progressive-release/extension.md). The key difference is in the definition of the [routemap](../routemap.md). We briefly describe how to extend the chart for an [Knative](https://knative.dev/docs/) application.
8
+
9
+
## Example (Knative Service)
10
+
11
+
For example, to extend the chart to support A/B/n testing for Knative services, the following files could be added:
12
+
13
+
-`_knative-istio.tpl` - wrapper for all objects that should be deployed
14
+
-`_knative-istio.version.ksvc.tpl` - the Knative service object that should be deployed for a version
15
+
-`_knative-istio.none.tpl` - wrapper for all objects that should be deployed to support the no automated traffic pattern
16
+
-`_knative-istio.none.routemap.tpl` - the routemap definition
17
+
-`_knative.helpers.tpl` - supporting functions
18
+
19
+
An implementation of these is [here](https://github.com/iter8-tools/docs/tree/v0.18.13/samples/knative-abn-extension).
20
+
21
+
Note that many of these additions are the same as in the example for progressive release. Of the above files, only `_knative-istio.tpl` is different.
22
+
23
+
Finally, update `release.yaml` to include `knative-istio` as a valid option:
24
+
25
+
```tpl
26
+
{{- else if eq "knative-istio" .Values.environment }}
27
+
{{- include "env.knative-istio" . }}
28
+
```
29
+
30
+
## Extend the controller
31
+
32
+
The Iter8 controller will need to be restarted with permission to watch Knative service objects. Re-install the controller using the following additional options:
To use the modified chart to run A/B tests for Knative services, see the example for [progressive release](../progressive-release/extension.md#using-the-modified-chart). Unset the `strategy` and `weight` fields.
Copy file name to clipboardExpand all lines: docs/user-guide/install.md
+15-5
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,21 @@
1
-
Iter8 can be installed and configured to watch resources either in a single namespace (namespace scoped) or in the whole cluster (cluster scoped).
1
+
---
2
+
template: main.html
3
+
---
4
+
5
+
# Install options
6
+
7
+
By default, Iter8 uses [BadgerDB](https://dgraph.io/docs/badger/) to store metrics from A/B/n and performance tests. BadgerDB is not suitable for production use. To install for production, use another database. Database options are listed [here](metrics_store.md).
2
8
3
9
## Install with `helm`
4
10
5
-
=== "Namespace scoped"
11
+
Iter8 can be installed and configured to watch resources either in a single namespace (namespace-scoped) or in the whole cluster (cluster-scoped).
Copy file name to clipboardExpand all lines: docs/user-guide/metrics_store.md
+12-5
Original file line number
Diff line number
Diff line change
@@ -4,14 +4,21 @@ template: main.html
4
4
5
5
# Metrics store
6
6
7
-
One of Iter8's key advantages is that it incorporates its own metrics store simplifying the set up and execution of A/B/n and performance tests. Iter8 currently supports the following databases:
7
+
Iter8 provides a metrics store for A/B/n and performance metrics. This simplifies the set up and execution of test scenarios. Iter8 currently supports the following databases:
8
8
9
-
- BadgerDB
10
-
- Redis
9
+
-[BadgerDB](https://dgraph.io/docs/badger/)
10
+
-[Redis](https://redis.com/)
11
11
12
-
Iter8 uses BadgerDB by default. Note, however, that BadgerDB is not suitable for production use and is only suitable for a single instance of Iter8. Support for other databases are in the works. See [below](#contribute-a-new-metrics-store-implementation) for details on how to contribute additional implementations.
12
+
By default, Iter8 uses BadgerDB. Note, however, that BadgerDB is not suitable for production use and is only suitable for a single instance of Iter8. Support for other databases is in the works. See [below](#contribute-a-new-metrics-store-implementation) for details on how you can contribute additional implementations.
13
13
14
-
## Using Redis as the metrics store
14
+
## BadgerDB configuration options
15
+
16
+
BadgerDB is a file system-based database. When installing Iter8, the following options can be configured:
17
+
18
+
-`bagderdb.storage` - The amount of space allocated to BadgerDB (default 50Mi).
19
+
-`badgerdb.storageClassName` - The Kubernetes storage class that should be used. The default (`standard`) may not work for all clusters.
20
+
21
+
## Redis configuration options
15
22
16
23
We assume that Redis is deployed. For example, for a basic deployment:
The Iter8 performance test task [`ready`](tasks/ready.md) ensures that an object exists and is ready. To use this task with new resource types, including CRDs, add the new resource type to the list of known types defined in the default [`values.yaml` file](https://github.com/iter8-tools/iter8/blob/v0.18.3/charts/iter8/values.yaml) for the chart. Alternatively, the new type can be specified at run time with the `--set` option.
8
+
9
+
### Example
10
+
11
+
To include a Knative service as part of a version definition, add the following to the map of `resourceTypes` in the [`values.yaml`](https://github.com/iter8-tools/iter8/blob/v0.18.3/charts/iter8/values.yaml) file used to configure the controller. The addition identifies the Kubernetes group, version, and resource (GVR) and the status condition that should be checked for readiness.
Copy file name to clipboardExpand all lines: docs/user-guide/performance/parameters.md
+13-5
Original file line number
Diff line number
Diff line change
@@ -4,9 +4,9 @@ template: main.html
4
4
5
5
# Performance test parameters
6
6
7
-
Iter8 is built on [Helm](https://helm.sh). Performance tests can be configured with parameters using the same mechanisms provided by Helm for [setting chart values](https://helm.sh/docs/chart_template_guide/values_files/#helm).
7
+
Iter8 performance tests are run using [Helm](https://helm.sh). Tests can be configured with parameters in the same manner as any other Helm command -- using [chart values](https://helm.sh/docs/chart_template_guide/values_files/#helm) and the `--set` option.
8
8
9
-
The set of configurable parameters for a performance test includes the parameters of the tasks involved in the test. Iter8 uses the convention that the parameters of a task are nested under the name of that task. In the following example, the `url` parameter of the `http` task is nested under the `http` object.
9
+
The `tasks` parameter is used to identify the sequence of tasks that should be executed. Each task has its own parameters. Iter8 uses the convention that the parameters of a task are nested under the name of that task. In the following example, the `url` parameter of the `http` task is nested under the `http` object.
10
10
11
11
```shell
12
12
helm upgrade --install \
@@ -15,11 +15,19 @@ helm upgrade --install \
15
15
--set http.url=https://httpbin.org/get
16
16
```
17
17
18
-
All the parameters of the performance test (including of all included tasks) are optional unless otherwise documented.
18
+
All the parameters of the performance test (including of all included tasks) are optional unless otherwise documented. The task-specific parameters are documented in the task documentation.
19
19
20
-
## Parameters
20
+
Currently available tasks are:
21
21
22
-
Global performance test parameters are described here. Task specific parameters are documented in each task description.
22
+
-[`http`](tasks/http.md) - generate synthetic load and capture performance metrics for HTTP endpoints
23
+
-[`grpc`](tasks/grpc.md) - generate synthetic load and capture performance metrics for gRPC methods
24
+
-[`ready`](tasks/ready.md) - check readiness of an object
25
+
-[`github`](tasks/github.md) - send a GitHub notification
26
+
-[`slack`](tasks/slack.md) - send a Slack notification
27
+
28
+
## Global parameters
29
+
30
+
In addition to task-specific parameters, the following global parameters are available:
Copy file name to clipboardExpand all lines: docs/user-guide/progressive-release/about.md
+1-1
Original file line number
Diff line number
Diff line change
@@ -23,5 +23,5 @@ The chart provided by Iter8 supports many common deployment scenarios including:
23
23
24
24
## Other deployment environments
25
25
26
-
The `release` chart can be easily [extended](extending.md) to include other deployment environments. Please consider contributing any extensions to Iter8.
26
+
The `release` chart can be easily [extended](extension.md) to include other deployment environments. Please consider contributing any extensions to Iter8.
Copy file name to clipboardExpand all lines: docs/user-guide/progressive-release/extension.md
+10-4
Original file line number
Diff line number
Diff line change
@@ -4,16 +4,16 @@ template: main.html
4
4
5
5
# Extending progressive release to other deployment environments
6
6
7
-
The `release` chart can be easily extended to include other deployment environments. Please consider contributing any extensions to Iter8. We briefly describe how to extend the chart for an [Knative](https://knative.dev/docs/) application.
7
+
The Iter8 `release` chart can be easily extended for applications/ML models composed of new types. We briefly describe how to extend the chart for an [Knative](https://knative.dev/docs/) application.
8
8
9
9
## Approach
10
10
11
-
The progressive release chart can be found in the `charts/release` sub-folder of the [Iter8 GitHub repository](https://github.com/iter8-tools/iter8). The file `release.yaml` is the starting point. For each valid environment, the chart contains a set of files defining the resources that should be created. These may include:
11
+
The `release` chart can be found in the `charts/release` sub-folder of the [Iter8 GitHub repository](https://github.com/iter8-tools/iter8). The file `release.yaml` is the starting point. For each valid environment, the chart contains a set of files defining the resources that should be created. These may include:
12
12
13
13
- application object(s)
14
14
-[routemaps](../routemap.md) for different traffic patterns
15
15
- configmaps used to specify request percentages
16
-
- service defining a common entry for requests (if needed)
16
+
-a service defining a common entry for requests (if needed)
17
17
18
18
Note that the file naming helps identify related template files.
19
19
@@ -30,6 +30,8 @@ For example, to implement a blue-green release for Knative services, the followi
30
30
31
31
An implementation of these is [here](https://github.com/iter8-tools/docs/tree/v0.18.11/samples/knative-bg-extension).
32
32
33
+
Note that this sample only implements the blue-green traffic pattern. A more complete implementation would include canary and mirroring traffic patterns.
34
+
33
35
Finally, update `release.yaml` to include `knative-istio` as a valid option:
34
36
35
37
```tpl
@@ -39,7 +41,7 @@ Finally, update `release.yaml` to include `knative-istio` as a valid option:
39
41
40
42
## Extend the controller
41
43
42
-
The Iter8 controller will need to be extended to give permission to Iter8 to watch Knative service objects. Configure the deployment of the controller to enable this, (re-)install the controller using the following additional options:
44
+
The Iter8 controller will need to be restarted with permission to watch Knative service objects. Re-install the controller using the following additional options:
0 commit comments