Improve the platform API and add docs (#3)

* Improve the platform API and add docs

Author: ThomasVitale

.github/workflows/release.yml

@@ -18,6 +18,6 @@ jobs:
       registry-server: ghcr.io
       registry-username: ${{ github.actor }}
       image: ${{ github.repository }}
-      version: 0.8.0
+      version: 0.9.0
     secrets:
       pull-request-token: ${{ secrets.GH_ORG_PAT }}

README.md

@@ -74,8 +74,12 @@ Documentation, tutorials and examples for this package are available in the [doc
 The Engineering Platform package can be customized via a `values.yml` file.
 
   ```yaml
-  excluded_blueprints:
-    - "config-template"
+  platform:
+    ingress:
+      domain: thomasvitale.com
+    oci_registry:
+      server: ghcr.io
+      repository: thomasvitale
   ```
 
 Reference the `values.yml` file from the `kctrl` command when installing or upgrading the package.
@@ -96,23 +100,41 @@ The Engineering Platform package has the following configurable properties.
 
 | Config | Default | Description |
 |-------|-------------------|-------------|
-| `packages.namespace` | `""` | The namespace where to install the platform. |
-| `packages.exclusions` | `[]` | A list of packages to exclude from being installed. |
+| `platform.namespace` | `kadras-packages` | The namespace where to install the platform. |
+| `platform.excluded_packages` | `[]` | A list of packages to exclude from being installed. |
+| `platform.ca_cert_data` | `""` | PEM-encoded certificate data to trust TLS connections with a custom CA. |
+| `platform.ingress.domain.issuer.type` | `private` | The type of ClusterIssuer the platform will use to enable TLS communications. Options: `private`, `letsencrypt_staging`, `letsencrypt`, `custom`. |
+| `platform.ingress.domain.issuer.name` | `""` | A reference to a custom ClusterIssuer previously created on the cluster where the platform will be installed. Required when the type is `custom`. |
+| `platform.oci_registry.server` | `""` | The server of the OCI Registry where the platform will publish and consume OCI images. |
+| `platform.oci_registry.repository` | `""` | The repository in the OCI Registry where the platform will publish and consume OCI images. |
+| `platform.oci_registry.credentials.username` | `""` | Username to access the OCI registry. Note: Use `_json_key` for GCR. |
+| `platform.oci_registry.credentials.password` | `""` | Token to access the OCI registry. Note: Use contents of service account key json for GCR. |
+| `platform.oci_registry.secret.name` | `supply-chain-registry-credentials` | The name of the Secret holding the credentials to access the OCI registry. |
+| `platform.oci_registry.secret.namespace` | `kadras-packages` | The namespace of the Secret holding the credentials to access the OCI registry. |
+| `platform.cosign.secret.name` | `supply-chain-cosign-key-pair` | The name of the Secret holding the Cosign key pair. |
+| `platform.cosign.secret.namespace` | `kadras-packages` | The namespace of the Secret holding the Cosign key pair. |
+
+Each Kadras package included in the platform can be configured independently.
+
+| Config | Default | Description |
+|-------|-------------------|-------------|
 | `buildpacks.catalog` | `{}` | Configuration for the Buildpacks Catalog package. |
 | `buildpacks.kpack` | `{}` | Configuration for the Kpack package. |
+| `cartographer.core` | `{}` | Configuration for the Cartographer Core package including Cartographer and Cartographer Conventions. |
 | `cartographer.blueprints` | `{}` | Configuration for the Cartographer Blueprints package. |
 | `cartographer.delivery` | `{}` | Configuration for the Cartographer Delivery package. |
 | `cartographer.supply_chains` | `{}` | Configuration for the Cartographer Supply Chains package. |
-| `cert_manager` | `{}` | Configuration for the Cert Manager package. |
+| `cert_manager.core` | `{}` | Configuration for the Cert Manager package. |
+| `cert_manager.issuers` | `{}` | Configuration for the Cert Manager Issuers package. |
 | `contour` | `{}` | Configuration for the Contour package. |
 | `conventions.spring_boot` | `{}` | Configuration for the Spring Boot Conventions package. |
 | `flux.source_controller` | `{}` | Configuration for the FluxCD Source Controller package. |
 | `knative.serving` | `{}` | Configuration for the Knative Serving package. |
 | `metrics_server` | `{}` | Configuration for the Metrics Server package. |
-| `namespace_setup` | `{}` | Configuration for the Namespace Setup package. |
 | `secretgen_controller` | `{}` | Configuration for the Secretgen Controller package. |
 | `tekton.catalog` | `{}` | Configuration for the Tekton Catalog package. |
 | `tekton.pipelines` | `{}` | Configuration for the Tekton Pipelines package. |
+| `workspace_provisioner` | `{}` | Configuration for the Workspace Provisioner package. |
 
 </details>
 

docs/install.md

@@ -0,0 +1,116 @@
+# Install the Kadras Engineering Platform
+
+## 1. Prerequisites
+
+* Kubernetes 1.24+
+* Carvel [`kctrl`](https://carvel.dev/kapp-controller/docs/latest/install/#installing-kapp-controller-cli-kctrl) CLI.
+* Sigstore [`cosign`](https://docs.sigstore.dev/cosign/installation/) CLI.
+* Carvel [kapp-controller](https://carvel.dev/kapp-controller) deployed in your Kubernetes cluster. You can install it with Carvel [`kapp`](https://carvel.dev/kapp/docs/latest/install) (recommended choice) or `kubectl`.
+
+  ```shell
+  kapp deploy -a kapp-controller -y \
+    -f https://github.com/carvel-dev/kapp-controller/releases/latest/download/release.yml
+  ```
+
+## 2. Add the Kadras Repository
+
+Add the Kadras repository to make all Kadras packages available to the cluster.
+
+  ```shell
+  kubectl create namespace kadras-packages
+  kctrl package repository add -r kadras-packages \
+    --url ghcr.io/kadras-io/kadras-packages \
+    -n kadras-packages
+  ```
+
+You can check the full list of available packages as follows.
+
+  ```shell
+  kctrl package available list -n kadras-packages 
+  ```
+
+## 3. Create Secret for OCI Registry
+
+First, create a Secret with the credentials to access your container registry in read/write mode. It will be used by the platform to publish and consume OCI artifacts.
+
+  ```shell
+  export SUPPLY_CHAIN_REGISTRY_HOSTNAME=<hostname>
+  export SUPPLY_CHAIN_REGISTRY_USERNAME=<username>
+  export SUPPLY_CHAIN_REGISTRY_TOKEN=<token>
+  ```
+
+* `<hostname>` is the server hosting the OCI registry. For example, `ghcr.io`, `gcr.io`, `quay.io`, `index.docker.io`.
+* `<username>` is the username to access the OCI registry. Use `_json_key` if the hostname is `gcr.io`.
+* `<token>` is a token with read/write permissions to access the OCI registry. Use the contents of the service account key json if the hostname is `gcr.io`.
+
+  ```shell
+  kubectl create secret docker-registry supply-chain-registry-credentials \
+  --docker-server="${SUPPLY_CHAIN_REGISTRY_HOSTNAME}" \
+  --docker-username="${SUPPLY_CHAIN_REGISTRY_USERNAME}" \
+  --docker-password="${SUPPLY_CHAIN_REGISTRY_TOKEN}" \
+  --namespace=kadras-packages
+  ```
+
+## 4. Create Secret for Cosign
+
+Next, use Cosign to generate a key-pair that will be used by the platform to sign and verify OCI artifacts.
+
+  ```shell
+  cosign generate-key-pair k8s://kadras-packages/supply-chain-cosign-key-pair
+  ```
+
+The previous command will create a cosign.pub file in the current directory. That's the public key you can use the verify OCI artifacts built and signed by the platform.
+
+## 5. Configure the Platform
+
+The installation of the Kadras Engineering Platform can be configured via YAML. Create a `values.yml` file with any configuration you need for the platform. The following is a minimal configuration example.
+
+```yaml
+platform:
+  ingress:
+    domain: <domain>
+
+  oci_registry:
+    server: <oci-server>
+    repository: <oci-repository>
+
+workspace_provisioner:
+  namespaces:
+    - name: default
+  git:
+    credentials:
+      username: <github-username>
+      password: <github-token>
+```
+
+* `<domain>` is the base domain name the platform will use to configure the Ingress controller. It must be a valid DNS name. For example, `lab.thomasvitale.com`.
+* `<oci-server>` is the server of the OCI registry where the platform will publish and consume OCI images. It must be the same used in step 3 when creating a Secret with the OCI registry credentials. For example, `ghcr.io`, `gcr.io`, `quay.io`, `index.docker.io`.
+* `<oci-repository>` is the repository in the OCI registry where the platform will publish and consume OCI images. It must be the same used in step 3 when creating a Secret with the OCI registry credentials. For example, it might be your username or organization name depending on which OCI server you're using.
+* `<github-username>` is your username to access your Git repositories on GitHub. It's not needed if you won't use the GitOps workflows offered by the platform and only use public Git repositories.
+* `<github-token>` is a token with read/write permissions to access your Git repositories on GitHub. It's not needed if you won't use the GitOps workflows offered by the platform and only use public Git repositories.
+
+## 6. Install the Platform
+
+Reference the `values.yml` file you created in the previous step and install the Kadras Engineering Platform.
+
+  ```shell
+  kctrl package install -i engineering-platform \
+    -p engineering-platform.packages.kadras.io \
+    -v ${VERSION} \
+    -n kadras-packages \
+    --values-file values.yml
+  ```
+
+You can find the `${VERSION}` value by retrieving the list of package versions available in the Kadras package repository installed on your cluster.
+
+  ```shell
+  kctrl package available list -p engineering-platform.packages.kadras.io -n kadras-packages
+  ```
+
+## 7. Verify the Installation
+
+Verify that all the platform components have been installed and properly reconciled.
+
+  ```shell
+  kctrl package installed list -n kadras-packages 
+  ```

docs/verify-release.md

@@ -1,4 +1,4 @@
-# Verifying the Tekton Pipelines Package Release
+# Verifying the Package Release
 
 This package is published as an OCI artifact, signed with Sigstore [Cosign](https://docs.sigstore.dev/cosign/overview), and associated with a [SLSA Provenance](https://slsa.dev/provenance) attestation.
 

package/config/buildpacks-catalog.yml

@@ -1,25 +1,41 @@
 #@ load("@ytt:data", "data")
+#@ load("@ytt:struct", "struct")
 #@ load("@ytt:yaml", "yaml")
 #@ load("/helpers.star", "is_package_enabled")
 
 #@ if is_package_enabled("buildpacks-catalog"):
 
+#@ def build_package_values():
+#@   values = {
+#@     "kp_default_repository": {}
+#@   }
+#@
+#@   if data.values.buildpacks.catalog:
+#@     values.update(struct.decode(data.values.buildpacks.catalog))
+#@   end
+#@   if data.values.platform.oci_registry.server and data.values.platform.oci_registry.repository and (not hasattr(data.values.buildpacks.catalog, "kp_default_repository") or not hasattr(data.values.buildpacks.catalog.kp_default_repository, "name") or not data.values.buildpacks.catalog.kp_default_repository.name):
+#@     values["kp_default_repository"]["name"] = data.values.platform.oci_registry.server.rstrip("/") + "/" + data.values.platform.oci_registry.repository.rstrip("/") + "/buildpacks"
+#@   end
+#@
+#@   return struct.encode(values)
+#@ end
+
 ---
 apiVersion: packaging.carvel.dev/v1alpha1
 kind: PackageInstall
 metadata:
   name: buildpacks-catalog
-  namespace: #@ data.values.packages.namespace
+  namespace: #@ data.values.platform.namespace
   annotations:
     kapp.k14s.io/change-group: buildpacks-catalog
-    kapp.k14s.io/change-rule.buildpacks-catalog: upsert after upserting kpack
+    kapp.k14s.io/change-rule.kpack: upsert after upserting kpack
     kapp.k14s.io/change-rule.service-account: delete before deleting serviceaccount  
 spec:
   serviceAccountName: kadras-install-sa
   packageRef:
     refName: buildpacks-catalog.packages.kadras.io
     versionSelection:
-      constraints: 0.6.0
+      constraints: 0.6.1
       prereleases: {}
   values:
   - secretRef:
@@ -29,8 +45,8 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: buildpacks-catalog-values
-  namespace: #@ data.values.packages.namespace
+  namespace: #@ data.values.platform.namespace
 stringData:
-  values.yaml: #@ yaml.encode(data.values.buildpacks.catalog)
+  values.yaml: #@ yaml.encode(build_package_values())
 
 #@ end

package/config/cartographer-blueprints.yml

@@ -1,15 +1,26 @@
 #@ load("@ytt:data", "data")
+#@ load("@ytt:struct", "struct")
 #@ load("@ytt:yaml", "yaml")
 #@ load("/helpers.star", "is_package_enabled")
 
 #@ if is_package_enabled("cartographer-blueprints"):
 
+#@ def build_package_values():
+#@   values = {}
+#@
+#@   if data.values.cartographer.blueprints:
+#@     values.update(struct.decode(data.values.cartographer.blueprints))
+#@   end
+#@
+#@   return struct.encode(values)
+#@ end
+
 ---
 apiVersion: packaging.carvel.dev/v1alpha1
 kind: PackageInstall
 metadata:
   name: cartographer-blueprints
-  namespace: #@ data.values.packages.namespace
+  namespace: #@ data.values.platform.namespace
   annotations:
     kapp.k14s.io/change-group: cartographer-blueprints
     kapp.k14s.io/change-rule.cartographer: upsert after upserting cartographer
@@ -21,7 +32,7 @@ spec:
   packageRef:
     refName: cartographer-blueprints.packages.kadras.io
     versionSelection:
-      constraints: 0.5.0
+      constraints: 0.5.1
       prereleases: {}
   values:
   - secretRef:
@@ -31,8 +42,8 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: cartographer-blueprints-values
-  namespace: #@ data.values.packages.namespace
+  namespace: #@ data.values.platform.namespace
 stringData:
-  values.yaml: #@ yaml.encode(data.values.cartographer.blueprints)
+  values.yaml: #@ yaml.encode(build_package_values())
 
 #@ end

package/config/cartographer-delivery.yml

@@ -1,15 +1,26 @@
 #@ load("@ytt:data", "data")
+#@ load("@ytt:struct", "struct")
 #@ load("@ytt:yaml", "yaml")
 #@ load("/helpers.star", "is_package_enabled")
 
 #@ if is_package_enabled("cartographer-delivery"):
 
+#@ def build_package_values():
+#@   values = {}
+#@
+#@   if data.values.cartographer.delivery:
+#@     values.update(struct.decode(data.values.cartographer.delivery))
+#@   end
+#@
+#@   return struct.encode(values)
+#@ end
+
 ---
 apiVersion: packaging.carvel.dev/v1alpha1
 kind: PackageInstall
 metadata:
   name: cartographer-delivery
-  namespace: #@ data.values.packages.namespace
+  namespace: #@ data.values.platform.namespace
   annotations:
     kapp.k14s.io/change-group: cartographer-delivery
     kapp.k14s.io/change-rule.cartographer: upsert after upserting cartographer
@@ -20,7 +31,7 @@ spec:
   packageRef:
     refName: cartographer-delivery.packages.kadras.io
     versionSelection:
-      constraints: 0.4.0
+      constraints: 0.4.1
       prereleases: {}
   values:
   - secretRef:
@@ -30,8 +41,8 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: cartographer-delivery-values
-  namespace: #@ data.values.packages.namespace
+  namespace: #@ data.values.platform.namespace
 stringData:
-  values.yaml: #@ yaml.encode(data.values.cartographer.delivery)
+  values.yaml: #@ yaml.encode(build_package_values())
 
 #@ end

package/config/cartographer-supply-chains.yml

@@ -1,15 +1,32 @@
 #@ load("@ytt:data", "data")
+#@ load("@ytt:struct", "struct")
 #@ load("@ytt:yaml", "yaml")
 #@ load("/helpers.star", "is_package_enabled")
 
 #@ if is_package_enabled("cartographer-supply-chains"):
 
+#@ def build_package_values():
+#@   values = {
+#@     "registry": {}
+#@   }
+#@
+#@   if data.values.cartographer.supply_chains:
+#@     values.update(struct.decode(data.values.cartographer.supply_chains))
+#@   end
+#@   if data.values.platform.oci_registry.server and data.values.platform.oci_registry.repository and (not hasattr(data.values.cartographer.supply_chains, "registry") or not hasattr(data.values.cartographer.supply_chains.registry, "server") or not hasattr(data.values.cartographer.supply_chains.registry, "repository") or not data.values.cartographer.supply_chains.registry.server or not data.values.cartographer.supply_chains.registry.repository):
+#@     values["registry"]["server"] = data.values.platform.oci_registry.server.rstrip("/")
+#@     values["registry"]["repository"] = data.values.platform.oci_registry.repository.rstrip("/") + "/workloads"
+#@   end
+#@
+#@   return struct.encode(values)
+#@ end
+
 ---
 apiVersion: packaging.carvel.dev/v1alpha1
 kind: PackageInstall
 metadata:
   name: cartographer-supply-chains
-  namespace: #@ data.values.packages.namespace
+  namespace: #@ data.values.platform.namespace
   annotations:
     kapp.k14s.io/change-group: cartographer-supply-chains
     kapp.k14s.io/change-rule.cartographer: upsert after upserting cartographer
@@ -20,7 +37,7 @@ spec:
   packageRef:
     refName: cartographer-supply-chains.packages.kadras.io
     versionSelection:
-      constraints: 0.5.0
+      constraints: 0.5.1
       prereleases: {}
   values:
   - secretRef:
@@ -30,8 +47,8 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: cartographer-supply-chains-values
-  namespace: #@ data.values.packages.namespace
+  namespace: #@ data.values.platform.namespace
 stringData:
-  values.yaml: #@ yaml.encode(data.values.cartographer.supply_chains)
+  values.yaml: #@ yaml.encode(build_package_values())
 
 #@ end