Commit f6ea8bc
authored
🤖 feat: add local dev workflow, envtest integration tests, and Kind e2e CI (#8)
## Summary
Add complete local cluster development workflow, envtest integration
testing, and CI coverage for the `coder-k8s` operator.
## Background
The repo had kubebuilder markers in API types and RBAC markers in the
controller, but no CRD YAML manifests, no integration tests, no local
dev workflow documentation, and no CI envtest or e2e coverage. This PR
addresses all of those gaps.
## Implementation
### Phase 1: Manifest generation pipeline
- Added `sigs.k8s.io/controller-tools v0.20.0` as a vendored dependency
(via `internal/deps/controllergen_tools.go` build-tagged file)
- Added `hack/update-manifests.sh` script with fail-fast assertions
(mirrors existing `hack/update-codegen.sh` pattern)
- Added `make manifests` target
- Generated and committed CRD (`config/crd/bases/`), RBAC
(`config/rbac/`), and sample CR (`config/samples/`)
### Phase 2: envtest integration tests
- Added `internal/controller/suite_test.go` — envtest `TestMain`
lifecycle harness with scheme registration and defensive assertions
- Added `internal/controller/codercontrolplane_controller_test.go` — 4
integration test cases:
- `TestReconcile_NotFound` — reconcile non-existent CR returns nil
- `TestReconcile_ExistingResource` — reconcile existing CR returns nil
- `TestReconcile_NilClient` — assertion error on nil client
- `TestReconcile_NilScheme` — assertion error on nil scheme
- Added `make setup-envtest` and `make test-integration` targets
- Updated `make test` to automatically set up envtest assets
### Phase 3: CI updates
- Updated `test` job to install envtest assets and export
`KUBEBUILDER_ASSETS`
- Added `e2e-kind` job: creates Kind cluster, builds/loads image,
deploys controller with CRDs/RBAC/e2e manifests, applies sample CR, and
verifies resource exists
- All actions SHA-pinned (matching repo convention)
### Documentation
- Expanded `README.md` with: project description, prerequisites,
OrbStack quick start, essential commands, testing strategy, and project
structure sections
## Risks
- **Low**: `setup-envtest@release-0.23` uses a branch ref rather than a
pinned commit. This tracks the controller-runtime v0.23.x release
branch. Could be pinned to a pseudo-version if strict immutability is
desired.
- **Low**: The `e2e-kind` job assumes `jq` is present on `ubuntu-latest`
(currently true).
---
<details>
<summary>📋 Implementation Plan</summary>
# Implementation Plan: Local cluster development + integration/e2e
testing for `coder-k8s`
## Context / Why
You want a concrete execution plan an agent can follow to: (1) develop
quickly against a local Kubernetes cluster, (2) add reliable integration
testing, and (3) add CI coverage in GitHub Actions. The current
repository is scaffold-level, so the plan prioritizes a fast inner loop
first (OrbStack), then deterministic tests (`envtest`), then a
real-cluster smoke test (Kind in CI).
This plan explicitly addresses your CRD note: **the API types/markers
exist, but generated CRD YAML manifests are not currently present in the
repo**.
## Evidence
- `api/v1alpha1/codercontrolplane_types.go` contains kubebuilder markers
(`+kubebuilder:resource`, `+kubebuilder:subresource:status`) and CRD
types.
- `hack/update-codegen.sh` only runs `deepcopy-gen`; no CRD/manifest
generation.
- `Makefile` has `test/build/lint/vuln/codegen` but no `manifests`,
`install`, or integration/e2e targets.
- `.github/workflows/ci.yaml` runs lint/unit/build checks only; no
envtest/Kind job.
- `main.go` uses `ctrl.GetConfigOrDie()`, so local out-of-cluster runs
depend on kubeconfig context.
- `main_test.go` currently contains unit-style checks only.
- Repo root currently has no `config/crd` or sample manifests (YAML
files are only workflows/tooling YAML plus vendor files).
These files are sufficient to design a full implementation path.
## Implementation details
### Planned edit map (ordered)
1. `internal/deps/deps.go` — import block: add blank import for
`sigs.k8s.io/controller-tools/cmd/controller-gen` (vendoring anchor).
2. `hack/update-manifests.sh` (new) — script entrypoint + fail-fast
assertions + `controller-gen` invocation.
3. `Makefile` — `.PHONY` list + new targets: `manifests`,
`setup-envtest`, `test-integration`.
4. `config/crd/bases/*`, `config/rbac/*`, `config/samples/*`
(new/generated) — committed generated manifests + sample CR.
5. `README.md` — new “Local development (OrbStack)” and “Testing
strategy” sections.
6. `internal/controller/suite_test.go` (new) — `TestMain` envtest
lifecycle harness.
7. `internal/controller/codercontrolplane_controller_test.go` (new) —
integration tests around `Reconcile` assertions and not-found/exists
flows.
8. `.github/workflows/ci.yaml` — `test` job envtest setup; new
`e2e-kind` job.
9. `test/e2e/*` or `config/e2e/*` (new) — minimal Kind smoke
deployment/test assets.
### Phase 1 — OrbStack-first local development workflow (recommended
first)
1. **Add manifest generation pipeline (CRD/RBAC) and commit generated
outputs.**
- Edit `internal/deps/deps.go` to keep `controller-tools` vendored, then
run `go mod tidy && go mod vendor`.
- Add `hack/update-manifests.sh` with fail-fast assertions and
deterministic output directories.
- Add `Makefile` target `manifests` and wire it into local docs/CI
checks where appropriate.
- Create and commit:
- `config/crd/bases/coder.com_codercontrolplanes.yaml`
- `config/rbac/role.yaml` (and related generated RBAC if produced)
- `config/samples/coder_v1alpha1_codercontrolplane.yaml`
```bash
# hack/update-manifests.sh (shape)
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
[[ -d "${SCRIPT_ROOT}/api/v1alpha1" ]] || {
echo "assertion failed: expected API package at
${SCRIPT_ROOT}/api/v1alpha1" >&2
exit 1
}
cd "${SCRIPT_ROOT}"
GOFLAGS=-mod=vendor go run
./vendor/sigs.k8s.io/controller-tools/cmd/controller-gen \
crd:crdVersions=v1 \
rbac:roleName=manager-role \
paths=./... \
output:crd:artifacts:config=config/crd/bases \
output:rbac:artifacts:config=config/rbac
```
2. **Add a simple OrbStack local-dev runbook in `README.md`.**
- Include exact commands for applying CRDs, running controller, applying
sample CR, and checking logs/events.
- Keep this as the default dev loop; no Kind required for normal
iteration.
```bash
make manifests
kubectl apply -f config/crd/bases/
GOFLAGS=-mod=vendor go run .
kubectl apply -f config/samples/coder_v1alpha1_codercontrolplane.yaml
kubectl get codercontrolplanes -A
```
3. **Acceptance criteria (Phase 1).**
- Fresh clone can generate/apply CRDs and create a sample
`CoderControlPlane`.
- Controller runs locally against OrbStack context without extra cluster
tooling.
---
### Phase 2 — Add `envtest` integration tests (highest-value test
investment)
1. **Create envtest suite harness for controller package.**
- Add `internal/controller/suite_test.go` that starts/stops
`envtest.Environment`.
- Register `client-go` and `coderv1alpha1` schemes and construct a
controller-runtime client.
- Keep defensive assertions (nil checks + explicit assertion-failed
text) in helpers.
```go
// internal/controller/suite_test.go (shape)
var (
testEnv *envtest.Environment
cfg *rest.Config
k8sClient client.Client
)
func TestMain(m *testing.M) {
testEnv = &envtest.Environment{CRDDirectoryPaths:
[]string{"../../config/crd/bases"}}
var err error
cfg, err = testEnv.Start()
if err != nil { panic(fmt.Errorf("assertion failed: envtest start: %w",
err)) }
// build scheme + client
code := m.Run()
if stopErr := testEnv.Stop(); stopErr != nil { panic(stopErr) }
os.Exit(code)
}
```
2. **Add reconciliation integration tests in
`internal/controller/codercontrolplane_controller_test.go`.**
- Cases to add now:
- Reconcile returns nil when CR exists.
- Reconcile returns nil on NotFound.
- Reconcile returns assertion error when `Client` is nil.
- Reconcile returns assertion error when `Scheme` is nil.
- Keep tests minimal and deterministic while reconciler is still
placeholder logic.
3. **Add explicit Make targets for integration tests + envtest binary
setup.**
- Add targets like:
- `make setup-envtest`
- `make test-integration`
- Ensure targets fail fast if envtest assets are missing.
4. **Acceptance criteria (Phase 2).**
- `make test` (or split unit/integration pair) passes locally with
vendoring.
- Integration tests run without requiring a running Kubernetes cluster.
---
### Phase 3 — CI updates (GitHub Actions)
1. **Extend existing `test` job in `.github/workflows/ci.yaml` to run
envtest-backed tests.**
- Install envtest binaries in CI.
- Export `KUBEBUILDER_ASSETS`.
- Run `go test ./...` with `GOFLAGS=-mod=vendor`.
```yaml
- name: Setup envtest assets
run: |
go run sigs.k8s.io/controller-runtime/tools/setup-envtest@v0.23.1 use
1.35.x --bin-dir ./bin -p path > /tmp/kubebuilder_assets_path
echo "KUBEBUILDER_ASSETS=$(cat /tmp/kubebuilder_assets_path)" >>
$GITHUB_ENV
- name: Run tests
env:
GOFLAGS: -mod=vendor
KUBEBUILDER_ASSETS: ${{ env.KUBEBUILDER_ASSETS }}
run: go test ./... -count=1
```
2. **Add separate Kind-based smoke e2e job (new job, not replacement).**
- Add `e2e-kind` job after `test`.
- Use `helm/kind-action` (or equivalent pinned action) to create
ephemeral cluster.
- Build image, load into Kind, apply CRDs, deploy controller manifest,
apply sample CR.
- Assert smoke outcome (resource exists; controller pod healthy; no
crashloop).
```yaml
e2e-kind:
runs-on: ubuntu-latest
needs: [test]
steps:
- uses: actions/checkout@...
- uses: helm/kind-action@...
- run: make manifests
- run: kubectl apply -f config/crd/bases/
- run: docker build -f Dockerfile.goreleaser -t coder-k8s:e2e .
- run: kind load docker-image coder-k8s:e2e
- run: kubectl apply -f config/e2e/ # deployment + RBAC + sample
- run: kubectl wait --for=condition=Available deploy/coder-k8s -n
coder-system --timeout=120s
```
3. **Acceptance criteria (Phase 3).**
- PR CI covers lint/unit/integration plus a real-cluster smoke job.
- Failures clearly indicate whether logic broke (`test`) or cluster
packaging/deploy broke (`e2e-kind`).
---
### Phase 4 — Optional DevSpace/Tilt layer (defer unless team asks)
1. **Decision gate:** only implement if multiple developers need
standardized hot-reload/dev deployment workflows.
2. If enabled, add `devspace.yaml` (or `Tiltfile`) that wraps the
**same** manifests and image build path used in CI.
3. Keep OrbStack path as baseline fallback and document both flows
succinctly.
<details>
<summary>Why deferred</summary>
This repo currently has a minimal reconciler and no complex dependent
resources yet. Adding DevSpace/Tilt immediately would increase
maintenance surface before there is enough deployment complexity to
justify it.
</details>
## Execution order (for agent handoff)
1. Phase 1 manifests + OrbStack docs.
2. Phase 2 envtest harness/tests.
3. Phase 3 CI envtest + Kind smoke job.
4. Phase 4 optional tooling only if requested.
## Validation checklist the implementing agent must run
- `make verify-vendor`
- `make manifests` (new target)
- `make test`
- `make build`
- If workflows changed: `go run
github.com/rhysd/actionlint/cmd/actionlint@v1.7.10`
- Optional local smoke: run controller against OrbStack and apply sample
CR.
</details>
---
_Generated with `mux` • Model: `anthropic:claude-opus-4-6` • Thinking:
`xhigh` • Cost: `$0.68`_
<!-- mux-attribution: model=anthropic:claude-opus-4-6 thinking=xhigh
costs=0.68 -->1 parent d7941ba commit f6ea8bc
382 files changed
Lines changed: 68749 additions & 729 deletions
File tree
- .github/workflows
- config
- crd/bases
- e2e
- rbac
- samples
- hack
- internal
- controller
- deps
- vendor
- github.com
- fatih/color
- gobuffalo/flect
- inconshreveable/mousetrap
- mattn
- go-colorable
- go-isatty
- spf13
- afero
- internal/common
- mem
- cobra
- pflag
- golang.org/x
- mod
- module
- semver
- net/http2
- sync/errgroup
- sys/unix
- term
- text/runes
- time/rate
- tools
- go
- ast
- astutil
- inspector
- packages
- types
- objectpath
- typeutil
- imports
- internal
- event
- core
- label
- gcimporter
- imports
- modindex
- stdlib
- typeparams
- typesinternal
- versions
- google.golang.org/protobuf
- internal
- encoding
- tag
- text
- filedesc
- genid
- impl
- version
- proto
- types
- descriptorpb
- known/timestamppb
- gopkg.in/yaml.v2
- k8s.io
- apiextensions-apiserver/pkg
- apis/apiextensions/v1beta1
- client
- applyconfiguration/apiextensions
- v1beta1
- v1
- clientset/clientset
- scheme
- typed/apiextensions
- v1beta1
- v1
- client-go/util/retry
- kube-openapi/pkg/validation/spec
- utils/buffer
- sigs.k8s.io
- controller-runtime
- pkg
- envtest
- internal
- flock
- testing
- addr
- certs
- controlplane
- process
- tools/setup-envtest
- env
- remote
- store
- versions
- version
- workflows
- controller-tools
- cmd/controller-gen
- pkg
- applyconfiguration
- crd
- markers
- deepcopy
- genall
- help
- pretty
- internal/crd
- loader
- markers
- rbac
- schemapatcher
- internal/yaml
- version
- webhook
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
111 | 118 | | |
112 | 119 | | |
113 | 120 | | |
114 | | - | |
| 121 | + | |
| 122 | + | |
115 | 123 | | |
116 | 124 | | |
117 | 125 | | |
118 | 126 | | |
119 | 127 | | |
120 | 128 | | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
121 | 189 | | |
122 | 190 | | |
123 | 191 | | |
| |||
167 | 235 | | |
168 | 236 | | |
169 | 237 | | |
170 | | - | |
| 238 | + | |
171 | 239 | | |
172 | 240 | | |
173 | 241 | | |
174 | 242 | | |
175 | 243 | | |
176 | 244 | | |
177 | 245 | | |
178 | | - | |
| 246 | + | |
| 247 | + | |
179 | 248 | | |
180 | 249 | | |
181 | 250 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
| 5 | + | |
4 | 6 | | |
5 | | - | |
| 7 | + | |
6 | 8 | | |
7 | 9 | | |
8 | 10 | | |
| |||
12 | 14 | | |
13 | 15 | | |
14 | 16 | | |
15 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
16 | 22 | | |
17 | 23 | | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
18 | 28 | | |
19 | 29 | | |
20 | 30 | | |
| |||
32 | 42 | | |
33 | 43 | | |
34 | 44 | | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
35 | 48 | | |
36 | 49 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
0 commit comments